digital-asset/daml
1
1
Fork 0
mirror of https://github.com/digital-asset/daml.git synced 2024-09-20 17:28:46 +03:00
Code Issues Projects Releases Wiki Activity
85998f8f06
daml/ledger/ledger-on-sql/BUILD.bazel

254 lines
8.4 KiB
Python
Raw Normal View History

A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
# Copyright (c) 2020 The DAML Authors. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
load(
"//bazel_tools:scala.bzl",
"da_scala_binary",
"da_scala_library",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"da_scala_test_suite",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
)
load("//ledger/ledger-api-test-tool:conformance.bzl", "conformance_test")
supported_databases = [
{
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
"name": "h2-memory",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"runtime_deps": [
"@maven//:com_h2database_h2",
],
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
"conformance_test_tags": [
"manual",
],
"conformance_test_server_args": [
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
"--jdbc-url=jdbc:h2:mem:daml-on-sql-conformance-test",
],
"conformance_test_tool_args": [
"--concurrent-test-runs=2",
"--timeout-scale-factor=4",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
],
},
{
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
"name": "h2-file",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"runtime_deps": [
"@maven//:com_h2database_h2",
],
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
"conformance_test_tags": [
"manual",
],
"conformance_test_server_main": "com.daml.ledger.on.sql.MainWithEphemeralDirectory",
"conformance_test_server_args": [
"--jdbc-url=jdbc:h2:%DIR/test",
],
"conformance_test_tool_args": [
"--concurrent-test-runs=2",
"--timeout-scale-factor=4",
],
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
},
{
"name": "postgresql",
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
"runtime_deps": [
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"@maven//:org_postgresql_postgresql",
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
],
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"conformance_test_server_main": "com.daml.ledger.on.sql.MainWithEphemeralPostgresql",
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
},
{
"name": "sqlite-memory",
"runtime_deps": [
"@maven//:org_xerial_sqlite_jdbc",
],
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"conformance_test_server_args": [
ledger-on-sql: Don't allow unnamed in-memory databases. (#4255) * ledger-on-sql: Don't allow unnamed in-memory databases. It seems that changing the connection pool size at runtime causes weird issues, predominantly that sometimes all connections are dropped and therefore the in-memory database is lost after migration. Obviously, it then stops working. This works around this problem by simply not allowing unnamed in-memory databases, guaranteeing that we can share the database between the various connection pools. This will probably be necessary anyway when attempting to share the database between the ledger and the index. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it clear we're generating a new JDBC URL in tests. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-29 13:34:00 +03:00
"--jdbc-url=jdbc:sqlite:file:daml-on-sql-conformance-test?mode=memory&cache=shared",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
],
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
},
{
"name": "sqlite-file",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"runtime_deps": [
"@maven//:org_xerial_sqlite_jdbc",
],
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
"conformance_test_server_main": "com.daml.ledger.on.sql.MainWithEphemeralDirectory",
"conformance_test_server_args": [
"--jdbc-url=jdbc:sqlite:%DIR/test.sqlite",
],
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
},
]
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
all_database_runtime_deps = {dep: None for db in supported_databases for dep in db["runtime_deps"]}.keys()
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
da_scala_library(
name = "ledger-on-sql",
srcs = glob(["src/main/scala/**/*.scala"]),
resources = glob(["src/main/resources/**/*"]),
tags = ["maven_coordinates=com.daml.ledger:on-sql:__VERSION__"],
visibility = [
"//visibility:public",
],
runtime_deps = [
"@maven//:ch_qos_logback_logback_classic",
"@maven//:ch_qos_logback_logback_core",
"@maven//:com_h2database_h2",
],
deps = [
"//daml-lf/data",
Expand `Runner`'s flexibility as a basis for participant servers (#4669) * Add overridable indexer, api and auth configuration to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable indexer and api metrics creation to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable api's `TimeServiceBackend` to `LedgerFactory` * :art: Fix formatting * Port SDK ledgers based on `Runner` (and the sandbox) to `TimeServiceBackend` * Revert to `TimeProvider` for committer usage and to `None` default for API server. Also removed now unused `TimeServiceProvider.wallClock()`. * Move TimeServiceBackend back to the API server. * :art: Remove unneeded argument passed for parameter w/default * Restore sandbox ledger time support * Simplify passing a `TimeProvider` to the sandbox ledger Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-26 11:31:26 +03:00
"//language-support/scala/bindings",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
"//ledger/ledger-api-common",
Sandbox & kvutils: Use the existing ledger ID if no ledger ID is specified. (#4607) * sandbox: If the ledger ID isn't provided, use the one in the database. Previously, we would fail if working against an existing ledger, and not explicitly providing the ledger ID. This was the case even if the ledger ID was randomly generated initially. CHANGELOG_BEGIN - [Sandbox] If no ledger ID is provided when running against an existing ledger, use the existing ID. Previously, Sandbox would fail to start. CHANGELOG_END * sandbox: The ReadOnlySqlLedger should always receive a ledger ID. It's read-only; it can't create one. * sandbox: Stop using `equal` in SqlLedgerSpec. * sandbox: Test that the ledger ID is as specified in SqlLedgerSpec. * sandbox: Let the top-level runner handle a ledger ID mismatch. And clean up the log text. * sandbox: Initialize the ledger properly when the ID is dynamic. * sandbox: Use `Vector`, not `List`, for SqlLedger initialization. Append with Vector, good. List, bad. * ledger-api-common: Make `LedgerApiMode.Dynamic` an object. And add Java-style static factory methods. * kvutils/app | ledger-on-{memory,sql}: Make `ledgerId` optional. It should be generated or retrieved from the persistence layer by the ledger itself. * kvutils: Make the ledger ID optional in the tests. * ledger-on-sql: Store the ledger ID, and reject conflicting IDs. * ledger-on-sql: Make more things final. * ledger-on-sql: Document the `ledger_meta.table_key` column better. * sandbox: Don't hardcode the number of packages in the test DAR. It changes. * ledger-on-sql: Merge the `head` resource owner with the `dispatcher`. * sandbox: Use backticks to simplify pattern match in ReadOnlySqlLedger. * ledger-on-sql: Extract methods in `owner`.
2020-02-20 13:35:16 +03:00
"//ledger/ledger-api-domain",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
"//ledger/ledger-api-health",
"//ledger/participant-state",
"//ledger/participant-state/kvutils",
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
"//libs-scala/contextualized-logging",
kvutils: Use resources in KeyValueParticipantState, not AutoClo… (#4177) * kvutils: Use resources in KeyValueParticipantState, not AutoCloseable. CHANGELOG_BEGIN - [kvutils] The simplified API now uses ``com.digitalasset.resources`` to manage acquiring and releasing resources instead of ``Closeable``. CHANGELOG_END * ledger-on-posix-filesystem: Remove dead code due to a refactoring fail.
2020-01-23 18:59:01 +03:00
"//libs-scala/resources",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
"@maven//:com_google_protobuf_protobuf_java",
"@maven//:com_typesafe_akka_akka_actor_2_12",
"@maven//:com_typesafe_akka_akka_stream_2_12",
"@maven//:com_typesafe_play_anorm_2_12",
"@maven//:com_typesafe_play_anorm_tokenizer_2_12",
"@maven//:com_zaxxer_HikariCP",
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
"@maven//:org_flywaydb_flyway_core",
Sandbox & kvutils: Use the existing ledger ID if no ledger ID is specified. (#4607) * sandbox: If the ledger ID isn't provided, use the one in the database. Previously, we would fail if working against an existing ledger, and not explicitly providing the ledger ID. This was the case even if the ledger ID was randomly generated initially. CHANGELOG_BEGIN - [Sandbox] If no ledger ID is provided when running against an existing ledger, use the existing ID. Previously, Sandbox would fail to start. CHANGELOG_END * sandbox: The ReadOnlySqlLedger should always receive a ledger ID. It's read-only; it can't create one. * sandbox: Stop using `equal` in SqlLedgerSpec. * sandbox: Test that the ledger ID is as specified in SqlLedgerSpec. * sandbox: Let the top-level runner handle a ledger ID mismatch. And clean up the log text. * sandbox: Initialize the ledger properly when the ID is dynamic. * sandbox: Use `Vector`, not `List`, for SqlLedger initialization. Append with Vector, good. List, bad. * ledger-api-common: Make `LedgerApiMode.Dynamic` an object. And add Java-style static factory methods. * kvutils/app | ledger-on-{memory,sql}: Make `ledgerId` optional. It should be generated or retrieved from the persistence layer by the ledger itself. * kvutils: Make the ledger ID optional in the tests. * ledger-on-sql: Store the ledger ID, and reject conflicting IDs. * ledger-on-sql: Make more things final. * ledger-on-sql: Document the `ledger_meta.table_key` column better. * sandbox: Don't hardcode the number of packages in the test DAR. It changes. * ledger-on-sql: Merge the `head` resource owner with the `dispatcher`. * sandbox: Use backticks to simplify pattern match in ReadOnlySqlLedger. * ledger-on-sql: Extract methods in `owner`.
2020-02-20 13:35:16 +03:00
"@maven//:org_scalaz_scalaz_core_2_12",
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
],
)
Sandbox: Preliminary work to rebuild Sandbox on top of kvutils. (#4422) * resources: Allow for startup exceptions without no stack trace. * sandbox: Remove dependency duplication in the BUILD file. * ledger-on-(memory|sql): Extract `Main` into its own file tree. This is so we can remove the dependency on kvutils/app from the library. logback.xml goes with it. * sandbox: Implement "Sandbox Next" on top of ledger-on-sql. CHANGELOG_BEGIN - [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML Ledger Integration Kit. Currently not exposed through the CLI. CHANGELOG_END * ledger-on-sql: Publish to Maven Central. * sandbox: If an unknown database is passed to Sandbox Next, handle it. * resources: Catch errors in owner construction or acquisition. Sometimes we're not in a future yet. * resources: Don't run ProgramResource error handling inside the EC. Otherwise we end up in deadlock which is only broken by `awaitTermination` timing out.
2020-02-06 20:34:17 +03:00
da_scala_library(
name = "ledger-on-sql-app",
srcs = glob(["src/app/scala/**/*.scala"]),
resources = glob(["src/app/resources/**/*"]),
visibility = ["//visibility:public"],
deps = [
":ledger-on-sql",
"//daml-lf/data",
Expand `Runner`'s flexibility as a basis for participant servers (#4669) * Add overridable indexer, api and auth configuration to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable indexer and api metrics creation to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable api's `TimeServiceBackend` to `LedgerFactory` * :art: Fix formatting * Port SDK ledgers based on `Runner` (and the sandbox) to `TimeServiceBackend` * Revert to `TimeProvider` for committer usage and to `None` default for API server. Also removed now unused `TimeServiceProvider.wallClock()`. * Move TimeServiceBackend back to the API server. * :art: Remove unneeded argument passed for parameter w/default * Restore sandbox ledger time support * Simplify passing a `TimeProvider` to the sandbox ledger Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-26 11:31:26 +03:00
"//language-support/scala/bindings",
"//ledger/ledger-api-auth",
Sandbox: Preliminary work to rebuild Sandbox on top of kvutils. (#4422) * resources: Allow for startup exceptions without no stack trace. * sandbox: Remove dependency duplication in the BUILD file. * ledger-on-(memory|sql): Extract `Main` into its own file tree. This is so we can remove the dependency on kvutils/app from the library. logback.xml goes with it. * sandbox: Implement "Sandbox Next" on top of ledger-on-sql. CHANGELOG_BEGIN - [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML Ledger Integration Kit. Currently not exposed through the CLI. CHANGELOG_END * ledger-on-sql: Publish to Maven Central. * sandbox: If an unknown database is passed to Sandbox Next, handle it. * resources: Catch errors in owner construction or acquisition. Sometimes we're not in a future yet. * resources: Don't run ProgramResource error handling inside the EC. Otherwise we end up in deadlock which is only broken by `awaitTermination` timing out.
2020-02-06 20:34:17 +03:00
"//ledger/ledger-api-health",
"//ledger/participant-state",
"//ledger/participant-state/kvutils",
"//ledger/participant-state/kvutils/app",
Expand `Runner`'s flexibility as a basis for participant servers (#4669) * Add overridable indexer, api and auth configuration to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable indexer and api metrics creation to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable api's `TimeServiceBackend` to `LedgerFactory` * :art: Fix formatting * Port SDK ledgers based on `Runner` (and the sandbox) to `TimeServiceBackend` * Revert to `TimeProvider` for committer usage and to `None` default for API server. Also removed now unused `TimeServiceProvider.wallClock()`. * Move TimeServiceBackend back to the API server. * :art: Remove unneeded argument passed for parameter w/default * Restore sandbox ledger time support * Simplify passing a `TimeProvider` to the sandbox ledger Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-26 11:31:26 +03:00
"//ledger/sandbox",
Sandbox: Preliminary work to rebuild Sandbox on top of kvutils. (#4422) * resources: Allow for startup exceptions without no stack trace. * sandbox: Remove dependency duplication in the BUILD file. * ledger-on-(memory|sql): Extract `Main` into its own file tree. This is so we can remove the dependency on kvutils/app from the library. logback.xml goes with it. * sandbox: Implement "Sandbox Next" on top of ledger-on-sql. CHANGELOG_BEGIN - [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML Ledger Integration Kit. Currently not exposed through the CLI. CHANGELOG_END * ledger-on-sql: Publish to Maven Central. * sandbox: If an unknown database is passed to Sandbox Next, handle it. * resources: Catch errors in owner construction or acquisition. Sometimes we're not in a future yet. * resources: Don't run ProgramResource error handling inside the EC. Otherwise we end up in deadlock which is only broken by `awaitTermination` timing out.
2020-02-06 20:34:17 +03:00
"//libs-scala/contextualized-logging",
libs-scala/ports: Wrap socket ports in a type, `Port`. (#4784) * libs-scala/ports: Wrap socket ports in a type, `Port`. * sandbox: Use `Port` for the API server port, and propagate. CHANGELOG_BEGIN CHANGELOG_END * extractor: Use `Port` for the server port. * ports: Make Port a compile-time class only. * ports: Allow port 0; it can be specified by a user. * ports: Publish to Maven Central.
2020-03-03 11:59:15 +03:00
"//libs-scala/ports",
Sandbox: Preliminary work to rebuild Sandbox on top of kvutils. (#4422) * resources: Allow for startup exceptions without no stack trace. * sandbox: Remove dependency duplication in the BUILD file. * ledger-on-(memory|sql): Extract `Main` into its own file tree. This is so we can remove the dependency on kvutils/app from the library. logback.xml goes with it. * sandbox: Implement "Sandbox Next" on top of ledger-on-sql. CHANGELOG_BEGIN - [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML Ledger Integration Kit. Currently not exposed through the CLI. CHANGELOG_END * ledger-on-sql: Publish to Maven Central. * sandbox: If an unknown database is passed to Sandbox Next, handle it. * resources: Catch errors in owner construction or acquisition. Sometimes we're not in a future yet. * resources: Don't run ProgramResource error handling inside the EC. Otherwise we end up in deadlock which is only broken by `awaitTermination` timing out.
2020-02-06 20:34:17 +03:00
"//libs-scala/resources",
"@maven//:com_github_scopt_scopt_2_12",
"@maven//:com_typesafe_akka_akka_actor_2_12",
"@maven//:com_typesafe_akka_akka_stream_2_12",
Expand `Runner`'s flexibility as a basis for participant servers (#4669) * Add overridable indexer, api and auth configuration to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable indexer and api metrics creation to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable api's `TimeServiceBackend` to `LedgerFactory` * :art: Fix formatting * Port SDK ledgers based on `Runner` (and the sandbox) to `TimeServiceBackend` * Revert to `TimeProvider` for committer usage and to `None` default for API server. Also removed now unused `TimeServiceProvider.wallClock()`. * Move TimeServiceBackend back to the API server. * :art: Remove unneeded argument passed for parameter w/default * Restore sandbox ledger time support * Simplify passing a `TimeProvider` to the sandbox ledger Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-26 11:31:26 +03:00
"@maven//:io_dropwizard_metrics_metrics_core",
Sandbox: Preliminary work to rebuild Sandbox on top of kvutils. (#4422) * resources: Allow for startup exceptions without no stack trace. * sandbox: Remove dependency duplication in the BUILD file. * ledger-on-(memory|sql): Extract `Main` into its own file tree. This is so we can remove the dependency on kvutils/app from the library. logback.xml goes with it. * sandbox: Implement "Sandbox Next" on top of ledger-on-sql. CHANGELOG_BEGIN - [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML Ledger Integration Kit. Currently not exposed through the CLI. CHANGELOG_END * ledger-on-sql: Publish to Maven Central. * sandbox: If an unknown database is passed to Sandbox Next, handle it. * resources: Catch errors in owner construction or acquisition. Sometimes we're not in a future yet. * resources: Don't run ProgramResource error handling inside the EC. Otherwise we end up in deadlock which is only broken by `awaitTermination` timing out.
2020-02-06 20:34:17 +03:00
],
)
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
da_scala_binary(
name = "app",
main_class = "com.daml.ledger.on.sql.Main",
visibility = ["//visibility:public"],
runtime_deps = all_database_runtime_deps,
deps = [
Sandbox: Preliminary work to rebuild Sandbox on top of kvutils. (#4422) * resources: Allow for startup exceptions without no stack trace. * sandbox: Remove dependency duplication in the BUILD file. * ledger-on-(memory|sql): Extract `Main` into its own file tree. This is so we can remove the dependency on kvutils/app from the library. logback.xml goes with it. * sandbox: Implement "Sandbox Next" on top of ledger-on-sql. CHANGELOG_BEGIN - [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML Ledger Integration Kit. Currently not exposed through the CLI. CHANGELOG_END * ledger-on-sql: Publish to Maven Central. * sandbox: If an unknown database is passed to Sandbox Next, handle it. * resources: Catch errors in owner construction or acquisition. Sometimes we're not in a future yet. * resources: Don't run ProgramResource error handling inside the EC. Otherwise we end up in deadlock which is only broken by `awaitTermination` timing out.
2020-02-06 20:34:17 +03:00
":ledger-on-sql-app",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
],
)
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
da_scala_library(
name = "ledger-on-sql-test-lib",
srcs = glob(["src/test/lib/scala/**/*.scala"]),
visibility = [
"//visibility:public",
],
deps = [
":ledger-on-sql",
Sandbox: Preliminary work to rebuild Sandbox on top of kvutils. (#4422) * resources: Allow for startup exceptions without no stack trace. * sandbox: Remove dependency duplication in the BUILD file. * ledger-on-(memory|sql): Extract `Main` into its own file tree. This is so we can remove the dependency on kvutils/app from the library. logback.xml goes with it. * sandbox: Implement "Sandbox Next" on top of ledger-on-sql. CHANGELOG_BEGIN - [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML Ledger Integration Kit. Currently not exposed through the CLI. CHANGELOG_END * ledger-on-sql: Publish to Maven Central. * sandbox: If an unknown database is passed to Sandbox Next, handle it. * resources: Catch errors in owner construction or acquisition. Sometimes we're not in a future yet. * resources: Don't run ProgramResource error handling inside the EC. Otherwise we end up in deadlock which is only broken by `awaitTermination` timing out.
2020-02-06 20:34:17 +03:00
":ledger-on-sql-app",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"//daml-lf/data",
Expand `Runner`'s flexibility as a basis for participant servers (#4669) * Add overridable indexer, api and auth configuration to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable indexer and api metrics creation to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable api's `TimeServiceBackend` to `LedgerFactory` * :art: Fix formatting * Port SDK ledgers based on `Runner` (and the sandbox) to `TimeServiceBackend` * Revert to `TimeProvider` for committer usage and to `None` default for API server. Also removed now unused `TimeServiceProvider.wallClock()`. * Move TimeServiceBackend back to the API server. * :art: Remove unneeded argument passed for parameter w/default * Restore sandbox ledger time support * Simplify passing a `TimeProvider` to the sandbox ledger Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-26 11:31:26 +03:00
"//language-support/scala/bindings",
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
"//ledger-api/rs-grpc-bridge",
"//ledger-api/testing-utils",
Expand `Runner`'s flexibility as a basis for participant servers (#4669) * Add overridable indexer, api and auth configuration to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable indexer and api metrics creation to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable api's `TimeServiceBackend` to `LedgerFactory` * :art: Fix formatting * Port SDK ledgers based on `Runner` (and the sandbox) to `TimeServiceBackend` * Revert to `TimeProvider` for committer usage and to `None` default for API server. Also removed now unused `TimeServiceProvider.wallClock()`. * Move TimeServiceBackend back to the API server. * :art: Remove unneeded argument passed for parameter w/default * Restore sandbox ledger time support * Simplify passing a `TimeProvider` to the sandbox ledger Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-26 11:31:26 +03:00
"//ledger/ledger-api-auth",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"//ledger/ledger-api-health",
"//ledger/participant-state",
"//ledger/participant-state/kvutils",
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
"//ledger/participant-state/kvutils:kvutils-tests-lib",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"//ledger/participant-state/kvutils/app",
Expand `Runner`'s flexibility as a basis for participant servers (#4669) * Add overridable indexer, api and auth configuration to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable indexer and api metrics creation to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable api's `TimeServiceBackend` to `LedgerFactory` * :art: Fix formatting * Port SDK ledgers based on `Runner` (and the sandbox) to `TimeServiceBackend` * Revert to `TimeProvider` for committer usage and to `None` default for API server. Also removed now unused `TimeServiceProvider.wallClock()`. * Move TimeServiceBackend back to the API server. * :art: Remove unneeded argument passed for parameter w/default * Restore sandbox ledger time support * Simplify passing a `TimeProvider` to the sandbox ledger Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-26 11:31:26 +03:00
"//ledger/sandbox",
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
"//libs-scala/contextualized-logging",
libs-scala/ports: Wrap socket ports in a type, `Port`. (#4784) * libs-scala/ports: Wrap socket ports in a type, `Port`. * sandbox: Use `Port` for the API server port, and propagate. CHANGELOG_BEGIN CHANGELOG_END * extractor: Use `Port` for the server port. * ports: Make Port a compile-time class only. * ports: Allow port 0; it can be specified by a user. * ports: Publish to Maven Central.
2020-03-03 11:59:15 +03:00
"//libs-scala/ports",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"//libs-scala/postgresql-testing",
"//libs-scala/resources",
"@maven//:com_github_scopt_scopt_2_12",
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
"@maven//:com_typesafe_akka_akka_actor_2_12",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"@maven//:com_typesafe_akka_akka_stream_2_12",
Expand `Runner`'s flexibility as a basis for participant servers (#4669) * Add overridable indexer, api and auth configuration to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable indexer and api metrics creation to `LedgerFactory` CHANGELOG_BEGIN CHANGELOG_END * Add overridable api's `TimeServiceBackend` to `LedgerFactory` * :art: Fix formatting * Port SDK ledgers based on `Runner` (and the sandbox) to `TimeServiceBackend` * Revert to `TimeProvider` for committer usage and to `None` default for API server. Also removed now unused `TimeServiceProvider.wallClock()`. * Move TimeServiceBackend back to the API server. * :art: Remove unneeded argument passed for parameter w/default * Restore sandbox ledger time support * Simplify passing a `TimeProvider` to the sandbox ledger Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-26 11:31:26 +03:00
"@maven//:io_dropwizard_metrics_metrics_core",
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
"@maven//:org_scalactic_scalactic_2_12",
"@maven//:org_scalatest_scalatest_2_12",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
],
)
da_scala_test_suite(
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
name = "ledger-on-sql-tests",
srcs = glob(["src/test/suite/**/*.scala"]),
data = [
"//ledger/test-common:Test-stable.dar",
],
resources = glob(["src/test/resources/*"]),
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
runtime_deps = all_database_runtime_deps,
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
deps = [
":ledger-on-sql",
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
":ledger-on-sql-test-lib",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
"//daml-lf/data",
"//ledger-api/rs-grpc-bridge",
"//ledger-api/testing-utils",
"//ledger/ledger-api-common",
"//ledger/ledger-api-health",
"//ledger/participant-state",
"//ledger/participant-state/kvutils",
"//ledger/participant-state/kvutils:kvutils-tests-lib",
ledger-on-sql: Don't allow unnamed in-memory databases. (#4255) * ledger-on-sql: Don't allow unnamed in-memory databases. It seems that changing the connection pool size at runtime causes weird issues, predominantly that sometimes all connections are dropped and therefore the in-memory database is lost after migration. Obviously, it then stops working. This works around this problem by simply not allowing unnamed in-memory databases, guaranteeing that we can share the database between the various connection pools. This will probably be necessary anyway when attempting to share the database between the ledger and the index. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it clear we're generating a new JDBC URL in tests. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-29 13:34:00 +03:00
"//libs-scala/contextualized-logging",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
"//libs-scala/postgresql-testing",
kvutils: Use resources in KeyValueParticipantState, not AutoClo… (#4177) * kvutils: Use resources in KeyValueParticipantState, not AutoCloseable. CHANGELOG_BEGIN - [kvutils] The simplified API now uses ``com.digitalasset.resources`` to manage acquiring and releasing resources instead of ``Closeable``. CHANGELOG_END * ledger-on-posix-filesystem: Remove dead code due to a refactoring fail.
2020-01-23 18:59:01 +03:00
"//libs-scala/resources",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
"@maven//:com_typesafe_akka_akka_actor_2_12",
"@maven//:com_typesafe_akka_akka_stream_2_12",
ledger-on-sql: Migrate tables using Flyway. (#4232) * ledger-on-sql: Pull out a superclass for the various integration tests. * ledger-on-sql: Use Flyway to migrate tables in an idempotent manner. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Make it easy to run it with Bazel for experimentation. * ledger-on-sql: Test that migrations will never change in the future. * ledger-on-sql: Add a prefix of "ledger_" to the tables. This is so we don't accidentally conflict with the index when the schemas are shared. I am letting myself modify the migrations because the existing migrations haven't been merged into `master` yet. * ledger-on-sql: Explain why we change the pool size after migration. * ledger-api-test-tool: Tests now specify a timeout scale, not a timeout. Makes it easier to change the default timeout in one place. * ledger-api-test-tool: Increase timeouts on slow tests. These tests produce a lot of volume and can make CI flaky. * ledger-on-sql: Only tear down PostgreSQL in tests. For other databases, we just create a new file for each test case. * ledger-on-sql: Reduce the log output in tests. * ledger-on-sql: Use a separate connection pool for Flyway when possible. Apparently `setMaximumPoolSize` doesn't really have the desired effect after the connection pool has already been used. The new test case will be flaky if we process more than one commit in parallel. For SQLite, it seems to be OK. * ledger-on-sql: Use a separate connection pool for Flyway with SQLite. Except in memory. * ledger-on-sql: Use a separate PostgreSQL database for each test. Because performance, innit. Don't have to tear them down. * ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects. * ledger-on-sql: Go into even more detail about pool size hijinks. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 19:16:23 +03:00
"@maven//:org_flywaydb_flyway_core",
ledger-on-sql: Make sure the ledger picks up where it left off. (#4266) * resources: Make it easy to use a resource and then release it. * ledger-on-sql: Move database transaction methods to `Database`. * ledger-on-sql: Make sure the ledger picks up where it left off. This means seeding the dispatcher with the correct head, rather than starting at 1 every time. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Move `Index` and `StartIndex` to the package object. They're used everywhere. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-30 13:16:32 +03:00
"@maven//:org_scala_lang_modules_scala_java8_compat_2_12",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
"@maven//:org_scalactic_scalactic_2_12",
"@maven//:org_scalatest_scalatest_2_12",
],
)
[
(
da_scala_binary(
name = "conformance-test-{}-bin".format(db["name"]),
main_class = db.get("conformance_test_server_main", "com.daml.ledger.on.sql.Main"),
visibility = ["//visibility:public"],
runtime_deps = db.get("runtime_deps", []),
deps = [
":ledger-on-sql",
ledger-on-sql: PostgreSQL support. (#4204) * ledger-on-sql: Don't bother cleaning up after integration tests. Turns out Bazel cleans up before each test run, so we only have one set of outputs at a time. This is far more useful for debugging anyway. * ledger-on-sql: Pull out a test-lib to simplify the conformance tests. * ledger-on-sql: Turn `Database` from a sealed trait into a case class. * ledger-on-sql: Support for PostgreSQL! CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run the conformance tests against PostgreSQL. * ledger-on-sql: Run the LotsOfParties conformance tests. * ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query. * ledger-on-sql: Ensure the reader connection pool is read-only. * ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible. Instead, use `ByteString#newInput()`, which returns an `InputStream`. This is supported by H2 and PostgreSQL, but unfortunately not SQLite. * ledger-on-sql: Run integration tests in parallel. * Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 17:33:11 +03:00
":ledger-on-sql-test-lib",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
],
),
conformance_test(
name = "conformance-test-{}".format(db["name"]),
ports = [6865],
server = ":conformance-test-{}-bin".format(db["name"]),
[kvutils] use new contract id scheme (#4741) * Make kvutils work with the new contract id scheme CHANGELOG_BEGIN - [KVUtils] uses random contract id. Contract ids are made of 65 hexa decimal characters. CHANGELOG_END Co-authored-by: Jussi Mäki <jussi.maki@digitalasset.com>
2020-03-02 19:50:38 +03:00
server_args = [
"--contract-id-seeding=weak",
"--participant participant-id=conformance-test,port=6865",
] + db.get("conformance_test_server_args", []),
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
tags = db.get("conformance_test_tags", []),
ledger-on-sql: Get H2 working in the conformance tests. (#4155) * kvutils: Make logback.xml a base file, rather than the only option. * kvutils/app: Simplify logback.base.xml. Mostly by getting rid of unnecessary appenders. * ledger-on-sql: Add trace logging for all database work. * Upgrade H2; there's a few useful bug fixes. * kvutils/app: Let the user override the server JDBC URL. * kvutils/app: Provide a way to specify the ledger ID. * ledger-on-sql: If there are missing entries in the log, fail on read. This can happen right now because we insert in parallel. Next step: stop doing that. * ledger-on-sql: Stop writing in parallel; it causes race conditions. This unfortunately means we also stop _reading_ in parallel, which is less fun. * Revert "ledger-on-sql: Use a sequential log entry ID." This reverts commit c58265bf43b96d16bfa62ed30a795f365b1e83f2. * ledger-on-sql: Create tables with columns that are not nullable. * ledger-on-sql: The H2 conformance tests now work, just slowly. CHANGELOG_BEGIN CHANGELOG_END * ledger-on-sql: Run tests against H2 and SQLite on memory and disk. * ledger-on-sql: Allow H2 to read from the log in parallel with writes. * ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes. * ledger-on-sql: Make sure to log the correlation ID. * ledger-on-sql: Do less while holding a database connection. * ledger-on-sql: Log the connection. * ledger-on-sql: We don't need to tell H2 not to drop the connection. The connection pool takes care of that. * ledger-on-sql: Disable H2 conformance tests on CI; they're too slow. * ledger-on-sql: Rename `loggingContext` to `logCtx`. * ledger-on-sql: Don't abuse log contexts; put the data in the message. * ledger-on-sql: Make the connection log line easier to read. And pull out the logger, even if it's only used once. * ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 18:54:51 +03:00
test_tool_args = db.get("conformance_test_tool_args", []) + [
Remove InMemoryKVParticipantState (#4674) This removes the sample/reference implementation of kvutils InMemoryKVParticipantState. This used to be the only implementation of kvutils, but now with the simplified kvutils api we have ledger-on-memory and ledger-on-sql. InMemoryKVParticipantState was also used for the ledger dump utility, which now uses ledger-on-memory. * Runner now supports a multi participant configuration This change removes the "extra participants" config and goes for consistent participant setup with --participant. * Run all conformance tests in the repository in verbose mode. This means we'll print stack traces on error, which should make it easier to figure out what's going on with flaky tests on CI. This doesn't change the default for other users of the ledger-api-test-tool; we just add the flag for: - ledger-api-test-tool-on-canton - ledger-on-memory - ledger-on-sql - sandbox Fixes #4225. CHANGELOG_BEGIN CHANGELOG_END
2020-02-26 17:45:35 +03:00
"--verbose",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
"--all-tests",
"--exclude=ConfigManagementServiceIT",
Implement new command deduplication (#4467) * Add TTL field to protobuf * Add command deduplication to index service * Wire command deduplication to DAO * Implement in-memory command deduplication * Remove Deduplicator * Implement JDBC command deduplication * Add TTL field to domain commands * Deduplicate commands in the submission service CHANGELOG_BEGIN - [Sandbox] Implement a new command submission deduplication mechanism based on a time-to-live (TTL) for commands. See https://github.com/digital-asset/daml/issues/4193 CHANGELOG_END * Remove unused command service parameter * fixup protobuf * Add configuration for TTL * Fix Haskell bindings * Rename SQL table * Add command deduplication test * Redesign command deduplication queries * Address review comment * Address review comment * Address review comments * Make command deduplication test optional * Disable more tests * Address review comments * Address review comments * Refine test * Address review comments * scalafmt * Truncate new table on reset * Store original command result * Rename table columns ... to be consistent with other upcoming tables * Rename migrations to solve conflicts Fixes #4193.
2020-02-26 14:00:02 +03:00
"--exclude=CommandDeduplicationIT",
ledger-on-whatever: Enable more conformance tests. (#4250) * ledger-on-{memory,posix-filesystem,sql}: Enable slow conformance tests. They're now fast enough. :-) CHANGELOG_BEGIN CHANGELOG_END * ledger-on-{memory,posix-filesystem,sql}: Enable config management tests. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-29 12:30:49 +03:00
],
),
conformance_test(
name = "conformance-test-config-management-{}".format(db["name"]),
ports = [6865],
server = ":conformance-test-{}-bin".format(db["name"]),
[kvutils] use new contract id scheme (#4741) * Make kvutils work with the new contract id scheme CHANGELOG_BEGIN - [KVUtils] uses random contract id. Contract ids are made of 65 hexa decimal characters. CHANGELOG_END Co-authored-by: Jussi Mäki <jussi.maki@digitalasset.com>
2020-03-02 19:50:38 +03:00
server_args = [
"--contract-id-seeding=weak",
"--participant participant-id=conformance-test,port=6865",
] + db.get("conformance_test_server_args", []),
ledger-on-whatever: Enable more conformance tests. (#4250) * ledger-on-{memory,posix-filesystem,sql}: Enable slow conformance tests. They're now fast enough. :-) CHANGELOG_BEGIN CHANGELOG_END * ledger-on-{memory,posix-filesystem,sql}: Enable config management tests. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-29 12:30:49 +03:00
tags = db.get("conformance_test_tags", []),
test_tool_args = db.get("conformance_test_tool_args", []) + [
Remove InMemoryKVParticipantState (#4674) This removes the sample/reference implementation of kvutils InMemoryKVParticipantState. This used to be the only implementation of kvutils, but now with the simplified kvutils api we have ledger-on-memory and ledger-on-sql. InMemoryKVParticipantState was also used for the ledger dump utility, which now uses ledger-on-memory. * Runner now supports a multi participant configuration This change removes the "extra participants" config and goes for consistent participant setup with --participant. * Run all conformance tests in the repository in verbose mode. This means we'll print stack traces on error, which should make it easier to figure out what's going on with flaky tests on CI. This doesn't change the default for other users of the ledger-api-test-tool; we just add the flag for: - ledger-api-test-tool-on-canton - ledger-on-memory - ledger-on-sql - sandbox Fixes #4225. CHANGELOG_BEGIN CHANGELOG_END
2020-02-26 17:45:35 +03:00
"--verbose",
ledger-on-whatever: Enable more conformance tests. (#4250) * ledger-on-{memory,posix-filesystem,sql}: Enable slow conformance tests. They're now fast enough. :-) CHANGELOG_BEGIN CHANGELOG_END * ledger-on-{memory,posix-filesystem,sql}: Enable config management tests. Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-29 12:30:49 +03:00
"--include=ConfigManagementServiceIT",
A simplified KVUtils ledger, implemented on SQL with JDBC. (#3940) * ledger-on-sql: Implement a ledger using the simplified kvutils on JDBC. This tests against H2; PostgreSQL coming soon. No changelog because we're not shipping this yet. CHANGELOG_BEGIN CHANGELOG_END * kvutils/app: Extract ledger construction into a trait. * ledger-on-sql: Run the conformance tests against H2. * ledger-on-sql: Use H2-specific queries to perform actions atomically. * ledger-on-sql: Extract queries to `Queries` and `H2Queries`. * ledger-on-sql: SQLite can't have more than one connection at a time. * ledger-on-sql: Run conformance tests against SQLite. * ledger-on-sql: Fix error message if the JDBC URL is missing. Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * kvutils/app: Rename LedgerConstructor to LedgerFactory. * ledger-on-sql: Don't release connections early. * ledger-on-sql: Rename `withConnection` so it's more meaningful. * ledger-on-sql: Use a sequential log entry ID. * ledger-on-sql: Disable H2 conformance tests. Until we can figure out why some fail. Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-01-16 15:02:16 +03:00
],
),
)
for db in supported_databases
]
Reference in New Issue Copy Permalink