The database-backed reset service can (understandably) go a bit slower than the one backed by the in-memory ledger.
This should help avoiding flaky tests.
CHANGELOG_BEGIN
CHANGELOG_END
Some Option2Iterable ignore annotations are not needed, others were needed for unused methods.
In a few occasions we were ignoring the warning for the very purpose for which is was there,
i.e. avoiding an implicit conversion. I'm all for not verifying this rule if we agree we
don't need it.
For ProcessFailedException it was a bit gratuitous, I changed the way in which the exception
message is built.
CHANGELOG_BEGIN
CHANGELOG_END
* Add test to ensure that the reset truncates all tables
The test can be adjusted over time to accomodate for exceptions (which are already there).
Unfortunataly we have to add a new couple of queries to support both Postgres and H2.
Fixes#5130
CHANGELOG_BEGIN
CHANGELOG_END
* Make loose check on configuration_entries
* sandbox: Clean up `MetricsReporting` a little.
Make sure it closes both reporters, and avoid starting things in a
constructor.
* sandbox: Add hidden options for enable metrics reporting.
* sandbox: Add a disambiguating name to the DB connection/thread pools.
CHANGELOG_BEGIN
- [Sandbox] DB connection pool metrics names have changed slightly, from
``daml.index.db.connection`` to ``daml.index.db.connection.sandbox``.
- [Ledger Integration Kit] DB connection pool metrics names have changed
to disambiguate the StandaloneApiServer from the
StandaloneIndexerServer. The former now has a ``.ledger-api-server``
suffix, and the latter now has a ``.indexer`` suffix.
CHANGELOG_END
* sandbox-next: Use the same metrics registry for the API and indexer.
* sandbox: Give a useful error message on an invalid metrics reporter.
And simplify the error messages.
With the arguments `--client-auth=foo --metrics-reporter=foo`, we now
get the output:
```
Error: Option --client-auth failed when given 'foo'. Must be one of
"none", "optional", or "require".
Error: Option --metrics-reporter failed when given 'foo'. Must be one of
"console", or "csv:PATH".
Try --help for more information.
```
* sandbox: Pull out more helpers in `MetricsReporting`.
* sandbox: Rename MetricsReporter classes so they don't clash.
* sandbox: Wrap the `name` parameter in a `ServerName` tagged string.
For safety. Yours, not mine.
* sandbox: Push metrics to Graphite with `--metrics-reporter=graphite`.
* sandbox: Make `MetricsReporter.Graphite` singly-lazy, not doubly-.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* sandbox: Replace `ServerName` with `ServerRole`.
* sandbox: Fix usage of `ServerRole.Testing` in `LedgerResource`.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
- New seeding type "Static" that uses a fixed seed for generating new seeds.
This is only used in Sandbox and Sandbox-next.
- Remove the fuzzing for submission time in Engine.scala
- DAML-on-SQL: Create new log entry IDs from a provided SeedService.
This allows for generating deterministic transaction IDs in
Sandbox-Next.
Fixes#5107
CHANGELOG_BEGIN
[Sandbox] Add contract-id-seeding=static to allow for predictable contract IDs. This is useful for documentation,
to be able to refer to a specific contract ID instead of having to write "note down the contract ID you see on the screen. we will use it later."
[DAML-on-SQL] Derive the next log entry ID using the provided SeedService. This allows us to also deterministically create transactionIds in static time mode together with `--contract-id-seeding=static`. This should only be used for demos or documentation.
CHANGELOG_END
* sandbox: Remove a few levels of indentation in `ApiCommandService`.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Get rid of the `LowLevelCommandServiceAccess` trait.
It only has one implementation. Just use it.
* sandbox: Remove unused type aliases from ApiCommandService.
* kvutils/app|sandbox: Rename `system` to `actorSystem`.
* sandbox: Pass a materializer in to the StandaloneIndexerServer.
There's no need for it to construct one when the caller always has one
available.
CHANGELOG_BEGIN
CHANGELOG_END
* recovering-indexer: Use `SubmissionId` instead of `LedgerString`.
Where appropriate.
* sandbox: Use the materializer implicitly in JdbcIndexer.
by reading parameters.ledger_end column instead of parameters.external_ledger_end.
CHANGELOG_BEGIN
[Ledger API Server]: Upon restart, ledger api server continues consuming unconsumed events rather than
all events from beginning.
CHANGELOG_END
Closes#5121
* Integrate transaction lookup on new schema in Ledger API
Re-wires all transaction lookups to the new schema
CHANGELOG_BEGIN
CHANGELOG_END
* Always return the agreement text
CHANGELOG_BEGIN
[Ledger API Server] The metric 'daml.index.lookup_transaction' has been
replaced by 'daml.index.lookup_flat_transaction_by_id' and
'daml.index.lookup_transaction_tree_by_id', which record the same events
but with more granularity regarding the type of lookup.
CHANGELOG_END
* Ensure agreement text invariant in a single place
* Do not compare the order in which witness parties appear in an event
* Hide command identifier from non-submitters in transaction trees
* Fix time assigned to transaction to be the ledger effective time and not the record time
* Store transactions from initial state into the new schema
* sandbox: Don't let just anyone construct a DbDispatcher.
It's the job of the JdbcLedgerDao.
* sandbox: Clean up the DbDispatcher a little.
* sandbox: JdbcLedgerDao now creates its own execution context.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Make `defaultNumberOfShortLivedConnections` private.
Makes storing and generating test transaction more composable (store just takes a generated transaction)
- Add a nonTransient utility method to retrieve contracts that have been created but not consumed as part of a transaction
- Add an addChildren utility method to add children to a transaction to allow to create more complex test transactions
- Add a transaction generator that uses the aforementioned addChildren to create a more complex transaction
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Remove duplicate parameters in JdbcIndexerFactory.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: `InitializedJdbcIndexerFactory` is just a ResourceOwner now.
* sandbox: Rename helper methods in `JdbcIndexerFactory`.
* sandbox: Make `JdbcIndexer#initialized` look like it does something.
Avoiding `damlc compile/package` commands (which we would like to deprecate), and replace with plain `damlc build` together with a post dar->dalf extraction step in the couple of places where we actually want the .dalf for testing.
changelog_begin
changelog_end
* Modularize JdbcLedgerDaoSpec
Adding more tests, but breaking it up a bit to make sense of it beforehand.
CHANGELOG_BEGIN
CHANGELOG_END
* Remove unnecessary suppressed warnings
* sandbox: Fail to start if a time mode is not explicitly specified.
CHANGELOG_BEGIN
- [Sandbox] Sandbox is switching from Static Time mode to Wall Clock
Time mode as the default. To ensure that our users know about this,
for one version, there will be no default time mode. Instead, users
will have to explicitly select their preferred time mode by means of
the `--static-time` or `--wall-clock-time` switches. In the next
release, Wall Clock Time will become the default, and users who are
happy with the defaults will no longer need to specify the time mode.
CHANGELOG_END
* daml-script|triggers: Specify time mode when testing against Sandbox.
* daml-assistant: Default the Sandbox to wall clock time.
CHANGELOG_BEGIN
- [DAML Assistant] Initializing a new DAML project adds a switch to
``daml.yaml`` to ensure Sandbox can continue to start with ``daml
start``::
sandbox-options:
- --wall-clock-time
CHANGELOG_END
* docs: Update the DAML Script and Triggers docs to use Wall Clock time.
It's now what Sandbox will use by default when using `daml init`.
* docs: Change the Quickstart to run Sandbox in wall clock time.
This explains why the contract IDs may vary.
It also updates the manual release testing script to match.
A "stable offset" in the context of the Participant Server is the offset
that was provided by the ledger backend (be it kvutils, corda, daml on sql).
The Participant Server does not keep a participant-local offset anymore.
In a single domain/kvutil setup, this makes offsets stable across participants,
since all participants will see the same offset for the same transaction.
The following changes were needed to achieve this:
- The participant server always uses the offset provided by the backend
AS IS (no more +1 magic).
- Offsets provided to the Ledger API in requests must be treated as
startExclusive and endInclusive (previously beginInclusive and
endExclusive).
CHANGELOG_BEGIN
[Ledger API]: Offsets have been redefined. Instead of being represented
by a number or a structured string, an offset is now an opaque string
that can be compared lexicographically.
[DAML Integration Kit]: The bounds for ``Dispatcher`` are now
startExclusive and endInclusive.
CHANGELOG_END
---------
ledger api:
ledger_offset.proto
Changed definition of offsets, since they can now be compared
lexicographically.
---------
participant-state-api:
Offset:
Changed from Array[Long] to ByteString. Ledgers need to make sure that the
offsets produced are strictly monotonically increasing according to
lexicographical order.
---------
akka-streams:
Dispatcher, DispatcherImpl, SubSource:
Changed interval handling to exclusive/inclusive.
---------
ledger-on-memory:
InMemoryLedgerReaderWriter, InMemoryState:
Changed interval handling to exclusive/inclusive.
---------
ledger-on-sql:
CommonQueries, SqlLedgerReaderWriter:
Change interval in query and boundary handling.
---------
kvutils:
KeyValueParticipantStateReader, KVOffset:
Convenience functions for kvutils to add or remove sub-indexes for
offsets.
KV ledger implementations can use KVOffset to construct a structured offset.
---------
Participant Server:
JdbcLedgerDao:
Use Offset instead of Long.
Fetch offsets directly as Offset from the database with proper anorm
integration.
Change interval handling to exclusive/inclusive.
CommandCompletionsReader, CommandCompletionsTable:
Change interval handling to exclusive/inclusive.
BaseLedger:
Use Offset instead of Long.
Change interval handling to exclusive/inclusive.
Conversions:
Anorm integration for using Offset in queries and result parsers.
JdbcIndexer:
Remove references to "extenalLedgerEnd" and participant-local Long
offset (headRef).
---------
sandbox:
In general:
Use the Offset type everywhere instead of Long.
SQL migrations:
Change all offset columns to bytea or BINARY.
LedgerBackedIndexService:
Proper bounds checking has been pushed down to Dispatcher, which
allowed simplifying the acceptedTransactions implementation.
InMemoryLedger, LedgerEntries:
Change interval handling to exclusive/inclusive.
Transaction lookup by ID is now O(n) because transaction IDs are not
necessarily the same as the offset.
SqlLedger:
Remove external offset references.
* Switched to ByteString from Array[Byte] on almost all simplified API interfaces.
* Sort output by keys.
* Added comment.
CHANGELOG_BEGIN
CHANGELOG_END
* Removed DamlLogEntryId from LedgerEntry.
* Return a SortedMap ordering output state by its keys' hash in order to have deterministic ordering.
* Code tidying.
* Added implicit conversion for anorm for ByteStrings to make SQL queries cleaner.
* Ooops, missed adding a header.
* Avoid copying bytes by anorm by using ByteString.newInput()
* Added some Scaladoc to simplified API interfaces.
* Added docs to LedgerStateAccess.
* Reverted some changes.
* Added some docs to ValidatingCommitter.
* Corrected some typos.
* Added package-level documentation to kvutils.api.
* Clarified convenience classes for LedgerStateOperations.
* Update ledger/participant-state/kvutils/src/main/scala/com/daml/ledger/participant/state/kvutils/Version.scala
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Minor rewording.
* Added missing header.
* Fixed problem with merge.
Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* kvutils: Remove the unnecessary execution context from the test base.
* kvutils: Remove the unnecessary execution context from the writer.
* ledger-on-sql: Make a proper owner so it has a proper execution context.
This means the parallelization now needs to come from the test, so I've
augmented ParticipantStateIntegrationSpecBase to take a proper execution
context instead of the serial one that ScalaTest provides, with a
default of `ExecutionContext.global`.
* ledger-on-memory: Make a proper owner with a proper execution context.
* kvutils/app: Remove `executionContext` from LedgerFactory.
Shouldn't need it in `ResourceOwner`. I was bad.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-on-memory: Make ResourceOwners real classes.
* ledger-on-sql: Make the ResourceOwner a real class.
* ledger-on-sql: Cause side effects on resource acquisition.
Not on owner construction.
This would fail only on PostgreSQL because `IN ()` is invalid. H2 seems
to be fine with it.
CHANGELOG_BEGIN
- [Ledger API Server] Support a call to `GetParties` with an empty list
of parties.
CHANGELOG_END
* Periodically clear expired deduplication entries
CHANGELOG_BEGIN
CHANGELOG_END
Fixes#4959
* Increase cache maintenance frequency
The previous value was only good for testing purposes
* Actually remove deduplication entries
* Clear deduplication cache for IndexAndWriteService
