* Use proper names for the DB connection pools.
* Add some trace logging for DB operations.
This does not capture timing of individual statements but
rather the time for "units of work".
For example "lookup contract" doesn't mean only loading
a single row from the contracts table, but also 2 additional
queries for looking up witnesses and divulgences.
This is not a problem, because this is trace level logging
that helps us debug problems and shouldn't be made sense of by
users at this stage.
* Make description a by-name parameter.
This avoids building up the string if trace logging
is not enabled (i.e. most of the time).
* Make sure ActiveContractService sends out a consistent snapshot
instead of looking up the latest ledger end, we can only take the latest known ledgerEnd in the scope of SqlLedger.
If we don't do that, we can miss contracts from a partially inserted batch insert of ledger entries
scenario:
1. batch insert transactions A and B at offsets 5 and 6 respectively; A is a huge transaction, B is a small transaction
2. B is inserted earlier than A and the ledger_end column in the parameters table is updated
3. A GetActiveContractsRequest comes in and we look at the latest ledger_end offset in the database. We will see 6 (from transaction B).
4. If we finish streaming the active contracts up to offset 6 before transaction A is properly inserted into the DB, the client will not see the contracts from transaction A
The fix to that is to use the latest known head to the dispatcher, which is updated AFTER a batch has been inserted completely.
This PR also factors out the basic readonly functionality that is
duplicated between SqlLedger and ReadOnlySqlLedger into a class
BaseLedger.
* copyright header
* add Numeric.java
* ledger-api: rename `decimal` field to `numeric` in value protobuf
* Address Gerolf's comment
* ledger-api: add missing renammings
* ledger-api: relax syntax of numbers that can be sent as numerics
* extractor: fix
* leger-api: change format of number though ledger api
* daml-lf: fix numeric regexp
* ledger: fix tests
* language: compile everything in the source directory
This removes the need to specify a 'main'. Instead we 'source' in
daml.yaml should point to the source root directory.
Both language extension are needed to improve the UX of generic templates.
Without them templates a la
```
template Template (A t) => B t with
```
would produce an error that we need `FlexibleContexts` for the
`Template (A t)` context and a warning suggesting to simplify it to
`AInstance t`. The latter would expose an implementation detail we prefer
to hide.
* Initial version for configuration submissions and authorization
* Refactor config submission based on review. Add tests.
* Cleanup test output and comments
* kvutils test-suite
- Add unit tests for kvutils
- Thread the input state into Commit monad
- Fix a bug in handling rejected transactions (from previous refactoring)
* Address PR review
- Add submissionId to configuration changes.
- Revert the dedup of the ParticipantNotAuthorized message. This did not buy much
and we need to duplicate it anyway when modularizing kvutils.
This switches the creation of the archive in `daml build` from
`zip-archive` to `zip`. This has a few advantages:
1. It gets rid of lazy IO for reading all the interface and source
files. This avoids the high usage of file handles in `daml build`.
2. It seems to be a slight improvement in max memory usage and runtime
and a giant improvement in allocations (but I think the latter
probably comes primarily from the fact that the locations are moved to
the bzip C library). The improvement in max memory usage is less than
I expected so probably there is still something off somewhere.
For now, I only switched over `createArchive`. Archive reading is
still done using `zip-archive`. We might want to switch that over in a
separate PR.
This fixes all flakiness in `damlc test` that I was able to
reproduce. Previously, I got it to fail in about 10% of the cases
whereas now I have successfully run tests 200 times under load without
issues.
There were two issues at play here:
1. We run scenarios in separate threads to be able to kill the running
Shake session quickly even if a scenario has an infinite loop or
something like that (there is a timeout but it’s quite long). This
could result in one of those left-over threads trying to issue a
request while we are already trying to shut down.
To fix that, we wait for the concurrency semaphore to be empty before
shutting down.
2. Just waiting for scenario executions is not quite sufficient as
`runAction` does not wait for all rules to finish (we could just use
runActionSync in `damlc test` but I’d rather make this work
properly). While we do wait for all scenario executions to finish
there is one gRPC request in offInterest that we do not wait for:
gcCtxs.
To fix this, I’ve now routed all gRPC requests through the semaphore
which means that we will also wait for these requests to finish (or
prevent them from spawning).
This makes more sense anyway as scenario executions are mostly fairly
cheap requests while things like setting up the context are expensive
so we want to limit their concurrency.
We should make the concurrency limit configurable but I’ll leave that
for a separate PR.
* daml-lf: make match2's fallback lazy, and add tuple2 syntax for better inference
* lf-value-json: when decoding, branch on type, *then* on value
- restores totality checking for interface ADT
- makes alternate input formats obvious at a glance
- lets us restructure in the future to cache type branch lookups
* lf-value-json: support numbers for int64s
* lf-value-json: test failures and more timestamp cases
* lf-value-json: adapt timestamp decoding to new rules
* lf-value-json: cleanup
* lf-value-json: decode Number as decimal
* lf-value-json: forbid yet another timezone format
* lf-value-json: don't copy
* lf-value-json: replace {Some, {None optional format with nest-sensitive [] format
* lf-value-json: round decimals rather than rejecting, in either string or number form
* lf-value-json: support output to JsNumber for int64s and decimals, separately
- not dependent on JavaScript safe integer range, because that is not
the purpose of this option
* lf-value-json: uncomment some now-tested rounding cases
* allow None record fields to be omitted in object syntax
* release notes
* Print stack traces in the scenario on failure
Currently, we only print the last source location, which is not
particularly helpful for debugging. Now, we put all source locations we
encounter during execution on the continuation stack and print them when
a scenario fails. This PR does not print the names of entered functions
or choices. We leave this for a future PR.
* Address Moritz' comments
* Upgrade ghc-libs supporting generic templates
* Update Proposal and ComposedKey tests to generic template syntax
* Temporarily patch daml-doc test output (will need to fix for template instances)
It looks like that closing the LedgerApiServer before closing the
AkkaExecutionSequencerPool (AESP) leads to a non-deterministic bunch of dead
letter warnings when stopping sandbox.
Closing the AESP first avoids these warnings.
Fixes#1886
* Only change project root once
As described in #2449, calling withProjectRoot' twice breaks with
relative paths and is also just silly so this PR fixes this by
factoring out the actual logic from init from execInit which does the
project root thingy.
* Add a regression test
This happened because we translate empty workflow ids to None in the
API-to-domain translation layer. When we store the workflow id, we actually
convert the None case back to an empty string (and not a null value).
When loading the workflow id from the database, we parse the column as a regular
LedgerString, but unfortunately these cannot be empty. Therefore using empty workflow
ids on postgres has been broken since workflow id has changed from a tagged string
to LedgerString.
* Read stakeholders through the active contract service
Fixes#2070
Also standardizes the use of American English spelling for `serialize`
and `deserialize`. Standardize. Serialize. Americanize.
* Remove Haskell bindings test workaround
* Address https://github.com/digital-asset/daml/pull/2316#discussion_r308193661
This is a breaking change on the ledger api. So far we could rely on the
assumption that contractId == eventId. This assumption doesn't hold
anymore in a daml-on-x setting, where the eventId is created by the
api server, but the absolute contractId is created by the ledger
implementation.
Instead of going through weird contortions to store the relevant data in
the existing database schema, it is more viable in the long term to
remove that field and instead provide facilities to lookup transactions
by contractId.
Contributes to #2068.
