daml/ledger-service
Stephen Compall b4d00317b4
detect unsynchronized contract table and retry (#10617)
* enumerating out-of-sync offsets at the DB level

* cleanup in lastOffset

* write the latest-requested-or-read offset when catching up

- Writing only the latest-read, as before, would imply unsynced offsets
  that are actually well-synced.  This puts the DB in a more uniform
  state, i.e. it should actually reflect the single value that the
  fetchAndPersist loop tries to catch everything up to.

* detecting lagging offsets from the unsynced-offsets set

- Treating every unsynced offset as a lag would make us needlessly retry
  perfectly synchronized query results.

* add Foldable1 derived from Foldable for NonEmpty

* nicer version of the unsynced function

* ConnectionIO scalaz monad

* rename Offset.ordering to `Offset ordering` so it can be imported verbatim

* finish aggregating in the lag-detector function, compiles

* port sjd

* XTag, a scalaz 7.3-derived tag to allow stacked tags

* make the complicated aggregation properly testable

* extra semantic corner cases I didn't think of

* tests for laggingOffsets

* a way to rerun queries if the laggingOffsets check reveals inconsistency

* if bookmark is ever different, we always have to rerun anyway

* boolean blindness

* incorporate laggingOffsets into fetchAndPersistBracket

* split fetchAndPersist from getTermination and clean up its arguments

* just compose functors

* add looping to fetchAndPersistBracket

* more mvo tests

* test unsyncedOffsets, too

* Lagginess collector

* supply more likely actual data with mvo tests; don't trust Java equals

* rework minimumViableOffsets to track sync states across template IDs

* extra note

* fix the tests to work against the stricter mvo

* move surrogatesToDomains call

* more tests for lagginess accumulator

* add changelog

CHANGELOG_BEGIN
- [JSON API] Under rare conditions, a multi-template query backed by database
  could have an ACS portion that doesn't match its transaction stream, if
  updated concurrently.  This conditions is now checked and accounted for.
  See `issue #10617 <https://github.com/digital-asset/daml/pull/10617>`__.
CHANGELOG_END

* port toSeq to Scala 2.12

* handle a corner case with offsets being too close to expected values

* didn't need XTag
2021-09-28 20:47:42 +00:00
..
cli-opts Simplify loading of logback file (#10592) 2021-08-17 08:46:37 +00:00
db-backend detect unsynchronized contract table and retry (#10617) 2021-09-28 20:47:42 +00:00
http-json detect unsynchronized contract table and retry (#10617) 2021-09-28 20:47:42 +00:00
http-json-cli [JSON-API] surrogate template id cache (#10806) 2021-09-10 10:48:58 +00:00
http-json-ledger-client [JSON-API] Use the token from incoming requests to update the package list (#10602) 2021-09-03 10:12:01 +00:00
http-json-oracle enable JSON search index on Oracle by default (#10539) 2021-08-31 09:29:36 +00:00
http-json-perf [JSON-API] Postgres perf job (#10986) 2021-09-23 09:16:58 +01:00
http-json-testing [DPP-589] Add CLI flag to select minimum enabled TLS version (#10854) 2021-09-14 12:37:38 +02:00
jwt [DPP-418] Protect Participant TLS keys (#10629) 2021-08-30 09:24:52 +02:00
lf-value-json Drop ContractId typeparameter from Value (#10827) 2021-09-16 08:46:57 +00:00
utils [JSON-API] Use the token from incoming requests to update the package list (#10602) 2021-09-03 10:12:01 +00:00