daml/libs-scala
Stephen Compall b4d00317b4
detect unsynchronized contract table and retry (#10617)
* enumerating out-of-sync offsets at the DB level

* cleanup in lastOffset

* write the latest-requested-or-read offset when catching up

- Writing only the latest-read, as before, would imply unsynced offsets
  that are actually well-synced.  This puts the DB in a more uniform
  state, i.e. it should actually reflect the single value that the
  fetchAndPersist loop tries to catch everything up to.

* detecting lagging offsets from the unsynced-offsets set

- Treating every unsynced offset as a lag would make us needlessly retry
  perfectly synchronized query results.

* add Foldable1 derived from Foldable for NonEmpty

* nicer version of the unsynced function

* ConnectionIO scalaz monad

* rename Offset.ordering to `Offset ordering` so it can be imported verbatim

* finish aggregating in the lag-detector function, compiles

* port sjd

* XTag, a scalaz 7.3-derived tag to allow stacked tags

* make the complicated aggregation properly testable

* extra semantic corner cases I didn't think of

* tests for laggingOffsets

* a way to rerun queries if the laggingOffsets check reveals inconsistency

* if bookmark is ever different, we always have to rerun anyway

* boolean blindness

* incorporate laggingOffsets into fetchAndPersistBracket

* split fetchAndPersist from getTermination and clean up its arguments

* just compose functors

* add looping to fetchAndPersistBracket

* more mvo tests

* test unsyncedOffsets, too

* Lagginess collector

* supply more likely actual data with mvo tests; don't trust Java equals

* rework minimumViableOffsets to track sync states across template IDs

* extra note

* fix the tests to work against the stricter mvo

* move surrogatesToDomains call

* more tests for lagginess accumulator

* add changelog

CHANGELOG_BEGIN
- [JSON API] Under rare conditions, a multi-template query backed by database
  could have an ACS portion that doesn't match its transaction stream, if
  updated concurrently.  This conditions is now checked and accounted for.
  See `issue #10617 <https://github.com/digital-asset/daml/pull/10617>`__.
CHANGELOG_END

* port toSeq to Scala 2.12

* handle a corner case with offsets being too close to expected values

* didn't need XTag
2021-09-28 20:47:42 +00:00
..
adjustable-clock update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
auth-utils update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
build-info update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
concurrent Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
contextualized-logging [Ledger API error codes] ErrorCode enrichments [DPP-591] (#10874) 2021-09-15 07:43:32 +02:00
db-utils enable JSON search index on Oracle by default (#10539) 2021-08-31 09:29:36 +00:00
doobie-slf4j Add //libs-scala/doobie-slf4j and replace usages of jdkLogHandler (#8731) 2021-02-03 10:45:05 +00:00
flyway-testing Upgrade Flyway to v7. (#10594) 2021-08-17 10:29:13 +00:00
fs-utils Example export (#9756) 2021-05-21 09:36:25 +02:00
gatling-utils Upgrade Scalatest to v3.2.9. (#10576) 2021-08-12 23:19:35 +00:00
grpc-reverse-proxy Enhance grpc-test-utils and grpc-reverse-proxy to use Resources (#8606) 2021-01-25 13:27:48 +01:00
grpc-server-reflection-client Enhance grpc-test-utils and grpc-reverse-proxy to use Resources (#8606) 2021-01-25 13:27:48 +01:00
grpc-test-utils Upgrade Scalatest to v3.2.9. (#10576) 2021-08-12 23:19:35 +00:00
grpc-utils Do not drop details when converting between gRPC Status classes [KVL-1084] (#10745) 2021-09-06 12:22:15 +02:00
logging-entries Move logging value definitions alongside their objects. (#10439) 2021-07-29 09:21:27 +00:00
nameof Publish nameof library (#9798) 2021-05-26 09:58:02 +00:00
oracle-testing Introduce dependency on Oracle image version (#10597) 2021-08-17 13:28:26 +00:00
ports FreePort draw from outside ephemeral port range (#10774) 2021-09-07 15:41:43 +02:00
postgresql-testing FreePort draw from outside ephemeral port range (#10774) 2021-09-07 15:41:43 +02:00
resources resources: Only mutate test mock objects inside synchronized blocks. (#9869) 2021-06-01 12:50:42 +00:00
resources-akka participant-integration-api: Use Scheduler, and add more tests to configuration initialization. [KVL-1046] (#10461) 2021-08-03 14:49:00 +00:00
resources-grpc Restrict extensibility of ServerResourceOwner (#8660) 2021-01-28 10:47:28 +00:00
scala-utils detect unsynchronized contract table and retry (#10617) 2021-09-28 20:47:42 +00:00
scalatest-utils Upgrade Scalatest to v3.2.9. (#10576) 2021-08-12 23:19:35 +00:00
timer-utils timer-utils: If RetryStrategy eventually fails, explain what happened. (#10511) 2021-08-09 07:32:25 +00:00