Commit Graph

2 Commits

Author SHA1 Message Date
Leonid Shlyapnikov
cb86dc4029
Retry on PostgreSQL UNIQUE_VIOLATION (#3425)
* Retry on UNIQUE_VIOLATION

* Update release notes

* Hiding method that is not supposed to be used directly

* using connection.raiseError

* Force re-run on stale offset update
2019-11-12 14:49:13 -05:00
Stephen Compall
d0ec381925 json-api: store contracts similar to Extractor's single-table (#3192)
* new library ledger-service/db-backend

* borrow contracts table schema from extractor

* borrow contract insertion, removing some data to be unused

* match contract schema with insert function

* factor insertContract arguments

* offset table declarations

* CLI argument for query store

* surrogate template IDs

* compute surrogate template IDs on-the-fly

* database init action

* incoherent typeclasses, eh

* newtype SurrogateTpId

* offset fetch/update functions

* bad sql

* bulk insert contracts, function for selecting contracts

* expose contract column name for query's usage

* Initializing DB on startup if configured

* dropping existing tables as part of initialization

* fix some query syntax errors

* createSchema flag

* function for streaming transactions with jwt party selected

* formatting

* usage

* collect acs contracts and the ledger offset at the end

* lastOffset

* fixing merge conflicts, updating the way 3rd party deps are specified

* Moving ContractDao into http-json module

so it can take domain AST as an input

* cleanup

* injecting new dependencies

* split transaction batches into inserts and deletes

* generate sql for deleting contracts

* `fetch_sources = True` for java_deps

* make the delete-constructed fragment more efficient; handle empty list here

* pass logHandler for insertContracts

* ContractDao returns ConnectionIO, it's up to the caller to wrap query into a transaction

* fixing typo

* minor cleanup, moving fromLedgerApi factory function into corresponding companion objects

* don't need it any more

* GetActiveContractsResponse => domain.Contract factory

* make concatFragment private

* add partition graph; move other contract-fetching experiments to ContractsFetch

* experimenting with akka sources

* introducing domain.Offset to work around API's empty/null offset cases

* minor cleanup

* decompose fetchActiveContractsFromOffset

* missed via

* ACS splitting graph

* finish doc for ACS splitting graph

* remove unneeded stages

* WIP

* lazily read a stream of ConnectionIO into a single ConnectionIO

* cancel on IO error

* figuring out how to put all the pieces together

* graph WIP

* Removing workflowId from the JSON API

* simplify acsAndBoundary; describe other flow pieces

* WIP

* use Vector in InsertDeleteStep; add variant for ACS (no deletes)

* `org.wartremover.warts.NonUnitStatements` enforced in `http-json` module

* evaluate InsertDeleteStep to a ConnectionIO

* database variant of LfValueCodec, using numbers for numbers

* convert input to JSON, combine insert plans, connect rest of contractsToOffsetIo

* remove strict contractsToOffset sink

* moving dao methods into an object

* putting pieces together

* contractsFromOffset WIP

* should be it

* cleanup

* cleanup

* contractsIo that takes List[domain.TemplateId.RequiredPkg])

* contractsIo that takes List[domain.TemplateId.RequiredPkg])

* cleanup

* put all pieces together, testing

something does not work yet

* diff is not required to return anything

that is why Sink.lastOption that gives Option[domain.Offset]

* factor out tuple split

* use traverse syntax in contractsIo2

* factor explicit flow steps out of graph DSL; remove aggregate

* locally model the Absolute/Begin distinction for offset bookmarking in DB

* Adding test cases to run HTTP Service with Postgres backend

the same set of test cases, run with and without DB backend

* make better use of domain.Offset in OffsetBoundary

* monomorphize InsertDeleteStep#append

* Disabling a test that fails with DB backend

* add release note

* add release note about workflowId

* a test case that checks the number or stored contracts

* trying to figure out why Postgres test fails on Windows with NPE
2019-11-06 13:53:16 -05:00