* report UnknownTemplateIds warning before sending an error
if no template IDs resolved, send warning before the error, this is for
consistency with the case when at least one ID resolved (we continue
processing request in this case).
changelog_begin
changelog_end
* adding an error message example
* change title
* clarifying what on-message means.
* formatting
* Addressing code review comments
* http-json: Ask for a free port by specifying port 0.
This will avoid race conditions.
* bindings-akka-testing: Delete RandomPorts; it's unused.
* ports: Fix the Bazel test glob.
* ports: Move FreePort to postgresql-testing and add a test case.
* postgresql-testing: Make `FreePort.find()` return a `Port`.
* postgresql-testing: Lock free ports until the server starts.
This uses a `FileLock`, which should work well on all our
supported operating systems as long as everyone agrees to use it.
CHANGELOG_BEGIN
CHANGELOG_END
* postgresql-testing: Try to find a free port 10 times, then give up.
* postgresql-testing: Use a shared directory for the port lock.
* postgresql-testing: Try an alternative way of getting `%LOCALAPPDATA%`.
* minor refactoring in preparation for the next test case
* allow only one input request + test, WIP
* allow only one input request + test, WIP
* Error when multiple requests coming over the same websocket connection
This never worked, we never returned anything when 2nd request received,
now we return an error and drop the connection.
changelog_begin
changelog_end
* update docs
* Fixing copyright header
* changing test description
* trying to clean wsMessageHandler up
* Reverting getTransactionSourceForParty changes
* ContractKeyStreamRequest with different types for whether offset-preceded or not
- should replace EnrichedContractKey as the WS StreamQuery type
* add the ContractKeyStreamRequest layer everywhere
* split StreamQuery parse from the other steps
* make StreamQuery type depend on the input JsValue
* only StreamQueryReader is a typeclass now
* scalafmt
* wrong type arg
* letting the request data and phantom archive removal choice flow
- from work with @leo-da
* finish threading request-derived phantom state to removal flow
- from work with @leo-da
* make it clear that hints are contract IDs in StreamQuery
- from work with @leo-da
* treat StreamQueryReader's type parameter fully as a phantom
- from work with @leo-da
* remove unused type alias
- from work with @leo-da
* test fetch resume, with and without various offsetHints
* document offsetHints
* missing ` in rst
- thanks @hurryabit
* rename offsetHint to contractIdAtOffset
CHANGELOG_BEGIN
- [JSON API - Experimental] New field ``contractIdAtOffset`` for fetch-by-key streams
to restore proper archive filtering.
See `issue #4511 <https://github.com/digital-asset/daml/issues/4511>`_.
CHANGELOG_END
* we never unify the two ContractKeyStreamRequest types
* doc update for contractIdAtOffset
changelog_begin
[JSON API - Experimental]
Remove ``ledgerEffectiveTime`` and ``maximumRecordTime`` fields from command ``meta``. Remove ``--default-ttl`` command line argument. The ledger time and TTL are automatically computed by the submission service now. See #5090.
changelog_end
Contributes to #4194.
Closes#4231.
Closes#5022.
CHANGELOG_BEGIN
- [Ledger API] The protobuf fields ledger_effective_time and maximum_record_time have been removed from
command submission. These fields were previously deprecated following the introduction
of a new ledger time model. See issue `#4194 <https://github.com/digital-asset/daml/issues/4194>`__.
[Java Bindings] removed the usage of ledgerEffectiveTime and
maximumRecordTime, and instead added minLedgerTimeAbsolute and
minLedgerTimeRelative in CommandSubmissionClient and CommandClient
CHANGELOG_END
* building a GenMap generator
* move Party Order to accessible place; test whether Name Equal is still in use
* add Order instance for SortedLookupList
* switch to Map for genMap's Inj; more Order instances
* remove all Order from TypedValueGenerators
* Revert "add Order instance for SortedLookupList"
This reverts commit 03a59a8249.
* moving the Equal instance means scenario-interpreter no longer direct-deps scalaz
* add a test using TypedValueGenerators GenMap
* remove Party Order
* refmt bazel
* remove stray import
* followup dep change to moving the Equal[Name] instance
* add changelog
CHANGELOG_BEGIN
CHANGELOG_END
* Name equal instance appears to be no longer used, but keeping anyway
* Streaming API error handling
* Stopping the stream after emitting the 1st error
* Cleaning up test cases, stricter type for non empty streaming fetch
* Update docs.
changelog_begin
[JSON API - Experimental] Change Streaming API error format to match synchronous API:
``{"status": <400 | 401 | 404 | 500>, "errors": <JSON array of strings> }``. See #4521
changelog_end
* a few words about warnings in the Streaming API
* a few words about warnings in the Streaming API
* Updating command formatting in the JavaScript example.
* remove List and TextMap support from the JSON query language
CHANGELOG_BEGIN
- [JSON API - Experimental] Removed support for maps and lists from the query language.
See `issue #4855 <https://github.com/digital-asset/daml/issues/4855>`_.
CHANGELOG_END
* GenMap won't be supported in general, remove FIXME
- thanks @leo-da
* more documentation on unsupported query types
- as suggested by @hurryabit; thanks
* Bumping up http-json tests size to large, adding some extra logging
* Removing the logging from prod code, adding some in tests.
changelog_begin
changelog_end
Avoiding `damlc compile/package` commands (which we would like to deprecate), and replace with plain `damlc build` together with a post dar->dalf extraction step in the couple of places where we actually want the .dalf for testing.
changelog_begin
changelog_end
* 2nd part of stabilizing flaky websocket integration test,
handling empty events, which we may receive a bunch in the end of the test
when there is no activity
changelog_begin
changelog_end
* fixing `Offset.semigroup`, thanks @S11001001!
* Allocate party JSON API endpoint
changelog_begin
[JSON API - Experimental]
Allocate party endpoint added: ``/v1/parties/allocate``. See #4638.
changelog_end
* cleanup
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/LedgerClientJwt.scala
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* capture the pattern of an optional stream input prefix
* allow LiveBegins to be appended
- should have been included with #4593
* StreamQuery will parse JsValues instead of Strings
* shortcut in HttpServiceTestFixture; clean up InsertDeleteStepTest
* compatibly add withOptPrefix stage to WS input
* parsing the offset argument
* add offset parsing to input stream, but drop it on the floor
* another form of the two cases
* percolate the starting offset to the next step
* percolate the starting offset to the next step
* percolate the starting offset to the next step
* use transactionsFollowingBoundary in ContractsService if begin offset specified
* integration test for scan from offset in WS query
* tool for catching the first offset in a stream
* documentation
* add changelog entry
CHANGELOG_BEGIN
- [JSON API - Experimental] WebSocket endpoints now support an optional offset argument.
See `issue #4509 <https://github.com/digital-asset/daml/issues/4509>`_.
CHANGELOG_END
* move offset argument doc to later
- suggested by @hurryabit; thanks
* remove stray TODO
- pointed out by @leo-da; thanks
* redesign withOptPrefix to make the usage a little easier to follow
- suggested by @leo-da; thanks
* Capture lastSeenOffset in the @volatile var
CHANGELOG_BEGIN
[JSON API - Experimental] Websocket stream now emits last seen offset instead of the heartbeat message.
``{"heartbeat": "ping"}`` is replaced by ``{"events":[],"offset":"<last seen offset>"}``. See #4510.
CHANGELOG_END
* updating docs
* moving the last seen offset into the stream, WIP
* adding in-stream state
* minor docs
* cleanup the heartbeat logic
* minot cleanup
* Change live and heartbeat msg handling + some debug logging (to be removed)
* fixing ts tests, cleaning up
* Adding todo with the reference to the follow-up ticket
* Adding todo with the reference to the follow-up ticket
Both were previously binding 0.0.0.0, which is inherently insecure. More
importantly to me, that meant running `bazel test //...` essentially
rendered my computer unusable for however long it took (which is
_long_), as it kept popping up focus-stealing dialogs about whether or
not I wanted to trust "java" to open an incoming network connection.
The ScenarioService does not seem to have an existing setup for CLI args
and my Scala-fu is not good enough to add one, so I just changed the
hard-coded path.
The JSON API already had an option, just with the wrong default. This is
technically a breaking change, but I'm hoping to pass it under the
"experimental" flag we still have on the JSON API.
CHANGELOG_BEGIN
- [JSON API - Experimental] As a security improvement, the JSON API
server will now bind on ``127.0.0.1`` by default. Previous behaviour was
to bind on ``0.0.0.0``; you can get that behaviour back by passing in
the (existing) flag ``--address 0.0.0.0``.
- [DAML SDK] The Scenario Service will now bind on ``127.0.0.1``. Previous
behaviour was to bind on ``0.0.0.0``.
CHANGELOG_END
* make append of two liveness markers proper with respect to offsets
- never happens in a real stream, but satisfying some random tests
- test randomly harder
CHANGELOG_BEGIN
CHANGELOG_END
* move Monoid and Semigroup instances for Steps to main code
- suggested by @leo-da; thanks
* stray empty InsertDeleteStep
* Filter out contracts received from gRPC stream, if template IDs is unknown (pending package reload).
* minor cleanup
CHANGELOG_BEGIN
[JSON API - Experimental] Fix ``java.util.NoSuchElementException: key not found: TemplateId(xxx,yyy,zzz)`` error in the log for query and fetch endpoints. See #4761
CHANGELOG_END
* libs-scala/ports: Wrap socket ports in a type, `Port`.
* sandbox: Use `Port` for the API server port, and propagate.
CHANGELOG_BEGIN
CHANGELOG_END
* extractor: Use `Port` for the server port.
* ports: Make Port a compile-time class only.
* ports: Allow port 0; it can be specified by a user.
* ports: Publish to Maven Central.
* Extract ErrorOps, use liftErr instead of leftMap
JSON error formatting cleanup
CHANGELOG_BEGIN
CHANGELOG_END
* Good we have tests for this stuff
* Apply https://github.com/scala/bug/issues/3664 work-around,
so JsonError can be used instead of JsonError.apply
* error formatting
* move BeginBookmark to util
* adding offsets to steps
* offsetAfter belongs in Txn, not InsertDeleteStep
* make transaction stream a ContractStreamStep.Txn stream
* add several ContractStreamStep append cases
* rewrite 'render' to emit offset in the right places
* make ContractStreamStep#append total again
* check for offset in a few tests
* revert useless whitespace changes
* missed argument
* simpler mapPreservingIds
* rewrite states for new "live" format
* remove invalidated "events" block structure assertions
* make shutdown in withHttpService deterministic, to try to catch race condition
* exhaustiveness checking somehow disabled; fixed fetch flow and all is well
* documentation and changelog
CHANGELOG_BEGIN
- [JSON API - Experimental] Remove ``{"live": true}`` marker from websocket streams;
instead, live data is indicated by the presence of an "offset".
See `issue #4593 <https://github.com/digital-asset/daml/pull/4593>`_.
CHANGELOG_END
* be more specific about what liveness marker may be in docs
* fix daml2ts websocket tests
* mention type rules for all cases in offset documentation
* Extend /party endpoint to allow specifying party ids
* Extend /party endpoint to allow specifying party ids
* Update docs
CHANGELOG_BEGIN
[JSON API - Experimental] Fetch Parties by their Identifiers. See #4512
``/v1/parties`` endpoint supports POST method now, which expects
a JSON array of party identifiers as an input.
CHANGELOG_END
* minor update
* minor update
* Use type alias
* Add warnings to the sync response
* test cases
* update docs, add test case for an empty input
* cleanup
* cleanup
* Addressing code review comments
* Avoid opening a server to the world when finding a free port.
This is very annoying on macOS because we get a focus-stealing popup for
a split second, asking for permission to allow the server through the
firewall. The popup pretty much always disappears before it can even be
read, when the server is closed.
This is almost certainly not an attack vector, because:
- we only do this in tests,
- the server is open for only a few milliseconds,
- nothing is served,
- and finding the port is tricky, because it's effectively random.
Nevertheless, it's very annoying.
CHANGELOG_BEGIN
CHANGELOG_END
* Extract a Bazel package for finding free ports.
We seem to do it in 4 different places, which I think is enough to
remove the duplication.
* kvutils: Extract a committer from the uses of `SubmissionValidator`.
This makes the clock injectable too.
* kvutils: Provide logging contexts in the `Runner`.
* sandbox: Remove the `StaticAllowBackwards` time provider type.
It's not used anywhere.
* sandbox: Fix warnings in CliSpec.
* sandbox: Ensure that we cannot specify both static and wall-clock time.
* sandbox-next: Crash if wall clock time is not specified.
* sandbox-next: Document more known issues in the new Sandbox.
* sandbox: Add a Clock (and some tests) to TimeServiceBackend.
* sandbox-next: Support static time.
CHANGELOG_BEGIN
- [Sandbox Next] Re-establish static time mode.
CHANGELOG_END
* ledger-on-(memory|sql): Expect a `() => Instant`, not a `Clock`.
* Inserts alias
* ContractStreamStep, extending InsertDeleteStep with a boundary
* partitionBimap, an IDS utility for partitionMap on both inserts and deletes
* make the flow setup of acsFollowingAndBoundary a little clearer
* use partitionBimap to simplify websocket response stream
* make the flow setup of transactionsFollowingBoundary a little clearer
* acsFollowingAndBoundary becoming a ContractStreamStep producer
* insertDeleteStepSource also becomes a ContractStreamStep producer
* porting StepAndErrors to ContractStreamStep
* remove Acs constructor of ContractStreamStep
- it's an interesting idea for future potential features, but not useful
right now so just a needless complication
* adapt convertFilterContracts to presence of LiveBegin marker
* adapt removePhantomArchives to presence of LiveBegin marker
* test that live marker is emitted in the right place
* document liveness marker
* add changelog
CHANGELOG_BEGIN
- [JSON API - Experimental] Add ``{"live": true}`` to WebSocket streams
to mark the beginning of "live" data. See `issue #4461
<https://github.com/digital-asset/daml/issues/4461>`_.
This marker is a placeholder feature; `issue #4509 bookmarks in query
streams <https://github.com/digital-asset/daml/issues/4509>`_ will
obsolete this marker, after which it will no longer be emitted. When
building features on the marker, be aware of this forthcoming
replacement.
CHANGELOG_END
* be a little more prepared for offsets
* sandbox: Move more resource acquisition into the `owner`.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Reimplement SandboxClientResource as a resources.Resource.
* codegen: Use resources in TestUtil.
* sandbox: Manage PostgreSQL in tests with ResourceOwners.
* Extract partial functions for DABL to re-use
CHANGELOG_BEGIN
CHANGELOG_END
* Actually DABL needs total functions,
they will map them to different endpoints
* move InsertDeleteStep to util
* turn append cid argument into a typeclass
* use Cid for appendForgettingDeletes as well
* add secondary data to the deletes in InsertDeleteStep with new covariant tparam
* make StepAndErrors hold ArchivedContracts so ws archives look like exercise archives
* add changelog
CHANGELOG_BEGIN
- [JSON API - Experimental] The format of ``archived`` responses from WebSocket endpoints
has changed to include template IDs, similar to exercise responses.
See `issue #4383 <https://github.com/digital-asset/daml/issues/4383>`_.
CHANGELOG_END
* rename "contracts" to "events" in JSON API exercise response
CHANGELOG_BEGIN
- [JSON API - Experimental] Exercise response field "contracts" renamed to "events".
See `issue #4385 <https://github.com/digital-asset/daml/issues/4385>`_.
CHANGELOG_END
* more events in doc
- pointed out by @leo-da; thanks
* Remove phantom archive when streaming events filtered by contract key
* Add fetch test cases, WIP
* Add fetch test cases, WIP
* Add fetch test cases, WIP
* Add fetch test cases, WIP
* Test case to validate phantom archive filter for stream/fetch
* Use `scan` instead of `statefulMapConcat` to filter out phantom archives
advantage(???) immutable state vs mutable
* make `com.digitalasset.http.WebsocketEndpoints.handleWebsocketRequest` public
so DABL can call it directly, `private[http]` does not work since DABL
is under com.projectdabl, see #4190
* Fixing typos
* Scalafmt
* Scalafmt
* Removing unused function
* Update docs
* Remove unused type alias
* CHANGELOG_BEGIN
[JSON API - Experimental]
- Added streaming version of fetch by key: ``/stream/fetch``. See #4075.
CHANGELOG_END
* Address code review comments
* Add test case with todo to address consistent handling of empty requests
* Fixing merge conflicts
* Addressing code review comments
* Addressing code review comments
Use `partition` instead of consecutive intersect and diff
[JSON API - Experimental]
Added a command line option to allow overriding default TTL.
``--default-ttl <value>`` Optional Time to Live interval to set if not provided in the command. Examples: 30s, 1min, 1h. Defaults to 30 seconds
CHANGELOG_END
* add SearchForeverRequest with one-or-many JSON model
* the least structured way to flatten a query union into a single stream
* add somewhere for indices to go in StreamQuery
* unzipMap utility
* doc Positive
* collect indices for SearchForever predicate, and use only one Map
* interface for rendering positions
* finish propagating positives from the predicate to the rendering phase
* add matchedQueries to every `created` in the searchForever results
* test that matchedQueries indices are included in query stream blocks
* document query union
* add changelog
CHANGELOG_BEGIN
- [JSON API - Experimental] ``/contracts/searchForever`` accepts multiple queries,
and includes with each ``created`` result the ``matchedQueries`` indicating which
queries matched.
See `issue #4363 <https://github.com/digital-asset/daml/pull/4363>`_.
CHANGELOG_END
* remove unused unzipMap
* ensure archive of a key happens to left of create of same key in each WebSocket result block
CHANGELOG_BEGIN
- [JSON API - Experimental] In websocket endpoints, if a 'created' and 'archived' contract
in the same result array share a contract key, the 'archived' is guaranteed to occur
earlier in the array than the 'created'.
See `issue #4354 <https://github.com/digital-asset/daml/issues/4354>`_.
CHANGELOG_END
* by key I mean contract key
- suggested by @hurryabit; thanks
* in searchForever, warn on unknown template IDs as long as at least one is known
* remove unused resolveTemplateId
* factor out WS request parts in WS integration test
* factor out IOU create
* test early template ID warning in searchForever stream
* document warnings case for searchForever
CHANGELOG_BEGIN
- [JSON API - Experimental] Precede stream with warnings of unknown template IDs, if any,
rather than failing outright.
See `issue #4290 <https://github.com/digital-asset/daml/issues/4290>`_.
CHANGELOG_END
* Fix the case when start offset > end offset
Actually skip the call completely, all needed data supposed to be
already in the query-store, fetched by a concurrent thread/client.
Moving the check into the `getCreatesAndArchivesSince` call
CHANGELOG_BEGIN
CHANGELOG_END
* Remove flaky, want to see the test failing.
* reformat
* Add AbsoluteOffsetOrdering so we can handle both offset formats
* Code review comments
* generators WIP
* generators WIP
* testing the laws
* remove `implicitly` call