Previously, each Protobuf source JAR was considered independent, and did
not require dependencies. This meant that the kvutils Protobuf sources
were published without the associated DAML-LF sources, making them
useless.
This change adds the source JARs as runtime dependencies to ensure that
when publishing a Protobuf source JAR, the release process will fail if
the dependencies are not also published.
CHANGELOG_BEGIN
- [Ledger API] We now publish Protobuf sources in JARs for DAML-LF
types to Maven Central, with the group "com.daml". The artifact names
follow the format "<name>_proto_jar".
CHANGELOG_END
CHANGELOG_BEGIN
- [Integration Kit] The kvutils Protobuf definition is now published to
Maven Central in a JAR, under the group "com.daml", with the artifact
name "participant-state-kvutils-proto".
CHANGELOG_END
* ledger-api: Use `proto_jars`.
CHANGELOG_BEGIN
- [Ledger API] The Scala JARs containing the gRPC definitions no longer
contain the *.proto files used to generate the ScalaPB-based classes.
CHANGELOG_END
* Create a source JAR for *.proto files in `proto_jars`.
* ledger-api: Publish the protobuf sources as "ledger-api-proto".
CHANGELOG_BEGIN
- [Ledger API] The *.proto files containing the gRPC definitions are now
provided by a new Maven Central artifact, with the group "com.daml"
and the artifact name "ledger-api-proto".
CHANGELOG_END
* release: We don't need the "main-jar" option.
* Bazel: Proto JARs will always have a Maven artifact suffix.
* Bazel: Simplify Protobuf source file TAR and JAR targets.
* Bazel: Extract out Protobuf functions.
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.
* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.
* Bazel: Factor out Java/Scala protobuf class generation into a helper.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/archive: Use `proto_jars`.
* Bazel: Remove the visibility modifier from `proto_jars`.
It's too confusing. Just make everything public.
* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.
* Bazel: Add comments to the various parts of `proto_jars`.
* daml-assistant: Do unpleasant things with `location` in Bazel.
* Document new integrity checker options
* Remove v2 from the bazel target in README
* Refactor integrity checker into published "tools" lib + binary
CHANGELOG_BEGIN
CHANGELOG_END
* Refine split
* Release tools lib
* Generalize IntegrityChecker.run
* Use glob for test source definition
* Widen glob for tools-tests to all (future) tool tests, not just integritycheck
* Simplify test source globs
* Simplify tools lib source glob
* Fix build
* resources: Move builders into //ledger/ledger-resources.
Keep the actual constructors in a trait, but instantiate it when working
with ledger code.
This allows us to later introduce an extra "context" type parameter to
ResourceOwner.
* resources-akka: Move the builders in to //ledger/ledger-resources.
* resources: Introduce an abstract `Context` parameter for owners.
This replaces the concrete `ExecutionContext`. While it _can_ be an
execution context, it really doesn't matter as long as we can get at one
somehow.
This is being introduced so we can wrap the context in a container,
either for type tagging or to include extra information.
Because our current context _is_ `ExecutionContext`, and an implicit is
provided to extract it, we can end up with two ways to get the same
value. We use shadowing to prevent this. This problem should go away in
the near future when a new context type is added.
CHANGELOG_BEGIN
- [Integration Kit] The `ResourceOwner` type is now parameterized by a
`Context`, which is filled in by the corresponding `Context` class in
the _ledger-resources_ dependency. This allows us to pass extra
information through resource acquisition.
CHANGELOG_END
* ledger-resources: Move `ResourceOwner` here from `resources`.
* ledger-resources: Remove dependencies from outside //ledger.
* ledger-resource: Wrap the acquisition execution context in `Context`.
So we can add a logging context to it.
* resources: Pass the Context, not the ExecutionContext, to Resource.
* Avoid importing `HasExecutionContext`.
* ledger-resources: Publish to Maven Central.
* resources: Make the small changes suggested by @stefanobaghino-da.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-resources: Pull out a trait for test resource contexts.
Saves a few lines of code.
* Restore some imports that were accidentally wildcarded.
* resources: Replace an `implicit def` with a couple of imports.
* participant-integration-api: Simplify the JdbcLedgerDaoBackend tests.
Try and use the right execution context where possible.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* conservatively move daml-script, trigger SValue interpreters to common library
* introduce expect and JavaList pattern for converters
* clean up trigger Converter Command interpretation
* add Church Free monad
* add an action language for trigger updates
* add expectE to remove some of the joins
* convert more of the converters to expect
* tool for unrolling Free/Roll
* split handleStepResult up and clean up its pattern
* handleStepFreeResult to interpret TriggerF
* replace Free Church with Pure/Roll free from Script
* newtype for ActionTrigger
* replace update in low-level Trigger with Free TriggerF
* submit one Commands at a time
* boolean blindness strikes again
* log missed TriggerF steps
* comment actual Submit contents
* match #7501 fromPureSExpr sig change in 00b80b8ea3
* avoid using forwardPort in runTrigger
* push State back into DAML, so it can be excluded from the action list
* push Message back into DAML, unifying the action language for initialState and update
* bringing TriggerF into initial state
* really add TriggerF into initial state, with all ports, tested
* add ActionTrigger class, express initialState in its terms
* add all TriggerF actions to existing TriggerA
* Trigger.rule will no longer have Time argument
* rename getS, setS to get, put, matching C.M.T.State from transformers
* make high-level Rule evaluate to the underlying TriggerF sequence
* Assert's testRule doesn't have a transform yet
* move DamlTuple2 to common converter library
- suggested by @cocreature; thanks
* combine the two Frees, provide from Script
* remove time argument from integration tests
CHANGELOG_BEGIN
- [Triggers] The ``Time`` argument was removed from the trigger rule function; instead, it
can be fetched within the ``TriggerA`` ``do`` block by ``getTime``, as with ``Update``
and ``Scenario``. The ``LowLevel`` trigger interface has been redesigned; such triggers
need to be rewritten or ported to high-level triggers.
See `issue #7456 <https://github.com/digital-asset/daml/pull/7456>`_.
CHANGELOG_END
* add trigger rule simulator to support Assert module
* missed new Free module
- left in script per @cocreature
* remove retract as we ended up using foldFree for that purpose instead
- suggested by @cocreature; thanks
* throw ConverterException instead of RuntimeException
- suggested by @cocreature; thanks
* remove Time argument from coin-upgrade-trigger
* port trigger service tests
* port trigger scenario test
* put TriggerSetup and TriggerRule into LowLevel.Trigger instead of unboxed Free
- suggested by @cocreature; thanks
* remove Time argument from trigger compatibility test
* submit commands as soon as each `emitCommands` is sequenced
- we still collect a list, but only for tracking commandsInFlight
* filter out compatibility tests for triggers before now
* remove commented imports, libraries from new shared converter
* make the TriggerF interpreter tail-recursive
* remove unused compatibility trait
* add back new state logging
* remove refactoring comment
* rewrite some LowLevel initialStates in do
* hide Daml.Script.Free from docs
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* remove forwardPortInitialState
- suggested by @cocreature; thanks
* manually port low-level updates
- suggested by @cocreature; thanks
* remove forwardPort
- suggested by @cocreature; thanks
* fail faster on unrecognized TriggerF
- suggested by @cocreature; thanks
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* concurrent: Tag DirectExecutionContext.
1. Tag `DirectExecutionContext` as `ExecutionContext[Nothing]`, thereby
stating that it works for any tagged `Future`.
2. Move `DirectExecutionContext` to the _libs-scala/concurrent_
library, as it requires it and it's tiny.
CHANGELOG_BEGIN
CHANGELOG_END
* concurrent: Fix the privacy of `DirectExecutionContextInternal`.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* add phantom-tagged ExecutionContext and Future to scala-utils concurrent package
* many new operations for Futures
* Future, ExecutionContext combinators from porting ledger-on-sql
- picked from 546b84ab9cdf4de2d93ec5682bdee6cfd6b385f8
* move Future, ExecutionContext companions into normal package
* lots of new docs
* many new Future utilities
* working zipWith
* tests for ExecutionContext resolution, showing what will be picked under different scenarios
* even more tests for ExecutionContext resolution
* tests showing some well-typed and ill-typed Future combinator usage
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* missed scalafmt
* one more doc note
* split concurrent package to concurrent library
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Moving `Statements.discard` from //ledger-server/http-json into //libs-scala/scala-utils
changelog_begin
changelog_end
* Add new module to the published artifacts
* `com.daml.scalautil` instead of `com.daml.scala.util`
@S11001001: That's because if this is in classpath and you import com.daml._,
you have a different scala in scope than the one you expect.
* Move public code into daml-integration-api
CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
This was never intentional, nobody even knew that this was possible
and we have an alternative, documented way of getting this via github
releases.
To avoid introducing this issue again, I’ve removed non-jar artifact
types from the Maven upload script.
fixes#448
changelog_begin
changelog_end
* Address https://github.com/digital-asset/daml/pull/6507#discussion_r446113575
* drop blindinginfo.proto
changelog_begin
changelog_end
* drop BlindingCoder
* Remove blindinginfo Protobuf definition JAR
changelog_begin
[DAML-LF] The blindinginfo Protobuf definition JAR, which was previously unused, has been pulled from the artifacts released on Maven
changelog_end
Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
* LF: rename library transaction-scalacheck to transaction-test-lib
CHANGELOG_BEGIN
CHANGELOG_END
* move files in com/daml
* missing change in release/artifacts.yaml
* remove 'com/dam' from the path
* equalz Scalatest matcher in new daml-lf/scalatest-tools library
* equalz typing tests
* a 'should' replacing design
* a 'MatcherFactory1' design
- this fails because the TC parameter should be a type member to avoid
scala/bug#5075 but it is not
* MatcherFactory1 with chained Lub+Equal typeclass
- requires partial-unification at point of use, which is not great
* LubEqual's extra tparam is probably unneeded
* better LtEqual
* demonstrate that HK LubEqual's resolve with DMT should + MatcherFactory
* remove unneeded 3rd param from LubEqual, again
* update dependency specs and license headers
* allow use with should, shouldNot in some cases, preserving the shouldx/shouldNotx alternatives
* move Equalz to libs-scala/scalatest-utils
* rename bzl targets and place in com.daml.scalatest package
* add scalatest-utils to release
* move *SpecCheckLaws, Unnatural to scalatest-utils
* missed scalacheck dep in scalatest-utils
* downstreams of *SpecCheckLaws now get them from scalatest-utils
* test equal-types case as well
* update LF documentation
CHANGELOG_BEGIN
CHANGELOG_END
* whitespace error
* Extract caching from participant-state as a library
This will be used to keep a cache of values to cut on LF translation cost when serving transactions.
changelog_begin
changelog_end
* Add dependency where missing
* factor TlsConfiguration parser from extractor
* move TlsConfigurationParser to new library
* link extractor to ledger-service/cli-opts properly
* use TlsConfigurationCli in http-json, pass SslContext to ledger-client
* test TLS options as used in http-json
- the TLS config code is shared with extractor, where it is more fully
tested; we just do a sanity check here
* doc TLS options for http-json
CHANGELOG_BEGIN
- [JSON API] New ``--pem``, ``--crt``, ``--cacrt``, and ``--tls`` options
for securing the connection between JSON API server and ledger.
See `issue #2540 <https://github.com/digital-asset/daml/issues/2540>`__.
CHANGELOG_END
* TLS off in daml-script JSON API test
* ledger/metrics: Move metric helpers to their own Bazel package.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use ledger/metrics.
* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.
Importing methods is harder than importing objects.
* metrics: Publish to Maven Central.
This PR removes the code for publishing to Bintray and updates the
documentation to point to github releases or Maven central.
I had to slightly change the docs formatting since trying to use
markup in code blocks results in a horrible layout and I don’t want to
fix that atm.
I’ve removed all artifacts that were only published to Bintray.
changelog_begin
changelog_end
* Use com.daml as groupId for all artifacts
CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END
* Add 2 additional maven related checks to the release binary
1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId
* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
* participant-state{,-index}: Move Timed*Service classes from Sandbox.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Metrics for the various read, write, and index
services.
CHANGELOG_END
* kvutils/app: Add timing metrics for read/write/index services.
* participant-state: Move metrics-related code to another Bazel package.
* participant-state-metrics: Add to artifacts.yml.
* participant-state-metrics: Move TimedIndexService back into Sandbox.
Cuts down on dependencies like nobody's business.
* Freeze DAML-LF 1.8 proto
This is a copy of the 1.dev proto with the following changes that
overall remove all references to 1.dev:
* Change the versions and the reference to the spec.
* Remove `dev` from version history at the top.
* Remove `GENMAP` and corresponding primitives.
* Remove general equality `EQUAL` and the comments that the
type-specific equality primitives are only available in < 1.dev
* Remove experimental text primitives.
changelog_begin
changelog_end
* windows is very bad
* bump windows hashes
* libs-scala/ports: Wrap socket ports in a type, `Port`.
* sandbox: Use `Port` for the API server port, and propagate.
CHANGELOG_BEGIN
CHANGELOG_END
* extractor: Use `Port` for the server port.
* ports: Make Port a compile-time class only.
* ports: Allow port 0; it can be specified by a user.
* ports: Publish to Maven Central.
I verified that there are no semantic changes, only cosmetic ones, by
running the following before and after, and checking the diff.
```
< release/artifacts.yaml ruby -rjson -ryaml -e 'puts JSON::generate YAML::load $stdin' | jq '. | sort_by(.target)' > ~/Desktop/artifacts.{old,new}.json
```
CHANGELOG_BEGIN
CHANGELOG_END
* resources: Allow for startup exceptions without no stack trace.
* sandbox: Remove dependency duplication in the BUILD file.
* ledger-on-(memory|sql): Extract `Main` into its own file tree.
This is so we can remove the dependency on kvutils/app from the library.
logback.xml goes with it.
* sandbox: Implement "Sandbox Next" on top of ledger-on-sql.
CHANGELOG_BEGIN
- [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML
Ledger Integration Kit. Currently not exposed through the CLI.
CHANGELOG_END
* ledger-on-sql: Publish to Maven Central.
* sandbox: If an unknown database is passed to Sandbox Next, handle it.
* resources: Catch errors in owner construction or acquisition.
Sometimes we're not in a future yet.
* resources: Don't run ProgramResource error handling inside the EC.
Otherwise we end up in deadlock which is only broken by
`awaitTermination` timing out.
* Remove damlc.jar
We never advertised or published this externally and our only internal
user has moved off this months ago already.
changelog_begin
changelog_end
* Remove dependency from navigator test lib on damlc jar
* libs-scala/resources: Move `Resource` and friends to their own package.
CHANGELOG_BEGIN
- [Ledger API Server] Publish the resource management code as a library
under ``com.digitalasset:resources``.
CHANGELOG_END
* libs-scala/resources: Add to the artifacts list.
* libs-scala/resources: Factor out the test helpers into test/lib.
* libs-scala/resources-akka: Make the Akka resources an optional dep.
* Moves CommandCompletionIT to Ledger API Test Tool
Closes#2841
Also refactors test observers into a single API and unifies the existing implementations of the DirectExecutionContext
* Allow explicit offsets to get checkpoints
* Raise checkpoint test timeout
* Consolidate checkpoint tests in a single test case
* Fix compilation issues
* Exclude command submission/completion tests for rev-v2
* Drop test not relevant for Ledger API implementations
* Fix compilation errors
* ledger-api-scala-logging: Fix errors in IntelliJ IDEA.
The Bazel plugin for IntelliJ doesn't seem to be smart enough to be able
to handle a Scala library that is part `src` directory and part
generated code from another Bazel rule. It just ignores the second part.
This means that IntelliJ cannot find the *ServiceLogging classes, as
they're not represented on the Bazel-generated classpath, and so
complains with lots of errors when working on the equivalent Api*Service
files.
To fix this, we can split these in two, compiling the base traits to
`ledger-api-scala-logging-base` and then the generated code separately.
It does result in an extra Bazel dependency for the users of
ledger-api-scala-logging, as Bazel doesn't realise transitive
dependencies for us.
* Release: Add `ledger-api-scala-logging-base` to the Maven list.
This still contains the main class so you can use it like you would
use the fat jar but publishing fat jars to maven central is apparently
bad practise and some peple have asked for the library.
This includes some slight tweaks to the scala_docs rule to make it
capable of coping with the generated source file and a hack in the
release script to avoid it complaining about the scenario proto
library not being published to Maven even though it is included in the
transitive deps.
* Spin off TokenHolder into a new library
Avoids having weird dependencies between different packages, makes TokenHolder available on Maven
* Fix auth-utils path
* ledger-api-test-tool: Increase the duration when watching health.
This should hopefully stop CI from flaking out.
* reference-v2/sandbox: Avoid unnecessary companion object constructors.
I like indirection… when it does something.
* ledger: Propagate empty health checks throughout the services.
* reference: Remove duplication from the ReferenceServer object.
* ledger-api-common: Actually query a "reporter" in the health service.
* ledger-api-common: Report health per-component when required.
* ledger-api-health: Use a Map to represent components for health checks.
* sandbox: Fix warnings in SqlLedgerSpec.
* ledger-api-common: Throw GrpcHealthService errors inside the Future.
* ledger: Implement health checks against the PostgreSQL connection.
Without proper testing, because I am not great at this.
* sandbox: Remove duplication and fix warnings in PostgresAround.
* sandbox: Test the SQL Ledger's health reporting on failure.
* sandbox: Don't report as unhealthy until 3 connections fail.
* ledger-api-health: Remove unused parts of the API.
Bit of premature design there.
* sandbox: Rename the "ledger" health check to "write".
* participant-state: Add the ReportsHealth trait to ReadService.
* ledger-api-common: `Future.fromTry(Try(…))` -> `Future(…)`.
* ledger-api-common: Make it clearer that StubReporter closes over health.
* ledger-api-common: Explain the HealthService watch tests with comments.
* sandbox: Clean up SqlLedger a bit.
* sandbox: Don't try and stop PostgreSQL twice in PostgresAround.
* bazel_tools: Windows rlocation lookups need to be with forward slashes.
* release: Fix case of "true".
* ledger-api-common: Make `GrpcHealthService::matchResponse` return a Try.
* ledger-api-common: Make `GrpcHealthServiceSpec` async.
* sandbox: Make a couple of DB classes final.
* sandbox: Avoid importing `X._` in PostgresAround.
* sandbox: Add clues to the SqlLedgerSpec's multiple assertions.
* sandbox: If PostgreSQL doesn't come back up, keep retrying.
* sandbox: Remove duplication in SqlLedgerSpec.
* sandbox: In SqlLedgerSpec, actually wait for the health to change.
* sandbox: In PostgresAround, make stopping PostgreSQL idempotent.
* sandbox: Simplify the SqlLedgerSpec to make it work on CI.
It's worth a shot.
* ledger-api-common: Simplify the GrpcHealthServiceSpec a little.
And add a changelog.
CHANGELOG_BEGIN
- [Ledger API Server] Add a health check endpoint conforming to the
`GRPC Health Checking Protocol <https://github.com/grpc/grpc/blob/master/doc/health-checking.md>`_.
- [Ledger API Server] Add health checks for index database connectivity.
- [Participant State API] Add a mandatory ``currentHealth()`` method to
``IndexService``, ``ReadService`` and ``WriteService``.
CHANGELOG_END
* sandbox: Improve the Javadoc layout for DbDispatcher.
* sandbox: Capitalize constants in SqlExecutor.
* ledger-api-health: Convert HealthStatus to an abstract class.
* Add participant-state configuration protobuf
- Move the "DamlConfiguration" from kvutils into participant-state/protobuf/ledger_configuration.proto.
- Add version number and spec (ala transaction.proto)
This is a preparation for indexing the configuration and having one canonical serialization for it.
* Initial thoughts on indexing the configuration
* Implement indexing of ledger configuration changes
* Add record time to all Updates. Wire through participant id.
and rename V7__Add_configuration to V8.
* Add ledger_configuration_java_proto to release artifacts
* Fix up release of ledger_configuration_java_proto
* Suggestions from review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* address rebase issues
* Resolve compilation errors after rebase/merge
* happy formatting for scala and bazel and friends
* Drop "openWorld" setting from configuration
And refactor the tests to allocate parties explicitly.
* Fix up migration and tests
* Drop authorizedParticipantIds from configuration
Implement configuration authorization in kvutils using the previous
the participant id of the previous configuration.
* Post-rebase fixes
* Add missing migrations
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Add missing mavenUpload to ledger-api-server
* Remove stateUpdateRecordTime
* Address code review
- Address PR review
- Merge TimeModelImpl and the traits. Remove TimeModel from ledger-api-common.
- Throw `Err` from KeyValueConsumption on parse errors instead of assert/sys.error
* Reformat
* Add missing protobuf file
* Fix compilation after TimeModel changes. Add version logs to participant-state{,-kvutils}.
* Fix TestUtil.scala build
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Address review
- synchronized access to ledgerConfiguration in InMemoryLedger
- store rejection if configuration generation is wrong
* Update ledger/participant-state/protobuf/ledger_configuration.rst
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Add authentication to Java identity client
Contributes to #3626
No changelog added right now, it will be added with the PR that
effectively closes the ticket.
Moves `LedgerCallCredentials` to a shared package.
Establishes the test framework for other Java clients.
* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731
* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731
* Rebase and fix compilation errors
* Fix type of jar in artifacts.yaml
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Fix more errors emerged after rebase
* Add dependency for DAML assistant ITs
* Add missing dependency
* Fix wrong dependency
* fix release
