* Freeze DAML-LF 1.8 proto
This is a copy of the 1.dev proto with the following changes that
overall remove all references to 1.dev:
* Change the versions and the reference to the spec.
* Remove `dev` from version history at the top.
* Remove `GENMAP` and corresponding primitives.
* Remove general equality `EQUAL` and the comments that the
type-specific equality primitives are only available in < 1.dev
* Remove experimental text primitives.
changelog_begin
changelog_end
* windows is very bad
* bump windows hashes
* libs-scala/ports: Wrap socket ports in a type, `Port`.
* sandbox: Use `Port` for the API server port, and propagate.
CHANGELOG_BEGIN
CHANGELOG_END
* extractor: Use `Port` for the server port.
* ports: Make Port a compile-time class only.
* ports: Allow port 0; it can be specified by a user.
* ports: Publish to Maven Central.
I verified that there are no semantic changes, only cosmetic ones, by
running the following before and after, and checking the diff.
```
< release/artifacts.yaml ruby -rjson -ryaml -e 'puts JSON::generate YAML::load $stdin' | jq '. | sort_by(.target)' > ~/Desktop/artifacts.{old,new}.json
```
CHANGELOG_BEGIN
CHANGELOG_END
* resources: Allow for startup exceptions without no stack trace.
* sandbox: Remove dependency duplication in the BUILD file.
* ledger-on-(memory|sql): Extract `Main` into its own file tree.
This is so we can remove the dependency on kvutils/app from the library.
logback.xml goes with it.
* sandbox: Implement "Sandbox Next" on top of ledger-on-sql.
CHANGELOG_BEGIN
- [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML
Ledger Integration Kit. Currently not exposed through the CLI.
CHANGELOG_END
* ledger-on-sql: Publish to Maven Central.
* sandbox: If an unknown database is passed to Sandbox Next, handle it.
* resources: Catch errors in owner construction or acquisition.
Sometimes we're not in a future yet.
* resources: Don't run ProgramResource error handling inside the EC.
Otherwise we end up in deadlock which is only broken by
`awaitTermination` timing out.
* Remove damlc.jar
We never advertised or published this externally and our only internal
user has moved off this months ago already.
changelog_begin
changelog_end
* Remove dependency from navigator test lib on damlc jar
* libs-scala/resources: Move `Resource` and friends to their own package.
CHANGELOG_BEGIN
- [Ledger API Server] Publish the resource management code as a library
under ``com.digitalasset:resources``.
CHANGELOG_END
* libs-scala/resources: Add to the artifacts list.
* libs-scala/resources: Factor out the test helpers into test/lib.
* libs-scala/resources-akka: Make the Akka resources an optional dep.
* Moves CommandCompletionIT to Ledger API Test Tool
Closes#2841
Also refactors test observers into a single API and unifies the existing implementations of the DirectExecutionContext
* Allow explicit offsets to get checkpoints
* Raise checkpoint test timeout
* Consolidate checkpoint tests in a single test case
* Fix compilation issues
* Exclude command submission/completion tests for rev-v2
* Drop test not relevant for Ledger API implementations
* Fix compilation errors
* ledger-api-scala-logging: Fix errors in IntelliJ IDEA.
The Bazel plugin for IntelliJ doesn't seem to be smart enough to be able
to handle a Scala library that is part `src` directory and part
generated code from another Bazel rule. It just ignores the second part.
This means that IntelliJ cannot find the *ServiceLogging classes, as
they're not represented on the Bazel-generated classpath, and so
complains with lots of errors when working on the equivalent Api*Service
files.
To fix this, we can split these in two, compiling the base traits to
`ledger-api-scala-logging-base` and then the generated code separately.
It does result in an extra Bazel dependency for the users of
ledger-api-scala-logging, as Bazel doesn't realise transitive
dependencies for us.
* Release: Add `ledger-api-scala-logging-base` to the Maven list.
This still contains the main class so you can use it like you would
use the fat jar but publishing fat jars to maven central is apparently
bad practise and some peple have asked for the library.
This includes some slight tweaks to the scala_docs rule to make it
capable of coping with the generated source file and a hack in the
release script to avoid it complaining about the scenario proto
library not being published to Maven even though it is included in the
transitive deps.
* Spin off TokenHolder into a new library
Avoids having weird dependencies between different packages, makes TokenHolder available on Maven
* Fix auth-utils path
* ledger-api-test-tool: Increase the duration when watching health.
This should hopefully stop CI from flaking out.
* reference-v2/sandbox: Avoid unnecessary companion object constructors.
I like indirection… when it does something.
* ledger: Propagate empty health checks throughout the services.
* reference: Remove duplication from the ReferenceServer object.
* ledger-api-common: Actually query a "reporter" in the health service.
* ledger-api-common: Report health per-component when required.
* ledger-api-health: Use a Map to represent components for health checks.
* sandbox: Fix warnings in SqlLedgerSpec.
* ledger-api-common: Throw GrpcHealthService errors inside the Future.
* ledger: Implement health checks against the PostgreSQL connection.
Without proper testing, because I am not great at this.
* sandbox: Remove duplication and fix warnings in PostgresAround.
* sandbox: Test the SQL Ledger's health reporting on failure.
* sandbox: Don't report as unhealthy until 3 connections fail.
* ledger-api-health: Remove unused parts of the API.
Bit of premature design there.
* sandbox: Rename the "ledger" health check to "write".
* participant-state: Add the ReportsHealth trait to ReadService.
* ledger-api-common: `Future.fromTry(Try(…))` -> `Future(…)`.
* ledger-api-common: Make it clearer that StubReporter closes over health.
* ledger-api-common: Explain the HealthService watch tests with comments.
* sandbox: Clean up SqlLedger a bit.
* sandbox: Don't try and stop PostgreSQL twice in PostgresAround.
* bazel_tools: Windows rlocation lookups need to be with forward slashes.
* release: Fix case of "true".
* ledger-api-common: Make `GrpcHealthService::matchResponse` return a Try.
* ledger-api-common: Make `GrpcHealthServiceSpec` async.
* sandbox: Make a couple of DB classes final.
* sandbox: Avoid importing `X._` in PostgresAround.
* sandbox: Add clues to the SqlLedgerSpec's multiple assertions.
* sandbox: If PostgreSQL doesn't come back up, keep retrying.
* sandbox: Remove duplication in SqlLedgerSpec.
* sandbox: In SqlLedgerSpec, actually wait for the health to change.
* sandbox: In PostgresAround, make stopping PostgreSQL idempotent.
* sandbox: Simplify the SqlLedgerSpec to make it work on CI.
It's worth a shot.
* ledger-api-common: Simplify the GrpcHealthServiceSpec a little.
And add a changelog.
CHANGELOG_BEGIN
- [Ledger API Server] Add a health check endpoint conforming to the
`GRPC Health Checking Protocol <https://github.com/grpc/grpc/blob/master/doc/health-checking.md>`_.
- [Ledger API Server] Add health checks for index database connectivity.
- [Participant State API] Add a mandatory ``currentHealth()`` method to
``IndexService``, ``ReadService`` and ``WriteService``.
CHANGELOG_END
* sandbox: Improve the Javadoc layout for DbDispatcher.
* sandbox: Capitalize constants in SqlExecutor.
* ledger-api-health: Convert HealthStatus to an abstract class.
* Add participant-state configuration protobuf
- Move the "DamlConfiguration" from kvutils into participant-state/protobuf/ledger_configuration.proto.
- Add version number and spec (ala transaction.proto)
This is a preparation for indexing the configuration and having one canonical serialization for it.
* Initial thoughts on indexing the configuration
* Implement indexing of ledger configuration changes
* Add record time to all Updates. Wire through participant id.
and rename V7__Add_configuration to V8.
* Add ledger_configuration_java_proto to release artifacts
* Fix up release of ledger_configuration_java_proto
* Suggestions from review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* address rebase issues
* Resolve compilation errors after rebase/merge
* happy formatting for scala and bazel and friends
* Drop "openWorld" setting from configuration
And refactor the tests to allocate parties explicitly.
* Fix up migration and tests
* Drop authorizedParticipantIds from configuration
Implement configuration authorization in kvutils using the previous
the participant id of the previous configuration.
* Post-rebase fixes
* Add missing migrations
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Add missing mavenUpload to ledger-api-server
* Remove stateUpdateRecordTime
* Address code review
- Address PR review
- Merge TimeModelImpl and the traits. Remove TimeModel from ledger-api-common.
- Throw `Err` from KeyValueConsumption on parse errors instead of assert/sys.error
* Reformat
* Add missing protobuf file
* Fix compilation after TimeModel changes. Add version logs to participant-state{,-kvutils}.
* Fix TestUtil.scala build
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Address review
- synchronized access to ledgerConfiguration in InMemoryLedger
- store rejection if configuration generation is wrong
* Update ledger/participant-state/protobuf/ledger_configuration.rst
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Add authentication to Java identity client
Contributes to #3626
No changelog added right now, it will be added with the PR that
effectively closes the ticket.
Moves `LedgerCallCredentials` to a shared package.
Establishes the test framework for other Java clients.
* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731
* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731
* Rebase and fix compilation errors
* Fix type of jar in artifacts.yaml
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Fix more errors emerged after rebase
* Add dependency for DAML assistant ITs
* Add missing dependency
* Fix wrong dependency
* fix release
* grpc-definitions: Delete health_service.proto
We can use the version in io.grpc:grpc-services instead.
* ledger: Delete ledger/API.md.
* sandbox: Fix warnings in ApiServices flagged by IntelliJ.
* sandbox: Implement a dummy grpc.health.v1.Health.Check endpoint.
* sandbox: Implement a dummy grpc.health.v1.Health.Watch endpoint.
* sandbox: Drop repeated elements from grpc.health.v1.Health.Watch.
* sandbox: Wrap the HealthService in basic tests.
* sandbox: Stop streaming the server health too.
* ledger-api-test-tool: Health check tests.
* Add a changelog entry for the health check endpoints.
CHANGELOG_BEGIN
- [Ledger API] Add healthcheck endpoints, conforming to the
`GRPC Health Checking Protocol <https://github.com/grpc/grpc/blob/master/doc/health-checking.md>`_.
It is always ``SERVING`` for now.
- [DAML Ledger Integration Kit] Add conformance test coverage for the
``grpc.health.v1.Health`` service.
CHANGELOG_END
* ledger-api-integration-tests: Increment the number of services.
* Apply suggestions from code review
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* sandbox: Use `AkkaBeforeAndAfterAll` in the HealthServiceSpec.
In an attempt to get it working on CI.
* sandbox: Change `dropRepeated` to `DropRepeated()`.
Keep it in one file.
* test-common: Use `Delayed.by` in `TimeBoundObserver`.
* test-common: Close the source when `TimeBoundObserver` completes.
* ./fmt.sh
That'll teach me not to `--no-verify` just because it's a merge commit.
* sandbox: Inline `HealthService.suppress`.
At some point it was being used twice.
* sandbox: Increase the timeout for HealthServiceSpec.
* sandbox: Reimplement HealthService using the Scala protobuf types.
* sandbox: Generate an Akka-compatible trait for the health service.
And refactor a lot of test code to make it easy to test.
* ledger-api-common: Move the HealthService here.
* rs-grpc-testing-utils: Publish to Maven.
* rs-grpc-testing-utils: Add Maven coordinates.
* Add helper to produce an empty zip files.
This is used to generate empty sources and javadoc jars for
deploy jars later on.
* Create empty auxilliary jars.
da_java_binary:
- empty javadoc jar
- empty sources jar
da_java_proto_library:
- empty javadoc jar
- the sources jar is automatically generated by java_proto_library as a side effect
da_scala_binary:
- empty javadoc jar
- empty sources jar
* Support maven upload for jar-deploy and jar-proto
For jar-deploy targets we don't check for internal dependencies,
because these should already be contained in the (fat-)jar itself.
Additionally, the release program now uploads javadocs and sources
for jar-proto and jar-deploy as well to comply with maven central.
* Upload ledger-api-test-tool and kvutils + dependencies to maven central.
This is the diff running the output of the release without and with these changes.
A few artifacts now also get their javadoc and sources uploaded (mostly to bintray,
but now they are ready for a maven central upload).
ledger-api-test-tool has the scala version removed from the artifact as it is a
deploy jar and nobody should care which specific scala version is used.
Only in release/com/daml/ledger/participant-state-kvutils-java-proto/100.13.35: participant-state-kvutils-java-proto-100.13.35-javadoc.jar
Only in release/com/daml/ledger/participant-state-kvutils-java-proto/100.13.35: participant-state-kvutils-java-proto-100.13.35-sources.jar
Only in release/com/daml/ledger/testtool: ledger-api-test-tool
Only in release-before/com/daml/ledger/testtool: ledger-api-test-tool_2.12
Only in release/com/digitalasset/daml/lf/engine/trigger/runner_2.12/100.13.35: runner_2.12-100.13.35-javadoc.jar
Only in release/com/digitalasset/daml/lf/engine/trigger/runner_2.12/100.13.35: runner_2.12-100.13.35-sources.jar
Only in release/com/digitalasset/daml-lf-blindinginfo-java-proto/100.13.35: daml-lf-blindinginfo-java-proto-100.13.35-javadoc.jar
Only in release/com/digitalasset/daml-lf-blindinginfo-java-proto/100.13.35: daml-lf-blindinginfo-java-proto-100.13.35-sources.jar
Only in release/com/digitalasset/daml-lf-transaction-java-proto/100.13.35: daml-lf-transaction-java-proto-100.13.35-javadoc.jar
Only in release/com/digitalasset/daml-lf-transaction-java-proto/100.13.35: daml-lf-transaction-java-proto-100.13.35-sources.jar
Only in release/com/digitalasset/daml-lf-value-java-proto/100.13.35: daml-lf-value-java-proto-100.13.35-javadoc.jar
Only in release/com/digitalasset/daml-lf-value-java-proto/100.13.35: daml-lf-value-java-proto-100.13.35-sources.jar
Only in release/com/digitalasset/damlc/100.13.35: damlc-100.13.35-javadoc.jar
Only in release/com/digitalasset/damlc/100.13.35: damlc-100.13.35-sources.jar
Only in release/com/digitalasset/extractor/100.13.35: extractor-100.13.35-javadoc.jar
Only in release/com/digitalasset/extractor/100.13.35: extractor-100.13.35-sources.jar
Only in release/com/digitalasset/ledger-service/http-json-deploy/100.13.35: http-json-deploy-100.13.35-javadoc.jar
Only in release/com/digitalasset/ledger-service/http-json-deploy/100.13.35: http-json-deploy-100.13.35-sources.jar
Only in release/com/digitalasset/navigator/100.13.35: navigator-100.13.35-javadoc.jar
Only in release/com/digitalasset/navigator/100.13.35: navigator-100.13.35-sources.jar
* Check internal dependencies of maven artifacts
All artifacts that are uploaded to maven need to also have
their dependencies uploaded to maven.
* inline tuple matching
* remove unnecessary parenthesis
* mark missing artifacts for maven upload
* new library ledger-service/db-backend
* borrow contracts table schema from extractor
* borrow contract insertion, removing some data to be unused
* match contract schema with insert function
* factor insertContract arguments
* offset table declarations
* CLI argument for query store
* surrogate template IDs
* compute surrogate template IDs on-the-fly
* database init action
* incoherent typeclasses, eh
* newtype SurrogateTpId
* offset fetch/update functions
* bad sql
* bulk insert contracts, function for selecting contracts
* expose contract column name for query's usage
* Initializing DB on startup if configured
* dropping existing tables as part of initialization
* fix some query syntax errors
* createSchema flag
* function for streaming transactions with jwt party selected
* formatting
* usage
* collect acs contracts and the ledger offset at the end
* lastOffset
* fixing merge conflicts, updating the way 3rd party deps are specified
* Moving ContractDao into http-json module
so it can take domain AST as an input
* cleanup
* injecting new dependencies
* split transaction batches into inserts and deletes
* generate sql for deleting contracts
* `fetch_sources = True` for java_deps
* make the delete-constructed fragment more efficient; handle empty list here
* pass logHandler for insertContracts
* ContractDao returns ConnectionIO, it's up to the caller to wrap query into a transaction
* fixing typo
* minor cleanup, moving fromLedgerApi factory function into corresponding companion objects
* don't need it any more
* GetActiveContractsResponse => domain.Contract factory
* make concatFragment private
* add partition graph; move other contract-fetching experiments to ContractsFetch
* experimenting with akka sources
* introducing domain.Offset to work around API's empty/null offset cases
* minor cleanup
* decompose fetchActiveContractsFromOffset
* missed via
* ACS splitting graph
* finish doc for ACS splitting graph
* remove unneeded stages
* WIP
* lazily read a stream of ConnectionIO into a single ConnectionIO
* cancel on IO error
* figuring out how to put all the pieces together
* graph WIP
* Removing workflowId from the JSON API
* simplify acsAndBoundary; describe other flow pieces
* WIP
* use Vector in InsertDeleteStep; add variant for ACS (no deletes)
* `org.wartremover.warts.NonUnitStatements` enforced in `http-json` module
* evaluate InsertDeleteStep to a ConnectionIO
* database variant of LfValueCodec, using numbers for numbers
* convert input to JSON, combine insert plans, connect rest of contractsToOffsetIo
* remove strict contractsToOffset sink
* moving dao methods into an object
* putting pieces together
* contractsFromOffset WIP
* should be it
* cleanup
* cleanup
* contractsIo that takes List[domain.TemplateId.RequiredPkg])
* contractsIo that takes List[domain.TemplateId.RequiredPkg])
* cleanup
* put all pieces together, testing
something does not work yet
* diff is not required to return anything
that is why Sink.lastOption that gives Option[domain.Offset]
* factor out tuple split
* use traverse syntax in contractsIo2
* factor explicit flow steps out of graph DSL; remove aggregate
* locally model the Absolute/Begin distinction for offset bookmarking in DB
* Adding test cases to run HTTP Service with Postgres backend
the same set of test cases, run with and without DB backend
* make better use of domain.Offset in OffsetBoundary
* monomorphize InsertDeleteStep#append
* Disabling a test that fails with DB backend
* add release note
* add release note about workflowId
* a test case that checks the number or stored contracts
* trying to figure out why Postgres test fails on Windows with NPE
* Fix the import paths of {value,transaction,blindinginfo}.proto
This cleans the imports in daml_kvutils.proto:
-import "daml-lf/transaction/src/main/protobuf/transaction.proto";
+import "com/digitalasset/daml/lf/transaction.proto";
This shouldn't be a breaking change to the outside as the proto files
themselves are not published, only the java libraries generated from them.
* Update missed deps
* new data-scalacheck library for ImmArray, FrontStack, et al
* add checkLaws functions as library to data-scalacheck
* make checkLaws more configurable at use points
* Update rules_haskell and static GHC
Remove patches that have been upstreamed or are no longer required.
Update still required patches to match the new rules_haskell version.
Previously we patched rules_haskell to coerce GHC into using static
Haskell libraries in most places. In particular we moved hs-libraries
entries into extra-libraries entries in the package configuration files.
A much cleaner approach is to compile GHC with a static RTS, then GHC
will by itself choose to load static Haskell libraries.
* Remove haskell_cc_import
* da-hs-daml-cli -> daml-cli
* da-hs-damlc-app -> damlc-app
We add a dependency in bindings-java to grpc-netty, so it is
automatically available in the classpath for users. Otherwise they get
an error like [0] if they don't add it explicitly.
We also publish daml-lf-archive-java to maven central.
[0] io.grpc.ManagedChannelProvider$ProviderNotFoundException: No
functional channel service provider found. Try adding a dependency
on the grpc-okhttp, grpc-netty, or grpc-netty-shaded artifact
* Fixes#1204: Release bindings and codegens to Maven Central.
Upload the Java and Scala Bindings with the respective code
generator binaries to Sonatype Open Source Repository
Host for synchronization with Maven Central.
