Commit Graph

17 Commits

Author SHA1 Message Date
Moritz Kiefer
1b32044a74
Build //libs-scala/... on 2.13 (#8282)
* Build //libs-scala/... on 2.13

One test is unfortunately disabled at the moment since I utterly
failed to figure out why I get a ClassNotFoundException on 2.13.

changelog_begin
changelog_end

* Copyright headers

changelog_begin
changelog_end

* I can’t bazel today

changelog_begin
changelog_end

* Apply suggestions from code review

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

* Update libs-scala/resources/src/main/2.13/com/daml/resources/UnitCanBuildFrom.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* No split on view

changelog_begin
changelog_end

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-12-14 17:36:53 +00:00
Samir Talwar
7f679b9896
resources: Customizable contexts. (#7678)
* resources: Move builders into //ledger/ledger-resources.

Keep the actual constructors in a trait, but instantiate it when working
with ledger code.

This allows us to later introduce an extra "context" type parameter to
ResourceOwner.

* resources-akka: Move the builders in to //ledger/ledger-resources.

* resources: Introduce an abstract `Context` parameter for owners.

This replaces the concrete `ExecutionContext`. While it _can_ be an
execution context, it really doesn't matter as long as we can get at one
somehow.

This is being introduced so we can wrap the context in a container,
either for type tagging or to include extra information.

Because our current context _is_ `ExecutionContext`, and an implicit is
provided to extract it, we can end up with two ways to get the same
value. We use shadowing to prevent this. This problem should go away in
the near future when a new context type is added.

CHANGELOG_BEGIN
- [Integration Kit] The `ResourceOwner` type is now parameterized by a
  `Context`, which is filled in by the corresponding `Context` class in
  the _ledger-resources_ dependency. This allows us to pass extra
  information through resource acquisition.
CHANGELOG_END

* ledger-resources: Move `ResourceOwner` here from `resources`.

* ledger-resources: Remove dependencies from outside //ledger.

* ledger-resource: Wrap the acquisition execution context in `Context`.

So we can add a logging context to it.

* resources: Pass the Context, not the ExecutionContext, to Resource.

* Avoid importing `HasExecutionContext`.

* ledger-resources: Publish to Maven Central.

* resources: Make the small changes suggested by @stefanobaghino-da.

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* ledger-resources: Pull out a trait for test resource contexts.

Saves a few lines of code.

* Restore some imports that were accidentally wildcarded.

* resources: Replace an `implicit def` with a couple of imports.

* participant-integration-api: Simplify the JdbcLedgerDaoBackend tests.

Try and use the right execution context where possible.

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-10-20 09:26:28 +00:00
Samir Talwar
99463577af
triggers + ports: Lock free ports before binding. (#6911)
* triggers: Use `FreePort.find()`.

* ports: Move `LockedFreePort` from postgresql-testing for reuse.

* triggers: Use `LockedFreePort` to avoid race conditions.

* ports + triggers: Move common port testing into the ports library.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-29 14:29:55 +00:00
Samir Talwar
d6fc2bbb58
ledger-api-client + participant-integration-api: Increase the default maximum inbound error size, and truncate errors well before that. (#6807)
* participant-integration-api: `GrpcServerOwner` -> `GrpcServer.Owner`.

Mostly so I can create a test class named `GrpcServerSpec`.

* ports: Move the free port search from postgresql-testing.

* participant-integration-api: Test the basics of GrpcServer.

This uses the HelloService to make sure the server behaves normally.

* ledger-api-client: Extract out channel configuration from LedgerClient.

So we can test it independently of the LedgerClient itself.

* ledger-api-client: Increase the default maximum inbound header size.

Increased from 8 KB to 1 MB.

* participant-integration-api: Reduce the maximum error message size.

Truncate GRPC error descriptions to 256 KB.

* participant-integration-api: Use `Port.Dynamic` instead of `FreePort`.

In tests.

* participant-integration-api: Explicit null checks when they're shorter.

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* ledger-api-client: Reduce the max inbound message size back to 8 KB.

And reduce the maximum size of an error description pushed out by the
server accordingly.

CHANGELOG_BEGIN
- [Integration Kit] Truncate GPRC error messages at 4 KB. This ensures
  that we won't trigger a protocol error when sending errors to the
  client.
CHANGELOG_END

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-07-21 15:50:33 +00:00
Samir Talwar
57a8d0b37e
CI: Run PostgreSQL once for all Scala tests. (#5919) 2020-05-14 09:06:34 +02:00
Samir Talwar
208d4a50da
postgresql-testing: Use a new, random database name every time. (#5911)
* postgresql-testing: Store the JDBC URL separately.

* postgresql-testing: Expose the username and password.

* postgresql-testing: Get the caller to create the database.

And make sure it's a random one, not "test".

CHANGELOG_BEGIN
CHANGELOG_END

* postgresql-testing: Only store the JDBC URL for tests.

Less mutable state, innit.

* postgresql-testing: Capture the individual JDBC URL parameters.

* Bazel: Fix PostgreSQL binary paths.

* postgresql-testing: Just recreate the database in PostgresAroundEach.

There's no need to restart the process with a different data directory.
2020-05-08 12:36:42 +00:00
Moritz Kiefer
0e5aa4bebf
Remove -o config from PostgresAround (#5707)
We already specify those values in the config file.

changelog_begin
changelog_end
2020-04-23 16:08:37 +00:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Samir Talwar
52dbcf5d95
postgresql-testing: "Lock" ports when starting PostgreSQL. (#5310)
* http-json: Ask for a free port by specifying port 0.

This will avoid race conditions.

* bindings-akka-testing: Delete RandomPorts; it's unused.

* ports: Fix the Bazel test glob.

* ports: Move FreePort to postgresql-testing and add a test case.

* postgresql-testing: Make `FreePort.find()` return a `Port`.

* postgresql-testing: Lock free ports until the server starts.

This uses a `FileLock`, which should work well on all our
supported operating systems as long as everyone agrees to use it.

CHANGELOG_BEGIN
CHANGELOG_END

* postgresql-testing: Try to find a free port 10 times, then give up.

* postgresql-testing: Use a shared directory for the port lock.

* postgresql-testing: Try an alternative way of getting `%LOCALAPPDATA%`.
2020-03-31 17:13:02 +00:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
Samir Talwar
1e13b8679e
postgresql-testing: On failure, logs are no longer an Option. (#5190)
`logs` used to be an `Option[Seq[String]]`. They were changed to
`Seq[String]` with a default of `Seq.empty`, but the code to print them
still assumed an `Option`, which means they were printing as
"ArrayBuffer(line 1, line 2, line 3)" instead of, well, nicely.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-26 08:09:24 +00:00
Stefano Baghino
181df8f3ab
Act on Option2Iterable wart (#5154)
Some Option2Iterable ignore annotations are not needed, others were needed for unused methods.

In a few occasions we were ignoring the warning for the very purpose for which is was there,
i.e. avoiding an implicit conversion. I'm all for not verifying this rule if we agree we
don't need it.

For ProcessFailedException it was a bit gratuitous, I changed the way in which the exception
message is built.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 11:25:05 +00:00
Samir Talwar
9bdfbb0534
Avoid opening a server to the world when finding a free port. (#4683)
* Avoid opening a server to the world when finding a free port.

This is very annoying on macOS because we get a focus-stealing popup for
a split second, asking for permission to allow the server through the
firewall. The popup pretty much always disappears before it can even be
read, when the server is closed.

This is almost certainly not an attack vector, because:
  - we only do this in tests,
  - the server is open for only a few milliseconds,
  - nothing is served,
  - and finding the port is tricky, because it's effectively random.

Nevertheless, it's very annoying.

CHANGELOG_BEGIN
CHANGELOG_END

* Extract a Bazel package for finding free ports.

We seem to do it in 4 different places, which I think is enough to
remove the duplication.
2020-02-25 13:09:28 +00:00
Samir Talwar
a589f4af0b
sandbox: Move more resource acquisition into the owner. (#4501)
* sandbox: Move more resource acquisition into the `owner`.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Reimplement SandboxClientResource as a resources.Resource.

* codegen: Use resources in TestUtil.

* sandbox: Manage PostgreSQL in tests with ResourceOwners.
2020-02-14 13:52:45 +00:00
Samir Talwar
791fad5309
kvutils: Test that the ledger can be restarted and will resume where it left off. (#4321)
* kvutils: Use `ResourceOwner#use` to simplify tests.

* kvutils: Merge in `RestartSpec` from ledger-on-sql.

CHANGELOG_BEGIN
- [KVUtils] Added a test case to the participant state tests to ensure
  your ledger state is resumable upon restart.
CHANGELOG_END

* ledger-on-posix-filesystem: Extract paths into their own class.

* ledger-on-posix-filesystem: Seed the dispatcher with the correct head.

* kvutils: In participant state tests, be clear about different strings.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-31 16:26:36 +00:00
Samir Talwar
c81cc2ab2a ledger-on-sql: Migrate tables using Flyway. (#4232)
* ledger-on-sql: Pull out a superclass for the various integration tests.

* ledger-on-sql: Use Flyway to migrate tables in an idempotent manner.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-sql: Make it easy to run it with Bazel for experimentation.

* ledger-on-sql: Test that migrations will never change in the future.

* ledger-on-sql: Add a prefix of "ledger_" to the tables.

This is so we don't accidentally conflict with the index when the
schemas are shared.

I am letting myself modify the migrations because the existing
migrations haven't been merged into `master` yet.

* ledger-on-sql: Explain why we change the pool size after migration.

* ledger-api-test-tool: Tests now specify a timeout scale, not a timeout.

Makes it easier to change the default timeout in one place.

* ledger-api-test-tool: Increase timeouts on slow tests.

These tests produce a lot of volume and can make CI flaky.

* ledger-on-sql: Only tear down PostgreSQL in tests.

For other databases, we just create a new file for each test case.

* ledger-on-sql: Reduce the log output in tests.

* ledger-on-sql: Use a separate connection pool for Flyway when possible.

Apparently `setMaximumPoolSize` doesn't really have the desired effect
after the connection pool has already been used. The new test case will
be flaky if we process more than one commit in parallel.

For SQLite, it seems to be OK.

* ledger-on-sql: Use a separate connection pool for Flyway with SQLite.

Except in memory.

* ledger-on-sql: Use a separate PostgreSQL database for each test.

Because performance, innit. Don't have to tear them down.

* ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects.

* ledger-on-sql: Go into even more detail about pool size hijinks.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 16:16:23 +00:00
Samir Talwar
f3f7def5d2
postgresql-testing: Extract PostgresAround from the Sandbox t… (#4197)
* postgresql-testing: Extract `PostgresAround` from the Sandbox tests.

CHANGELOG_BEGIN
CHANGELOG_END

* postgresql-testing: Split into several files.

* postgresql-testing: Pull out the host and database names into variables.

* postgresql-testing: Don't publish to Maven.

I have no idea why the build wasn't failing before.

* postgresql-testing: Assimilate the `@postgresql_dev_env` dependency.
2020-01-24 12:48:36 +01:00