* Limit SDK/platform tests to the SDK HEAD
```
# Instead of testing the full cartesian product of all SDK versions with
# all platform (~= Sandbox/JSON API) versions, we test the latest version of
# each with all versions of the other. This gives us a reasonable feedback
# with regards to maintaining backwards-compatibility without causing the
# test runs to grow quadratically.
```
changelog_begin
changelog_end
* Address https://github.com/digital-asset/daml/pull/8893#discussion_r578310492
* daml-lf/interpreter: Avoid `.iterator.map(...).toArray`.
It ends up being pretty heavy on calls, making the stack really big when
we recurse. This doesn't solve the recursion problem, but it does make
it a little lighter.
* daml-lf/archive: Unroll a foldLeft to reduce the stack size.
CHANGELOG_BEGIN
CHANGELOG_END
Some of our targets, e.g., //ledger/ledger-api-health have no
dependencies. On Scala 2.13, an empty classpath option produces a
bunch of confusing but afaict harmless warnings, on 2.12 it doesn’t
but there is still no point in passing an empty classpath so we can
omit it on both.
changelog_begin
changelog_end
* Stop pretending strings are booleans
Sorry for all the mess here. I’m not capable of programming in yaml.
It turns out is_release is a string not a boolean and
```
and('false', eq('true', 'true'))
```
is true.
I hate everything about this.
changelog_begin
changelog_end
* Name bash step to reduce confusion
changelog_begin
changelog_end
* fix version in test
changelog_begin
changelog_end
* names can’t have spaces apparently
changelog_begin
changelog_end
* Fixup condition for running publish_mvn_npm
This needs to run for both linux and linux-scala-2.13
changelog_begin
changelog_end
* Update ci/build-unix.yml
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Fixup scala 2.13 check
Somehow I managed to misread the helpcheck and get confused by my
experiments and thought semver produces an exit code of 1,0,-1 but
actually it writes that to stdout.
changelog_begin
changelog_end
* Update ci/build.yml
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Update ci/build.yml
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Port Ledger API Test Tool to Scala 2.13
And with that we’re finally at //... building on Scala 2.13.
changelog_begin
changelog_end
* Fix build on 2.12
changelog_begin
changelog_end
* Fix kvutils export on 2.13
changelog_begin
changelog_end
As evidenced by #8856, daml script dumps currently fail to compile if
we generate a dump including templates which are newer than the
default LF version in the compiler. This PR addresses this by
including a --target flag in the generated daml.yaml.
changelog_begin
changelog_end
* Replaced diff with diffx
* Explicitly added magnolia and mercator dependencies to fix automatical type class derivation
* CHANGELOG_BEGIN
CHANGELOG_END
* Removed unnecessary Diff type class instance for Seq[T]
* Removed ai.x.diff leftovers
* Added an explanatory comment for magnolia and mercator dependencies
* Formatted changes
* Added optional scaladoc parameter to Bazel's da_scala_library_suite()
* Formatted changes
Spun out of #8863. This seems cleaner, more consistent with how we
handle the conversion in submitTree and it has the nice side effect
that it means that conversion now only happens after comitting which
is crucial for #8863.
changelog_begin
changelog_end
* separate OracleQueries from PostgresQueries
- with some changes from 8161e63189 courtesy @cocreature
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* abstract BIGINT
* json, signatories, observers columns
* compatible lastOffset
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* oracle functions for select (single template ID), insert
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* add oracle branch to integration tests
* oracle CLI configuration for json-api
* run integration tests with ojdbc in classpath
* update maven_install for ojdbc
* drop table if exists for Oracle
* make create DDLs and drops more planned out; drop in reverse order for Oracle integrity
* repin maven
* port agreement_text
* port (by removal) array part of ledger offset update
* use CASE instead of JSON map lookup for multiparty offset update
* simplify self types
* fix contract archival
* repin
* remove selectContracts in favor of selectContractsMultiTemplate
* move Oracle test execution to separate build target
* move websocket test to itlib
* make a bad array instance for Oracle
* report actually-available JDBC drivers only
* configure Oracle test from CI
* attempt with platforms and constraints
* a mismash of bazel to get it to conditionally enable oracle testing
* fix dep resolution in Scala 2.13
* make the Oracle test a stub (inits and does empty DB query)
* remove commented unused deps
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* repin
* we never supply a value for the surrogate ID columns
- suggested by @cocreature; thanks
* add not null to json in DB-specific place
- suggested by @cocreature; thanks
* why DBContractKey
- suggested by @cocreature; thanks
* textType isn't finalized
- suggested by @cocreature; thanks
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* kvutils: Split `Raw.Key` into `Raw.LogEntryId` and `Raw.StateKey`.
We almost always know which one we're dealing with, so let's be more
specific.
* kvutils: Rename `Raw.Value` to `Raw.Envelope`.
That's what it is.
* kvutils: Deprecate conversions between `Raw` values.
And conversions to and from `ByteString`.
It's time to migrate everything.
CHANGELOG_BEGIN
- [Integration Kit] The various uses of `ByteString` in kvutils are now
strongly typed. Users of kvutils will need to use the `Raw.Bytes`
subtypes to represent the various keys and values that flow through a
kvutils-based driver. ByteStrings representing keys need to be wrapped
in either `Raw.LogEntryId` or `Raw.StateKey`, and ByteStrings
representing value envelopes need to be wrapped in `Raw.Envelope`.
This prevents accidentally confusing one for the other.
Implicit conversions have been provided for Scala users, which should
ease the transition. There are marked `@deprecated` and will be
removed before the release of DAML SDK v1.12.
CHANGELOG_END
This fixes Scaladoc and our pom file generation.
It also clears up the confusing error around gatling and removes a
redundant dependency on sbt (no idea why we had that in the first
place) both of which resulted in Scala 2.12 dependencies in our 2.13
lockfile which is obviously bad.
With this, we should now be ready to publish Scala 2.13 artifacts once
the ledger API test tool PR lands.
changelog_begin
changelog_end
Tests are still missing and blocked on #8821.
The main change here is the switch from `ArraySeq[Byte]` to
`ArraySeq.ofByte`. `ArraySeq` allows for boxed and unboxed
representaitons. That means that `ArraySeq[Byte]unsafeArray` does not always return
an Array[Byte] (boxed version would be Array[AnyRef]).
Apparently collection-compat has taken the yolo approach and pretends
it can give you an Array[Byte] anyway 🤷 Scala 2.13 on the other
hand, does things properly in this regard which means the code relying
on `unsafeArray` fails to compile.
`ArraySeq.ofByte` is the specialized unboxed version where none of
this is an issue on both 2.13 and 2.12.
changelog_begin
changelog_end
* Draw the rest of the Scala 2.13 owl
Not quite but pretty close and this switches us over from inclusions
to exclusions which makes it much easier to track.
Ledger API test tool should be fixed by #8821. Non-repudiation needs a
tiny bit of work since unwrapArray doesn’t work the same on 2.13 but
shouldn’t be hard to fix.
changelog_begin
changelog_end
* Fix ScriptService tests
Those tests were all dumb. They asserted on a fixed order while the
function to sort the things was broken so we ended up with the random
Map order which is unsurprisingly not the same.
This is easily fixed by fixing the sort function.
There is also a second issue with query not sorting.
changelog_begin
changelog_end
* Turns out if you fix one test the next one breaks
And clearly nobody ever tested this or give this a second thought.
changelog_begin
changelog_end
* damlc: fix: catch UserError instead IOException
When the package-db metadata can not be parsed, an exception is thrown
and needs to be catched. Previously we were catching IOException, but
in this case it's actually a UserError that is thrown.
CHANGELOG_BEGIN
CHANGELOG_END
* damlc test: show full coverage report
We add a flag `show-coverage` to `damlc-test` to show templates that
are never created and choices that are never executed during the tests.
CHANGELOG_BEGIN
[damlc test] A new flag `show-coverage` shows full test coverage
reports.
CHANGELOG_END
* Add first metrics to non-repudiation proxy
changelog_begin
changelog_end
Contributes to https://github.com/digital-asset/daml/issues/8635
Add a few key metrics for the non-repudiation proxy, with more to follow,
in particular keeping track of the performance overhead associated with
accessing the underlying database.
All metrics can be seen in com.daml.nonrepudiation.Metrics
Running the conformance tests successfully shows a summary of those
metrics with the expected period (five seconds).
* Address https://github.com/digital-asset/daml/pull/8766#discussion_r575044128
* Added logging incoming requests in API services: Submission, ConfigManagement, PackageManagement, ParticipantPruning and PartyManagement
CHANGELOG_BEGIN
- Logging incoming requests in API services
CHANGELOG_END
* Logging transactions and transaction trees streamed by the ApiTransactionService
CHANGELOG_BEGIN
- Logging transactions and transaction trees returned by the ApiTransactionService
CHANGELOG_END
* Logging storing the db in JdbcLedgerDao
* Changed log severity
* Factored out logging util to ContextualizedLogger
* Review improvements
* Removed unused import
* Formatted changes
* Logging completions stream items
* Fixed log message
* Logging complete transactions and transaction trees
* Removed duplicated keys from logging context
* Formatted changes
* Reduced logging for completions and transactions
* Removed redundant log
* Removed update* prefix in Indexer's logging context
* Minor improvement
* Minor improvement
fixes#8498
This fixes the error in 2.13 wtr to the location change of Predef. It
doesn’t yet address the warning wtr to the import of higherKinds. For
now, our build ignores that warning. Trying to figure out if we can
get away with a breaking change here or if we need to hide that change
behind a flag but either way, no need to block fixing the actual error
on that.
changelog_begin
changelog_end
It does not seem like CI machines recover from a failed clean-up. This
is not the most elegant solution possible, but it's a cheap one that
should work.
Not: shutting down the machine in the middle of the build will not
provide an error message to Slack for main branch builds (because the
`tell_slack_failed` step would need to run on the same machine) but will
correctly report failure for PRs (that was the original purpose of the
`collect_build_data` step).
An alternative here would be to give a delay to the shutdown command,
and try to calibrate it so that it's long enough for this job to
correctly report its failure to both Azure and Slack, while making it
short enough that no other job gets assigned to the machine. I'm not
clear enough on how often Azure assigns jobs to try and bet on that.
CHANGELOG_BEGIN
CHANGELOG_END
This PR fix a regression performence intoriduce in #8804, were we
calculate exercise node seed twice.
Here is the result of a benchmark:
before:
CollectAuthority.bench //daml-lf/scenario-interpreter/CollectAuthority.dar CollectAuthority:test avgt 20 48.968 ± 0.667 ms/op
after:
CollectAuthority.bench //daml-lf/scenario-interpreter/CollectAuthority.dar CollectAuthority:test avgt 20 41.772 ± 0.374 ms/op
CHANGELOG_BEGIN
CHANGELOG_END
* treeCids --> treeCreatedCids
* Only bind referenced contract ids
Adds an additional traversal of the tree to determine all referenced
contract ids.
Filters out contract ids that are never referenced before creating
bindings for them.
changelog_begin
changelog_end
* Add treeReferencedCids unit tests
* Add encodeTree unit tests for contract id bindings
* Collect cids in created arguments as well
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This PR adds a generic function to compare to transaction trees (or
rather sequences thereof) modulo alpha equivalence wtr to contract ids
and party ids.
This allows us to abstract away the concrete integration test for Daml
Script dump we had before and turn it into a generic process:
1. Run some setup code to generate transactions.
2. Query and store transaction stream.
3. Create the dump, build it and run it for a different set of
parties.
4. Query and store transaction stream for new parties.
5. Diff transaction streams.
This PR does not change tests or add new ones. I think it might be
nice to use Daml Scripts for 1 as well but I’ll leave that for a
separate PR. The infrastructure here already supports this.
fixes#8772
changelog_begin
changelog_end
[ledger-api-server] Make database connection pool size configurable
CHANGELOG_BEGIN
[Daml Driver for PostgreSQL]: Added CLI option --database-connection-pool-size
to configure the size of the database connection pool
[Integration Kit]: Added the CLI options api-server-connection-pool-size and
indexer-server-connection-pool-size to configure the database connection pool size
for the Ledger API Server and the indexer respectively.
CHANGELOG_END
Also adds a comment to make sure we don’t forget it in the
future. Note for reviewers: Our code already only looks at the first
line, so the comment in the second line does not break anything.
changelog_begin
changelog_end
