* Experiment: replacing row OFFSET with ledger offset in the flat transactions query for one party.
if this improves the perf numbers, the rest of the queries can be updated.
* Flat transaction query optimization:
replacing row offset with ledger offset
* Flat transaction query optimization:
replacing row offset with ledger offset
* transaction tree query optimization
changelog_begin
[Sandbox-next/Postgres]
Flat Transaction Stream, Transaction Tree Stream SQL query optimizations.
Pagination based on Ledger Offset instead of SQL Row Offset.
changelog_end
* Addressing code review comments
* Expose the DAML profiler in Sandbox Classic
This PR adds an option `--profile-dir` to Sandbox Classic which takes
a directory as its argument. When this option is used, all command
submissions to the Sandbox are run in profiling more and the resulting
profiling information is written to the directory specified via
`--profile-dir`. See `/ledger/sandbox/README.md#Profiling` for further
details on this.
CHANGELOG_BEGIN
CHANGELOG_END
* Adapt DAML-LF REPL
CHANGELOG_BEGIN
CHANGELOG_END
CHANGELOG_BEGIN
[DAML Ledger Integration Kit]: Added 4 new metrics for more detailed execution time statistics:
- Timer ``daml.execution.lookup_active_contract_per_execution``: measures the accumulated time spent for looking up active contracts per execution
- Histogram ``daml.execution.lookup_active_contract_count_per_execution``: measures the number of active contract lookups per execution
- Timer ``daml.execution.lookup_contract_key_per_execution``: measures the accumulated time spent for looking up contract keys per execution
- Histogram ``daml.execution.lookup_contract_key_count_per_execution``: measures the number of contract key lookups per execution
CHANGELOG_END
CHANGELOG_BEGIN
- [JSON API] The WebSocket endpoints ``/v1/stream/query`` and ``/v1/stream/fetch`` are now
stable; these endpoints in SDK v1.0.0 are fully compatible with this first officially
stable version.
See `issue #6152 <https://github.com/digital-asset/daml/pull/6152>`_.
CHANGELOG_END
This PR extends the existing sandbox migration tests with a query to
the transaction service before and after running each step and
corresponding validations for that in the migration runner.
changelog_begin
changelog_end
* remove boilerplate from GenNode.map3
- There is no need to enumerate the unchanged fields, because if you
miss one, you get a type error. Unfortunately foreach3 cannot benefit
from the same typechecking.
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* Upgrade puppeteer
We’ve seen a couple of issues in the compatibility tests of the form
```
Error: Protocol error (Runtime.callFunctionOn): Target closed.
```
Looking at the issue tracker in puppeteer this might be fixed in newer
versions and I don’t see why we should stick to a fairly old version
anyway.
changelog_begin
changelog_end
* Upgrade nodejs
changelog_begin
changelog_end
* temporary add a step to kill node_modules
changelog_begin
changelog_end
* Kill live server and try to fix Windows
changelog_begin
changelog_end
* Undo rm
changelog_begin
changelog_end
* Pass JdbcConfig object to TriggerDao apply
* No need to return TriggerDao from init db
* Refactor introducing RunningTrigger type
* Rename triggerId -> triggerInstance and triggerOrigId -> triggerName
Note this also changes the start request parameter name to triggerName.
However I have not yet renamed triggerId in the response messages. We
should probably make it triggerInstance there too but in a later PR.
changelog_begin
changelog_end
* disable Any wart
* first pass removal of Any suppressions for false positives
* second pass removal of Any suppressions for false positives
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* third pass removal of Any suppressions for false positives
* fourth pass removal of Any suppressions for false positives
* reformat newly single-suppressions into single lines
- suggested by @SamirTalwar-DA; thanks
* move integration-type tests in http-json to separate integration-tests target
* remove unused library deps from split tests
* remove resources from unit tests, set to medium size
* move resources to it subtree
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* Don't store archived divulged contracts
Specifically, don't keep contracts that
have been divulged AND archived
in the same transaction.
Even if the archival of a divulged contract
might not be visible to all locally hosted
parties, the ledger would reject all
commands that try to use archived
contracts anyway.
CHANGELOG_BEGIN
CHANGELOG_END
* Add test
* Fix compile error
This is preparatory work for hitting the transaction service (and
others but that one is the most important) which isn’t possible via
DAML Script. To ease review, this PR only switches from a DAML Script
to a custom Scala binary. It does not yet change what is tested.
changelog_begin
changelog_end
* Add a module-prefixes field to rename all modules in a pkg
This PR adds a `module-prefixes` field to `daml.yaml` a a shorthand
for specifying a `--package` flag that renames all modules in a
package to have the same prefix.
The docs are updated to describe how you can use this field and there
is a test case that makes sure it works.
fixes#4948
changelog_begin
- [DAML Compiler] You can now use the new ``module-prefixes`` field in
``daml.yaml`` to add a prefix to all modules from a dependency. This
is particularly useful for handling colliding module names during
upgrades. See
https://docs.daml.com/daml/reference/packages.html#handling-module-name-collisions
for more information.
changelog_end
* Update compiler/damlc/daml-opts/daml-opts-types/DA/Daml/Options/Packaging/Metadata.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Improve docs and fix broken suggestion
changelog_begin
changelog_end
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* add law checking for SValue Ordering
CHANGELOG_BEGIN
CHANGELOG_END
* nested example for randomComparableValues
* match comparableAbsCoidsGen -> comparableCoidsGen renaming
* register scala-collection-compat with java deps list
* add scala-collection-compat to http-json deps
* remove breakOut throughout http-json, replaced with view/to or iterator/to
* use scala 2.13-style `to` calls in http-json
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* use 2.13-style to in lf-value-json
* some fused size comparisons
* remove low-hanging breakOuts in daml-lf
* regenerate maven_install.json for scala-collection-compat
* regenerate maven_install.json for scala-collection-compat
* regenerate maven_install.json for scala-collection-compat
The `gawk`-based method currently referenced does not work in a
multi-line `LATEST` world. I don't want to scare people with the command
that would work here.
CHANGELOG_BEGIN
CHANGELOG_END
In light of #6127, I kept wondering why rebuilding 1.1.1 would fail. The
problem addressed by #6127 is that we tried to rebuild it, which we
shouldn't, but the reason I noticed it is because the build failed, and
there is no good reason for the 1.1.1 docs to not build anymore. Looking
at the logs confused me even more as it failed with (elided):
```
docs/source/support/new-assistant.rst:
WARNING: document isn't included in any toctree
```
and that change happened _after_ 1.1.1. So I went back to the code, and
discovered I somehow had gotten confused as I changed the approach
mid-way through editing the file. If we're overwriting the
`release-notes.html` file post-build, which we are now doing (and is the
reason for ignoring it when checking checksums), then we should not be
touching the `release-notes.rst` file pre-build.
CHANGELOG_BEGIN
CHANGELOG_END
* equalz Scalatest matcher in new daml-lf/scalatest-tools library
* equalz typing tests
* a 'should' replacing design
* a 'MatcherFactory1' design
- this fails because the TC parameter should be a type member to avoid
scala/bug#5075 but it is not
* MatcherFactory1 with chained Lub+Equal typeclass
- requires partial-unification at point of use, which is not great
* LubEqual's extra tparam is probably unneeded
* better LtEqual
* demonstrate that HK LubEqual's resolve with DMT should + MatcherFactory
* remove unneeded 3rd param from LubEqual, again
* update dependency specs and license headers
* allow use with should, shouldNot in some cases, preserving the shouldx/shouldNotx alternatives
* move Equalz to libs-scala/scalatest-utils
* rename bzl targets and place in com.daml.scalatest package
* add scalatest-utils to release
* move *SpecCheckLaws, Unnatural to scalatest-utils
* missed scalacheck dep in scalatest-utils
* downstreams of *SpecCheckLaws now get them from scalatest-utils
* test equal-types case as well
* update LF documentation
CHANGELOG_BEGIN
CHANGELOG_END
* whitespace error
* Update Jackson library version to latest published to resolve several reported vulnerabilities
CHANGELOG_BEGIN
Upgrade jackson version to 2.11.0 from 2.9.9.3
CHANGELOG_END
* regenerate updated jackson dependencies
* io.grpc:grpc-xxxx to 1.29.0 (from 1.22.1)
io.netty:netty-xxxx to .1.50.Final (from 4.1.37.Final)
io.nett.netty-tcp-native-boringssl-static to 2.0.30.Final (from 2.0.25.Final)
To resolve open vulnerabilities with these versions
netty-4.1.37.Final vulnerabilities
BDSA-2018-4022 (Medium)
BDSA-2019-2610 (Medium)
BDSA-2019-3119 (CVE-2019-16869) (Medium)
BDSA-2020-0130 (Medium)
BDSA BDSA-2019-4230 (CVE-2019-20445) (Low)
BDSA BDSA-2019-4231 (CVE-2019-20444) (Low)
BDSA BDSA-2020-0666 (CVE-2020-11612) (Low)
BDSA BDSA-2019-2642 (Low)
BDSA BDSA-2019-2649 (Low)
BDSA BDSA-2019-2643 (Low)
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
–
* Update spray versions to address vulnerabilities
CVE-2018-18853 and CVE-2018-18854
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
* do not change io.grpc version since reflection seems to be misbehaving
* Clarify how to bump grpc/netty/protobuf versions
Also "downgrade" netty to 4.1.48, according to
https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty
CHANGELOG_BEGIN
CHANGELOG_END
* Load protobuf deps after haskell deps to avoid loading an older version of rules_cc
* Upgrade protoc and protobuf-java to 3.11.0
* buildifier reformat
* regen unique int after rebase
* remove commented patch
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
The docs cron is supposed to ignore the release-notes.html page when
checking whether a docs folder is corrupted, because we manually
override it. However, that currently doesn't work, either because the
`sed` version we are using does not support changing the delimiters, or
because no version of `sed` does and I just imagined it.
CHANGELOG_BEGIN
CHANGELOG_END
Changed by #6123, relevant part of the diff is:
```
ledger.lookupGlobalContract(ParticipantView(committers.head),
effectiveAt, acoid) match {
- case LookupOk(_, result) =>
+ case LookupOk(_, result, _) =>
cachedContract = cachedContract + (step -> result)
```
which seems benign enough.
CHANGELOG_BEGIN
CHANGELOG_END
The comment on it says it should be replaced by `local_jdk//:javadoc`
once [a patch](https://github.com/bazelbuild/bazel/pull/7898) is merged
into Bazel. Not only has that patch been merged, but I could not find
any reference to `javadoc_dev_env` to replace.
CHANGELOG_BEGIN
CHANGELOG_END
* Build CopyTrigger example in compatibility
* Test DAML trigger with test DAML script
Defines a custom client script that runs the trigger runner in the
background and uses a DAML script in the foreground to test that the
trigger runs and its effects are observable.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Use bazel_tools:versions.bzl%versions.is_at_least
Also align versions.is_at_least|most with skylib
The arguments to the Skylib function are in the opposite order. I've
included a docstring to explain
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Disentangle the execution of closures and builtin functions.
Both closures and builtins take arguments, but only closures have free variables.
Consequently, we split the machine component `frame` into `frame` and `actuals`.
Allowing them to be managed independently, and simplifying lookup for `SELocF` and `SELocA`.
The `KFun` continuation is split into 3 variants: `KFun`, `KBuiltin` and `KPap`.
These handle the execution of three different cases of function application:
- execution of a user function (a closure)
- execution of a builtin
- execution of a partial application (builtin or closure, it doesn't matter)
The choice of which continuation to push is made in the `KArg` continuation when the
function-expression has been evaluated and we have discovered what it is - builtin or
closure - and we know it's arity.
The prior code made some related decisions (for example, is this an over-application) but
left other choices to `KFun` (code was: `enterFullyAppliedFunction`). This required that
the `KFun` continuation had to stash the entire `Prim` (closure/builtin) and `arity`, only
to then re-examine it when the continuation was entered.
Now, all decisions are made within `KArg`, making the structure and execution of the three
new continuations much simpler. In particular, the `machine.frame` is only set when we
enter `KFun`.
All continuations which need to preserve their environment, now save both frame & actuals,
along with the env-stack-size. And `restoreEnv` is adapted accordingly.
changelog_begin
changelog_end
* improve variable name
* with SomeArrayEquals
I have never used Scoop before and I'm not sure how to actually test
this, but I thought this might at least get a conversation going.
Plus, if my reading of the documentation of both toxiproxy and Scoop is
correct, this may even work as is.
CHANGELOG_BEGIN
CHANGELOG_END
This PR removes a few scoop-related files that seem to have been there
primarily to help us discover how Scoop worked when we first started
using it. They have not been touched in a long time and as far as I can
tell have not been run in a long time either.
I'm a firm believer in the idea that dead code belongs to the git
history, and has no place in the current worktree.
CHANGELOG_BEGIN
CHANGELOG_END
This should be merged after #6080. This PR adds a patch (and
consequently updates the `ci/cron/perf/compare.sh` script) to apply the
same logical change as #6080 on top of the baseline commit, so our
performance comparison remains "apples to apples".
I am well aware that managing patches is not going to be a great way
forward. The rate of changes on the benchmark seems to be slow enough
that this is good enough for now, but should we change the benchmark
more often and/or want to add new benchmarks, a better approach would be
to handle the changes at the Scala level. That is:
- Create a "rest of the world" (world = Speedy, its compiler, and all of
the associated types) interface that benchmarks would depend on,
rather than depend directly on the rest of the codebase.
- Create two implementations of that interface, one that compiles
against the current state of the world, and one that compiles against
the baseline.
- Change the script to load the relevant implementation, and then run
all the benchmarks as-is, with no match necessary.
CHANGELOG_BEGIN
CHANGELOG_END
The aim of the `CollectAuthority` benchmark is to track performance improvements made to the
DAML compiler & the speedy interpreter. Unfortunately the benchmark was spending at least
20% of the time being benchmarked outside of the speedy machine execution code, and
instead interacting with the ledger. We would like to minimize this as much as possible.
The solution is to cache the responses from the ledger made during the setup() run, and
replay them during the benchmark `run()`s. To perform the caching, we no longer make use
of the scenario-interpreter, but instead have our own simplified runner, specialized for
this benchmark, and managing the cache.
using the cache speeds up the benchmark by a factor of about x1.25 (Which is the origin of
the claim above about 20% outside of speedy execution).
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
95.188 ±(99.9%) 0.410 ms/op [Average]
(min, avg, max) = (94.177, 95.188, 95.890), stdev = 0.472
CI (99.9%): [94.779, 95.598] (assumes normal distribution)
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
75.881 ±(99.9%) 0.288 ms/op [Average]
(min, avg, max) = (75.077, 75.881, 76.491), stdev = 0.332
CI (99.9%): [75.593, 76.169] (assumes normal distribution)
Because of how we run perf benchmark on CI -- by running the benchmark for now vs. some
fixed time in the past -- it ought to be fine to change the benchmark like this, although
we might need some help from Gary!
As well as improving the focus of this benchmark going forwards, we should also gain a
retrospective improvement on the speedup work already committed, since they will no longer
we dragged down by time which is outside of our control.
changelog_begin
changelog_end
