* Added ledger writer that chooses between instances based on estimate interpretation cost.
CHANGELOG_BEGIN
CHANGELOG_END
* Code tidying.
* Delegate to pre-executing writer in case thershold is set to 0.
* Added ability to change metrics.
* Added metrics.
* Code tidying.
* Update ledger/participant-state/kvutils/src/main/scala/com/daml/ledger/participant/state/kvutils/api/InterpretationCostBasedLedgerWriterChooser.scala
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
I like keeping track of what happens in the areas I’m responsible
for. I’ve also removed Shayne since it doesn’t make sense to keep him.
changelog_begin
changelog_end
* add -Xlint:doc-detached
- reverts 1feae964e3 from #6798
* attach several scaladocs where they'll actually be included
* no changelog
* attach several more scaladocs where they'll actually be included
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
This fixes 3 issues:
1. Switch the order in which we assign to `$proc` and
`waitOn`. Without this the cleanup will not do anthing if `beforeAll`
fails since the variable has not been assigned. This results in jest
hanging forever until we hit the Bazel timeout.
2. Increase the timeout on `beforeAll` since we occasionally hit this
on CI.
3. Allocate 2 CPUs to reflect the number of resources required by this test.
changelog_begin
changelog_end
* Change error code for invalid offsets for transaction stream and completion stream requests
* Expanded application architecture docs on how to build application with ledger api failover capabilities.
Fixes#6842.
CHANGELOG_BEGIN
- [Ledger API] The error code for requesting a transaction stream
with an offset beyond the ledger end changed from INVALID_ARGUMENT
to OUT_OF_RANGE. This makes it easier to handle scenarios where
an application fails over to a backup participant which hasn't
caught up with the ledger yet.
- [Ledger API] The command completion service now validates the offset and
returns the OUT_OF_RANGE error if the request offset is beyond the ledger end.
- [Documentation] Added a section on how to write DAML applications
that can fail over between multiple eventually consistent Ledger API endpoints
where command deduplication works across these Ledger API endpoints, which
can be useful for addressing HA and/or DR scenarios.
CHANGELOG_END
* Expose type synonyms in data-dependencies.
This PR converts (non-unit) type synonyms during LF conversion, and then
exposes them via data-dependencies. This is possible only when using an
LF version that supports type synonyms (LF version >= 1.7), the
type synonym isn't of unit type (because it clashes with empty
typeclasses, and the (fully applied) type synonym has kind *
(e.g. you can't define a * -> * synonym in LF).
This fixes issue #6306
changelog_begin
- [DAML Compiler] The DAML compiler will now compile type synonyms
(``type X = Y``) into the DAR, whenever possible, and will expose
these synonyms via data-dependencies.
changelog_end
* Add more tests
* Fix formatting
We currently use two different mechanisms for breaking loops in the
`Serializable` companion objects generated for each type:
1. Explicit thunks of the form `() => Decoder<A>`, e.g., in the
definition of the `Serializable` interface itself.
2. the `lazyMemo` combinator, e.g. in enum types, `Optional`s and lists.
This is inconsistent and leads to the introduction of more loop
breakers than actually needed.
This PR changes the situation to use the `lazyMemo` combinator as the
only loop breaker. To this end, we change the definition of the
`Serializable` interface and related similar interfaces to use `Decoder`
directly rather than the thunked up version. This is a *breaking change*
of the affected interfaces. However, the libraries `@daml/ledger` and
`@daml/react` are updated accordingly and hence nothing breaks when
using `daml2js` in combination with these libraries. Furthermore, the
exact definition of the decoder types is considered an implementation
detail since we don't want to tie ourselves long-term to the use of the
`json-type-validation` library, which is mostly unmaintained.
Using `lazy` as the only loop breaker effectively pushes all loop
breakers to the top-level and allows for removing those nested more
deeply in types.
This change should not negatively impact the performance of the
decoding process since it only replaces thunks by memoized thunks and
removes a few calls to `lazyMemo` completely. In fact, we should gain
performance since the decoders in the companion objects are now
memoized.
CHANGELOG_BEGIN
CHANGELOG_END
Currently, if you have a record type `T` with a field of type, say,
`Optional S` and you're decoding a list of type `[T]`, then the decoder
for `S` has to be reconstructed for each element of the list. This is
because the `lazy` combinator from the `json-type-validation` does not
memoize the decoder it receives as a thunk.
This PR adds a new combinator `lazyMemo` which behaves like `lazy` but
also memoizes the decoder on its first invocation. All use sites of the
old `lazy` combinator are then replaced with `lazyMemo`.
We could consider upstreaming `lazyMemo` but I'm not sure how much
effort this is given that `json-type-validation` seems to be in
maintenance mode rather than active development.
CHANGELOG_BEGIN
CHANGELOG_END
We’ve had a few confused users run into issues because of
this. `fsevents` (which is basically impossible to avoid as a
dependency) requires NodeJS 8.16 but for some reason Ubuntu 18.04
sticks to the unsupported 8.10.
changelog_begin
changelog_end
* DAML REPL :show imports
changelog_begin
- [DAML REPL] You can now list the currently imported modules using
the REPL command ``:show imports``.
changelog_end
* Accept \r\n on Windows.
* REPL functests use text mode file for stdin/out
So that `\r\n` will be mapped to `\n` automatically on Windows.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* daml-on-sql + sandbox: Remove unused Bazel `load` declarations.
* daml-on-sql: Build a Docker image.
* daml-on-sql: Move the example Kubernetes file over from Sandbox.
* Add a changelog entry that was dropped.
CHANGELOG_BEGIN
* [DAML on SQL] DAML on SQL is now available with every release,
from the GitHub Releases page.
CHANGELOG_END
* Upgrade the Java Docker base image to the latest version.
After the node resets this should hopefully not be necessary
anymore (we still had an issue this morning but I believe all nodes
that hit the issue also got the fix and if not, I’ll schedule a
targetted clean --expunge). I’ve also added node_modules to
.bazelignore to match the other node_modules directories.
changelog_begin
changelog_end
Not quite sure why this didn’t fail on the PR but now it complains
that the version of @bazel/typescript and rules_nodejs are
incompatible.
changelog_begin
changelog_end
* Log all authorization errors
CHANGELOG_BEGIN
- [Ledger API Server] The ledger API server now prints detailed log messages
whenever a request was rejected due to a failed
authorization.
CHANGELOG_END
I also switched zlib from a random looking commit to the corresponding
tag (which is the latest release) which points to the same commit.
changelog_begin
changelog_end
Currently, the decoders for the container types `List` and `TextMap` use
`jtv.lazy` _within_ their container decoders `jtv.array` and `jtv.dict`.
Since `jtv.lazy` does not memoize its result, this has the disadvantage
that the thunk producing the decoder for the element type is invoked
for each element of the container rather than once for the whole
container. This is a potential performance bottleneck.
This PR moves the use of `jtv.lazy` one level further out in oder to
avoid the potential performance issues without adding any further
downsides.
CHANGELOG_BEGIN
CHANGELOG_END
* Extend `daml new` to accept template as an option
The two positional arguments keep confusing users so this PR changes
things to allow the template to be passed via `--template`. Using a
positional argument still works so this is not breaking.
I’ve updated all docs to use the less confusing syntax.
changelog_begin
- [DAML Assistant] You can now use ``daml new project-name
--template=template-name`` instead of ``daml new project-name
template-name``. The old CLI syntax continues to be supported.
changelog_end
* Update docs/source/getting-started/index.rst
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Deduplicate REPL imports and store in map
changelog_begin
changelog_end
* DAML REPL subsumed imports functest
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* DAML REPL help and module commands
changelog_begin
- [DAML REPL] You can now type ``:help`` at the REPL prompt to see a
list of possible REPL commands.
- [DAML REPL] You can now add and remove module imports using
``:module [+-] Some.Module``.
changelog_end
* REPL :module functests
* Improve module not imported error message
* DAML REPL document REPL commands
* Update docs/source/daml-repl/index.rst
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
I think testing that constructing the decoders doesn't loop deserves
its own very targeted test on top of the more intergration style test
we already have.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix time model error message
CHANGELOG_BEGIN
CHANGELOG_END
* Restore ledger time based error message
* Add test for the error message
* Fix the error message
now for real?
* daml-on-sql: Pull out a new `Main` object that wraps sandbox-classic.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Fail if a JDBC URL is not provided or not for PostgreSQL.
* sandbox-classic: Rename the conformance test H2 database.
* daml-on-sql + sandbox-classic: Report configuration errors cleanly.
This means letting `ProgramResource` catch the errors, log, and exit.
* daml-on-sql: Change the name logged on startup.
* daml-on-sql: Change the default participant ID.
* sandbox-common: Give the ledger name its own tagged string type.
* sandbox-classic: Generate random ledger IDs using the ledger name.
* daml-on-sql: Remove the banner, replacing it with a blank line.
* daml-on-sql: Enable strong seeding by default.
And weak seeding in the conformance tests.
* sandbox-classic: Move the ledger name to a separate parameter.
It's not really configurable.
* sandbox-classic: Move LedgerName from sandbox-common.
* daml-on-sql: Remove "-participant" from the participant ID.
* daml-on-sql: Use `Name` where possible.
* daml-on-sql: Make the ledger ID mandatory.
* Revert "sandbox-classic: Move LedgerName from sandbox-common."
This reverts commit 0dad1584a7.
* daml-on-sql: Print "DAML-on-SQL" in the CLI help, not "Sandbox".
* daml-on-sql + sandbox + sandbox-classic: Split out custom CLI parsing. (#6846)
* participant-state: Simplify naming the seeding modes.
Based on feedback from @nickchapman-da, this PR aims at making the
release process easier by:
- Automatically opening a release PR on Wednesday morning. The goal here
is that by the time we start working, there is a release already
built, so we save about an hour on waiting for that. This obviously
doesn't help with ad-hoc releases.
- On a release PR build, posting to Slack when the release is ready to
merge.
- On a release master build, posting to Slack when a release is ready to
be tested.
My hope is that this makes the release process less tedious. This is not
trying to address the actual release testing, but hopefully should
reduce the annoyance of having to constantly go and check if the release
is ready.
CHANGELOG_BEGIN
CHANGELOG_END
GHC hates its users and defaults to optimizing out assertions. We
fixed that in Buck at some point but clearly that got lost when
migrating to Bazel.
Turns out enabling assertions catches bugs. This insight was brought to
you from the people that also brought you “Turns out writing tests
catches bugs”.
fixes#5624
changelog_begin
changelog_end
simplify docs cron
This commit changes the "live state" to be that all versions are there
on S3, most of them hidden the way snapshots currently are, and only
displays in the drop-down the list of "supported" versions, i.e. stable
and >= 1.0.0.
The docs cron will now:
- Get list of versions from GitHub (as it does now)
- Get list of versions from S3 (as it does now: versions.json +
snapshots.json, though it assumes we'll have a follow-up PR to change
the latter to hidden.json)
- Compare; if the sets of versions are the same, stop there. (Note: this
"set of versions" here includes the notion of which versions are shown,
not just which ones exist. See the Versions data type in the code.)
- If there is a new hidden version, just build that, push it, change
nothing else. No need to download any of the existing versions or mess
around with anything else (except updating `hidden.json`, otherwise
we're going to be doing this way too often.)
- If there is a new visible version:
- check if we have it locally (i.e. from the previous step: it's a
version we just added)
- figure out the old and new default versions, and then apply the diff
to the top-level directory. Basically download the two folders, list
files that exist in the old one and not in the new one, delete those
from S3, then push the new one to the top-level on S3.
- update versions.json & hidden.json (and for now snapshots.json)
This means that:
- we never mess with the existing versions; we don't need to download
them, we don't need to change them, we don't clean them up. Old links
keep working forever.
- The running time for the docs cron is roughly constant, in that it
should very rarely have to either build or upload (or download) more
than 2 versions per run, and if those instances happen they'd be
accidents (we made 3 actual releases in an hour), not build-up over
time.
* Populate out-of-time-bounds entry when we set min/max record time.
* Populate out-of-time-bounds entry when we set max record time for ConfigCommitter.
* Code tidying.
* Do not throw in case no min/max record time has been specified and there's no out-of-time-bounds log entry.
CHANGELOG_BEGIN
CHANGELOG_END
* Code tidying.
* Add ledger-on-memory conformance test with pre-execution enabled
* Revert "Add ledger-on-memory conformance test with pre-execution enabled"
This reverts commit d2c4364a
* Apply suggestions from code review
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
This does not change the version of rules_apple, it only pins the http
archive instead of fetching via git tag.
To avoid Bazel warnings of the following form since Bazel 3.3.1
```
DEBUG: Rule 'build_bazel_rules_apple' indicated that a canonical reproducible form can be obtained by modifying arguments commit = "ff6a37b24fcbbd525a5bf61692a12c810d0ee3c1", shallow_since = "1559833568 -0700" and dropping ["tag"]
DEBUG: Repository build_bazel_rules_apple instantiated at:
no stack (--record_rule_instantiation_callstack not enabled)
Repository rule git_repository defined at:
/home/aj/.cache/bazel/_bazel_aj/f66bee630c6a2cd906f92a0f5cdf8769/external/bazel_tools/tools/build_defs/repo/git.bzl:195:33: in <toplevel>
```
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This PR extends DAML REPL to also support `let` bindings which plays
well with the improved support for pure expressions. We support both
pattern bindings and function bindings.
changelog_begin
- [DAML REPL] DAML REPL can now run without a ledger. Take a look at
the documentation for details.
- [DAML REPL] DAML REPL now supports ``let`` bindings to bind pure
expressions. Take a look at the documentation for details.
changelog_end
This PR makes the ``--ledger-host`` and ``--ledger-port`` parameters
optional so DAML REPL works without a ledger which is useful now that
we have better. support for pure expressions. This just piggybacks on
DAML Script’s multiparticipant support so there are no significant
changes on the service.
Docs are updated and we have a testcase.
Side note: What is still missing is `let x = …` in DAML REPL. I’ll
tackle that in a separate PR.
changelog_begin
changelog_end
* Upgrade nixpkgs revision
* Remove unused minio
It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.
* Update Bazel on Windows
changelog_begin
changelog_end
* Fix hlint warnings
The nixpkgs update implied an hlint update which enabled new warnings.
* Fix "Error applying patch"
Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.
* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681
Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.
```
git diff ':(exclude)WORKSPACE'
```
* Update rules_nixpkgs
* nixpkgs location expansion escaping
* Drop --noincompatible_windows_native_test_wrapper
* client_server_test using sh_inline_test
client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.
This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.
* daml_test using sh_inline_test
* daml_doc_test using sh_inline_test
* _daml_validate_test using sh_inline_test
* damlc_compile_test using sh_inline_test
* client_server_test find .exe on Windows
* Bump Windows cache for Bazel update
Remove `clean --expunge` after merge.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
