This has come up today where someone didn’t realize that removing the
field is sufficient to fix the warning.
I’ve also taken this as an opportunity to move the code out to a
helper function.
changelog_begin
changelog_end
* Add time provider type to ledger API server
Static time sometimes needs special treatment
* Delay submissions of transactions with future time
Fixes#5480.
CHANGELOG_BEGIN
- [Sandbox] The sandbox now properly delays command submissions using
minLedgerTimeAbs or minLedgerTimeRel. See
`issue #5480 <https://github.com/digital-asset/daml/issues/5480>`_.
CHANGELOG_END
* ledger-on-memory: Add metrics to the operations.
* ledger-on-memory: Speed up reads by using views.
* kvutils: Time event processing.
* ledger-on-(memory|sql): Make classes private and final where possible.
* kvutils: Factor out a TimedLedgerStateOperations class.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Report timing metrics for ledger state
operations.
CHANGELOG_END
* ledger-on-sql: Record log read metrics.
* ledger-on-sql: Record database transaction timing metrics.
CHANGELOG_BEGIN
- [Sandbox] Record ledger database timing metrics under "daml.ledger".
CHANGELOG_END
* ledger-on-sql: Time queries.
* metrics: Add graphs for read events and DAML-on-SQL.
* ledger-on-memory: Simplify the tuple swap in reading the log.
Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Fix conversion from contract to node ids in the scenario service
Previously we only included the mapping from the ledger. However, this
does not include any contracts which might have been created in a
partial transaction that failed. This caused a
`NoSuchElementException` when converting the error which is obviously
not what we want.
This PR extends the mapping to include any contracts created in the
current partial transaction.
changelog_begin
- [DAML Studio] Fix a bug where a failed transaction that included
references to a transient contract returned a gRPC error instead of
the expected error message.
changelog_end
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Improve error messages in daml repl on calls to `error`
There were two issues with calls to `error`:
1. This one is harmless but somewhat annoying: When calling `error` we
run into the log statement in `stepToValue` which prints out the
error message in a fairly reasonable form (you can argue whether
Error: User abort: is a super useful prefix but that’s a relatively
minor issue). Afterwards we then call `println` on the failed
future. However, that will just print the type of the exception
which isn’t something we want to show to users. I’ve just disabled
the println statement if the exception is `SError`.
2. This one is a bigger issue: `throw x` is not the same as
`Future.failed(x)`. I only fully realized the difference fairly
recently. The former fails before it produces a future. So `(throw
x).onComplete(…)` will never execute the callback. The latter is
just a failed future. It is rather confusing to have a function
that returns a future but then throws an exception instead of a
future and it confuses the grpc library which prints out a horrible
exception. I’ve changed all calls to `throw` in `runWithClients` to
instead use `Future.failed` and `flatMap` (in the form of
for-comprehensions).
There are still a few calls in `run` left which I’ll leave for a
separate PR.
I think we need to factor out some helper functions here to make this
a bit more manageable (e.g. for the Converter.toFuture) stuff but I’ll
leave that for a separate PR. You probably want to view this with
whitespace diffs disabled.
changelog_begin
- [DAML Repl] DAML Repl now produces better error messages on calls to
`error` and `abort`.
changelog_end
* Switch stepToValue to return Either
* resources: Rename `tearDownDuration` to `tearDownTimeout`.
* resources: Don't time out `ProgramResource` on startup.
Let it go for as long as it needs to.
Sometimes Sandbox takes a while to start, especially if you're
preloading large DARs. We should not try and predict how long it will
take.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Construct PartyConfiguration later.
* sandbox: Provide an option to disable implicit party allocation.
CHANGELOG_BEGIN
- [Sandbox] You can now disable implicit party allocation by passing the
flag, ``--implicit-party-allocation=false``. This makes it easier to
test as you would against another ledger which does not support this
feature.
CHANGELOG_END
mkConfFile was a bit of a mess before. Half of the arguments we passed
in via `PackageConfigFields` were unused as shown by the fact that we
set them to `error …` in various places where we did not have that
information. More importantly rather than passing in the unit ids
which need to end up in the GHC package config, we passed in file
names which required us to parse the DAR again and have a bunch of
hacks where file names with .dalf endings were not translated.
This PR changes this to only pass in the fields we need and pass in
dependencies directly as unit ids.
changelog_begin
changelog_end
This PR updates two points in the release process:
1. The process for getting release notes for a stable release based on
my experience of doing that for the past 2 releases.
1. It adds testing for the getting started guide. I did not remove the
testing of `quickstart-java`. I do want to test scenarios and
navigator and a bit of VSCode so the only potentially redundant
point there is the `mvn compile` step but that verifies that Maven
artifacts have been published successfully so I think it is still
useful.
changelog_begin
changelog_end
set up macOS nodes
This PR documents how to create and manage macOS CI nodes. Because macOS
is not supported by our current cloud providers, these instructions are
geared towards creating VMs on physical machines we would need to host
and manage ourselves, i.e. these notes are mostly targeted at Ed.
CHANGELOG_BEGIN
CHANGELOG_END
I recently noticed that the `check_for_release` and
`check_standard_change_label` jobs do not currently report their
runtime, so including them in the build data is a bit moot (we always
get `""` for all three values). Given that they usually run in under 3
seconds, I've decided the best way to fix this is to remove them from
the build data, rather than add the required steps to collect their
build times.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger/metrics: Move metric helpers to their own Bazel package.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use ledger/metrics.
* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.
Importing methods is harder than importing objects.
* metrics: Publish to Maven Central.
It looks like the change in Windows agent names has caused an issue:
because Windows agents are not always properly cleaned up on shutdown,
i.e. they do not always have time to tell Azure they are going away, and
because GCP likes to reuse the same names for machines in a group, we've
been seeing errors like:
```
ERROR: The running command stopped because the preference variable
"ErrorActionPreference" or common parameter is set to Stop: Pool 11
already contains an agent with name VSTS-WIN-3QCX.
```
recently. Today, only 2 out of our 6 agents have managed to register
with Azure. This PR should fix that.
ChaNGELOG_BEGIN
CHANGELOG_END
Looking at the behaviour of `succeededOrFailed`, it looks like it does
not do what we want at all: both steps now only run on failures. My
current hypothesis is that `write_ledger_dump` being skipped switches
the state of the last job to something that is neither success nor
failure.
It would be really nice if Azure had a way to detect cancellation. :(
CHANGELOG_BEGIN
CHANGELOG_END
This change adds a simple retry logic to the ledger websocket methods.
If the connection is closed, and the connection was previously healthy,
meaning it was live and running for more than 30 seconds, we try to
reconnect to the stream.
CHANGELOG_BEGIN
CHANGELOG_END
pgp.keyserver.io seems to have lost our key
somehow. pool.sks-keyservers.net is the pool that includes the MIT gpg
keyserver and others so hopefully it is somewhat reliable.
changelog_begin
changelog_end
* Diff with messaging feature and some noise manually removed
* Bazel target to use patch file in other build targets
* Patch file as data dep for integration tests
* Attempt to patch and test messaging feature in create-daml-app test
changelog_begin
changelog_end
* Use exports_files instead of filegroup
* Remove file existence checks that don't make sense
* Add patch to dev_env and reference it from integration tests
* Include patch on windows for later
* Set up yarn env again after codegen
* Restore file check
* Fix typo in comment on util function
* Add Tasty steps to make process explicit
* Use messaging patch for code snippets in GSG
* Use messaging code from template instead of copy
* Remove copied message code
* Refactor script to copy template code with messaging patch
* No need to retry second yarn install (only local deps should be updated)
* Used `daml codegen java` instead of calling the codegen from maven
This should hopefully fix the issues with mismatched versions of
slf4j.
changelog_begin
changelog_end
* Move config to daml.yaml
* Remove alternative invocation via maven from docs
We started serving the LedgerAPI from the new schema in #5104, #5305, #5346,
and #5397.
We must therefore also add a migration that moves the data from
ledger_entries to the new tables.
To keep the migration relatively stable with respect to refactoring the
code, I copied a bunch of files to
`db.migration.postgres.v25_backfill_participant_events`.
As a bonus, I added a line to logback.xml so that we get the usual
sandbox output on startup.
CHANGELOG_BEGIN
CHANGELOG_END
* participant_contracts write path
Writes contracts alongside events for interpretation and validation.
changelog_begin
changelog_end
* Fix self-referencing foreign key
* Address https://github.com/digital-asset/daml/pull/5522#discussion_r406289145
* Compute divulgence only if necessary
* Do not attempt at persisting witnesses for transient contracts
* Drop new tables when resetting
* Fix unstableSplitOn
unstableSplitOn is supposed to behave like splitOn but fast™ but it
didn’t live up to that promise for a few reasons:
1. Java’s split method by default drops empty strings at the
end. Passing a second negative parameter avoids that.
2. Java’s split method does regex splitting so we need to quote the
string.
3. Java’s split method splits on an empty pattern whereas we want to
simply preserve the input.
I’ve added regression tests for this and for unstableDrop and a few
other functions while I was at it.
changelog_begin
changelog_end
* Update compiler/damlc/tests/daml-test-files/UnstableText.daml
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* sandbox: Time command execution.
* sandbox: Tiny cleanups in StoreBackedCommandExecutor.
* sandbox: Add metrics for fetching within execution.
CHANGELOG_BEGIN
- [Ledger API Server] Add metrics under "daml.execution" to measure
command execution time, and fetching within command execution.
CHANGELOG_END
* sandbox: Revert the ApiSubmissionService metric prefix.
It can be private again.
* sandbox: Record a mark on a meter on command execution retries.
* Haskell: Add assertFileExists to DA.Test.Util
This PR adds a helper function `assertFileExists` that captures the
`doesFileExist ... >>= assertBool ...` pattern that is very common in
our Haskell test suites. It also adds the inverse
`assertFileDoesNotExist` function. Both functions are now used where
appropriate.
CHANGELOG_BEGIN
CHANGELOG_END
* Add directory dependency to test-util lib
CHANGELOG_BEGIN
CHANGELOG_END
This is a spin off from the work for making module prefixing
work. There I wanted to be able to use the `GeneratePackageMap` rule
to get access to the modules in an LF package which requires to be in
an Action rather than in IO.
This resulted in a much larger refactoring than I expected, so here is
a list of changes to ease review:
1. I’ve split off Development/IDE/Core/RuleTypes/Daml.hs into its own
Bazel targets to avoid cycles. This allows us to use a rule without
depending on the implementation of that rule.
2. Development/IDE/Core/IdeState/Daml.hs has moved into its own Bazel
target. There is a lot of stuff that needs `withDamlIdeState` now
since the `ghcide` `initialise` function does not add our Shake rule
for getting the GHC session and making all of that depend on
damlc:lib creates a mess.
3. The actual logic of the initialization has stayed the same but
moved into a rule in Development/IDE/Core/Rules/Daml.hs.
4. As mentioned above, a few things that could previously get away
with only the rules provided by ghcide now need our own rules so
they switched to `withDamlIdeState`. This mainly affects `damlc
docs`.
5. Making daml-docs work with the ofInterestRule is a bit tricky.I did
find the issue and included the fix but also disabled the
ofInterestRule since it really does not make sense to generate Core
in the background.
changelog_begin
changelog_end
- removes unnecessary reference to offsets
- pushes event identifier translation out to enable code re-use
- event identifier translation is inline in other iterations, no significant perf degradation
- this will allow to share code between event and contract witness table objects
changelog_begin
changelog_end
* sandbox: Remove `ErrorCause.Sequencer`.
It doesn't seem to be constructed anywhere.
Feel free to close this if it's used outside the repository.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Don't deconstruct when we can check types in a pattern match.
* sandbox: Move command execution into its own package.
* sandbox: Make LedgerTimeHelper into a CommandExecutor implementation.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Rename CommandExecutorImpl to StoreBackedCommandExecutor.
And pass it objects, not functions.
* sandbox: Reorder result cases in StoreBackedCommandExecutor.
* sandbox: Inject the LedgerTimeAwareCommandExecutor.
* sandbox: Pull out the default time provider type into a constant.
* sandbox: Name ResetService tests consistently.
* sandbox: Reset the time service when resetting in Static Time mode.
The test, unfortunately, is ``@Ignore`d due to flakiness in CI, so won't
actually be run. However, I _hope_ we're going to remove that
annotation eventually, and it allowed me to test-drive the fix on my
machine, so it's still helpful.
CHANGELOG_BEGIN
- [Sandbox] Fix a regression in the ResetService which did not reset the
TimeService in static time mode.
CHANGELOG_END
Turns out not every participant can support seeding (yet).
CHANGELOG_BEGIN
- [Ledger API Server] Re-introduce an option to disable seeding. This
does not affect Sandbox.
CHANGELOG_END
