--branches='*' seems to only include local branches not branches in
the remote. It looks like --branches='*' --remotes='*' would work but
--all seems simpler. I struggled to find any docs on this but this
matched what I got when testing locally.
changelog_begin
changelog_end
This PR gets yarn test running the Puppeteer end-to-end tests in the GSG integration test.
The first step of the test is adding the extra dependencies (Jest, Puppeteer and more). The GSG recommends a yarn add command, but this does not work against HEAD. This is because yarn add does not use resolutions in the parent package.json, and then complains about unknown versions 0.0.0 of the daml TS libaries. The solution here is to hack in the extra dependencies into the ui/package.json and then yarn install. This works, but it hard codes version numbers which we would need to maintain. I would like to be able to say version "latest", which is what yarn add would install.
The next step of the test is to copy the index.test.ts file and run yarn test in CI mode.
I've moved the GSG test to a new create-daml-app-tests target. It's marked "exclusive" in Bazel until we figure out how to avoid hardcoding port numbers. This is a bit tricky since the HTTP port is hardcoded in a couple of places in ui/package.json.
Finally, this PR gets the GSG testing docs to reference the code in the new templates/create-daml-app-test-resources folder.
changelog_begin
changelog_end
This PR finally puts together the refactorings from the last two PRs:
1. We now write out a JSON file (because JSON seems like a reasonable
choice and it doesn’t really matter) containing the main unit ids
when setting up the package db.
2. When initializing the GHC session, we now simply read that file
instead of reading the daml.yaml again and parsing all
dependencies and data-dependencies again.
changelog_begin
changelog_end
Instead, opt-in explicitly in _ledger-on-memory_ and _ledger-on-sql_.
Wrapping the operations can confuse other users of `LedgerStateAccess`.
CHANGELOG_BEGIN
CHANGELOG_END
This is another refactoring PR for createProjectPackageDb. There are
no actual changes but the `DependencyInfo` now includes the list of
main unit ids. This is not used in this PR but the plan is to start
emitting a file here containing those and then we only have to parse
that file instead of reading all DARs again just to figure out the
--package flags. (The reason why it needs to be written to a file
instead of simply returned is that this needs to work even with
--init-package-db=no which is going to become more important for
incremental package db initialization).
changelog_begin
changelog_end
* Upgrade rules_nodejs to version 1.6.0
closes#5367
This includes the fixes for the issues in jest that we’ve been seeing.
changelog_begin
changelog_end
* Fix eslint rules
* A bit of progress
* Try to add LinkablePackageInfo (doesn’t seem to work yet)
* Add rootDirs
* revert da_ts_library
* da_ts_library: add LinkablePackageInfo info
* Remove react hook workaround
Since rules_nodejs 1.6.0 this fails with the following error:
```
● Test suite failed to run
Configuration error:
Could not locate module react mapped as:
/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.
Please check your configuration for these entries:
{
"moduleNameMapper": {
"/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
},
"resolver": null
}
49 | // like a promis without being one.
50 | /* eslint-disable @typescript-eslint/no-floating-promises */
> 51 | var react_1 = __importStar(require("react"));
| ^
52 | var react_hooks_1 = require("@testing-library/react-hooks");
53 | var index_1 = __importStar(require("./index"));
54 | var events_1 = require("events");
at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
at Object.<anonymous> (index.test.js:51:28)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```
* rootDirs is not needed for tsc
This is only required for ts_project
* Update yarn Bazel packages
* docs/theme add missing dependencies
* Remove unused attribute module_root
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
As currently written, the check will compare the current commit with the
base commit the branch was forked from. The intention was to only list
the changes from the current PR, and to make the check work for PRs
against release branches (the previous version of the heck always
compared to master, regardless of what branch the PR was targeting).
However, this does not work as expected because the "current commit" is
not the tip of the PR, but the merge commit supplied by GitHub.
Therefore, the diff here will include not only the changes in this PR,
but also all the changes that happened on the target branch since
forking. This is not an issue if the PR is properly rebased, but that's
hard to control in a world where other people work too.
This PR corrects this by explicitly computing the diff between the fork
point on the target branch and the tip of the PR, ignoring the
currently-checked-out commit.
CHANGELOG_BEGIN
CHANGELOG_END
/gsg doesn’t work since it creates a file called gsg which your
browser will try to download.
the old quickstart url appears in google searches so we need to
redirect it.
changelog_begin
changelog_end
* Properly mangle/unmangle names in the scenario service
Previously, we just passed in whatever we had in DAML land which broke
the scenario service as soon as a name needed mangling. This PR fixes
mangling when passing identifiers to the scenario service and
unmangles them again when pretty printing.
fixes#5593
changelog_begin
- [DAML Studio] Fix a bug where scenarios with names that are mangled
in DAML-LF resulted in a crash in the scenario service.
changelog_end
* Mangle module names in context updates
This PR adds an --output-file option to DAML Script that writes the
result of a DAML Script to a file and complements the --input-file option.
changelog_begin
- [DAML Script] ``daml script`` now has a `--output-file`` option that
can be used to specify a file the result of the script should be
written to. Similar to ``--input-file`` the result will be output in
the DAML-LF JSON encoding.
changelog_end
* factor TlsConfiguration parser from extractor
* move TlsConfigurationParser to new library
* link extractor to ledger-service/cli-opts properly
* use TlsConfigurationCli in http-json, pass SslContext to ledger-client
* test TLS options as used in http-json
- the TLS config code is shared with extractor, where it is more fully
tested; we just do a sanity check here
* doc TLS options for http-json
CHANGELOG_BEGIN
- [JSON API] New ``--pem``, ``--crt``, ``--cacrt``, and ``--tls`` options
for securing the connection between JSON API server and ledger.
See `issue #2540 <https://github.com/digital-asset/daml/issues/2540>`__.
CHANGELOG_END
* TLS off in daml-script JSON API test
* ledger-api-common: Add a metric for command submission validation.
CHANGELOG_BEGIN
- [Ledger API Server] Add a metric for command validation upon
submission, ``daml.commands.validation``.
CHANGELOG_END
* ledger-api-common: Move the submissions metric from sandbox.
This commit aims at enabling future patch releases; it is the
master-branch equivalent of #5569 (applied to the 1.0 release branch).
The only change between the two changelogs should be that this one also
changes the docs cron so it can find the trigger commits for patch
releases.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Add a test for MetricsInterceptor for non-streaming endpoints.
* sandbox: Simplify MetricsInterceptor by overriding `ServerCall`.
* sandbox: Time all submissions, not just through the submission service.
* sandbox: Don't bother timing transaction submissions twice.
They're already timed by TimedWriteService.
* sandbox: Move command submission metrics to "daml.commands".
CHANGELOG_BEGIN
- [Ledger API Server] A number of command submission metrics have moved,
and we have added a couple.
- ``daml.commands.submissions`` is a new timer that measures all
submissions.
- ``daml.commands.valid_submissions`` is a new meter that counts
valid (unique, interpretable) submissions.
- ``daml.lapi.command_submission_service.failed_command_interpretations``
has been moved to ``daml.commands.failed_command_interpretations``.
- ``daml.lapi.command_submission_service.deduplicated_commands`` has
been moved to ``daml.commands.deduplicated_commands``.
- ``daml.lapi.command_submission_service.delayed_submissions`` has
been moved to ``daml.commands.delayed_submissions``.
- ``daml.lapi.command_submission_service.submitted_transactions`` has
been moved to ``daml.services.write.submit_transaction``.
- All ``daml.execution.*`` metrics have moved to
``daml.commands.execution``.
CHANGELOG_END
* metrics: Update the dashboard to include the new command metrics.
This is a first step in probably a series of PRs to cleanup the
packaging logic.
There is no change in functionality (or at least none that is
intended). The changes boil down to two things:
1. Add a `DependencyInfo` type that stores the various maps and has
detailed comments what exactly is in each map and hopefully somewhat
descriptive field names.
2. Move a lot of the logic in `createProjectPackageDb` into various
helper functions.
There is definitely a lot more cleanup and refactoring that we can do
here but I’ll try to keep it in manageable chunks.
changelog_begin
changelog_end
This has come up today where someone didn’t realize that removing the
field is sufficient to fix the warning.
I’ve also taken this as an opportunity to move the code out to a
helper function.
changelog_begin
changelog_end
* Add time provider type to ledger API server
Static time sometimes needs special treatment
* Delay submissions of transactions with future time
Fixes#5480.
CHANGELOG_BEGIN
- [Sandbox] The sandbox now properly delays command submissions using
minLedgerTimeAbs or minLedgerTimeRel. See
`issue #5480 <https://github.com/digital-asset/daml/issues/5480>`_.
CHANGELOG_END
* ledger-on-memory: Add metrics to the operations.
* ledger-on-memory: Speed up reads by using views.
* kvutils: Time event processing.
* ledger-on-(memory|sql): Make classes private and final where possible.
* kvutils: Factor out a TimedLedgerStateOperations class.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Report timing metrics for ledger state
operations.
CHANGELOG_END
* ledger-on-sql: Record log read metrics.
* ledger-on-sql: Record database transaction timing metrics.
CHANGELOG_BEGIN
- [Sandbox] Record ledger database timing metrics under "daml.ledger".
CHANGELOG_END
* ledger-on-sql: Time queries.
* metrics: Add graphs for read events and DAML-on-SQL.
* ledger-on-memory: Simplify the tuple swap in reading the log.
Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Fix conversion from contract to node ids in the scenario service
Previously we only included the mapping from the ledger. However, this
does not include any contracts which might have been created in a
partial transaction that failed. This caused a
`NoSuchElementException` when converting the error which is obviously
not what we want.
This PR extends the mapping to include any contracts created in the
current partial transaction.
changelog_begin
- [DAML Studio] Fix a bug where a failed transaction that included
references to a transient contract returned a gRPC error instead of
the expected error message.
changelog_end
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update compiler/scenario-service/server/src/main/scala/com/digitalasset/daml/lf/scenario/Conversions.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Improve error messages in daml repl on calls to `error`
There were two issues with calls to `error`:
1. This one is harmless but somewhat annoying: When calling `error` we
run into the log statement in `stepToValue` which prints out the
error message in a fairly reasonable form (you can argue whether
Error: User abort: is a super useful prefix but that’s a relatively
minor issue). Afterwards we then call `println` on the failed
future. However, that will just print the type of the exception
which isn’t something we want to show to users. I’ve just disabled
the println statement if the exception is `SError`.
2. This one is a bigger issue: `throw x` is not the same as
`Future.failed(x)`. I only fully realized the difference fairly
recently. The former fails before it produces a future. So `(throw
x).onComplete(…)` will never execute the callback. The latter is
just a failed future. It is rather confusing to have a function
that returns a future but then throws an exception instead of a
future and it confuses the grpc library which prints out a horrible
exception. I’ve changed all calls to `throw` in `runWithClients` to
instead use `Future.failed` and `flatMap` (in the form of
for-comprehensions).
There are still a few calls in `run` left which I’ll leave for a
separate PR.
I think we need to factor out some helper functions here to make this
a bit more manageable (e.g. for the Converter.toFuture) stuff but I’ll
leave that for a separate PR. You probably want to view this with
whitespace diffs disabled.
changelog_begin
- [DAML Repl] DAML Repl now produces better error messages on calls to
`error` and `abort`.
changelog_end
* Switch stepToValue to return Either
* resources: Rename `tearDownDuration` to `tearDownTimeout`.
* resources: Don't time out `ProgramResource` on startup.
Let it go for as long as it needs to.
Sometimes Sandbox takes a while to start, especially if you're
preloading large DARs. We should not try and predict how long it will
take.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Construct PartyConfiguration later.
* sandbox: Provide an option to disable implicit party allocation.
CHANGELOG_BEGIN
- [Sandbox] You can now disable implicit party allocation by passing the
flag, ``--implicit-party-allocation=false``. This makes it easier to
test as you would against another ledger which does not support this
feature.
CHANGELOG_END
mkConfFile was a bit of a mess before. Half of the arguments we passed
in via `PackageConfigFields` were unused as shown by the fact that we
set them to `error …` in various places where we did not have that
information. More importantly rather than passing in the unit ids
which need to end up in the GHC package config, we passed in file
names which required us to parse the DAR again and have a bunch of
hacks where file names with .dalf endings were not translated.
This PR changes this to only pass in the fields we need and pass in
dependencies directly as unit ids.
changelog_begin
changelog_end
This PR updates two points in the release process:
1. The process for getting release notes for a stable release based on
my experience of doing that for the past 2 releases.
1. It adds testing for the getting started guide. I did not remove the
testing of `quickstart-java`. I do want to test scenarios and
navigator and a bit of VSCode so the only potentially redundant
point there is the `mvn compile` step but that verifies that Maven
artifacts have been published successfully so I think it is still
useful.
changelog_begin
changelog_end
set up macOS nodes
This PR documents how to create and manage macOS CI nodes. Because macOS
is not supported by our current cloud providers, these instructions are
geared towards creating VMs on physical machines we would need to host
and manage ourselves, i.e. these notes are mostly targeted at Ed.
CHANGELOG_BEGIN
CHANGELOG_END
I recently noticed that the `check_for_release` and
`check_standard_change_label` jobs do not currently report their
runtime, so including them in the build data is a bit moot (we always
get `""` for all three values). Given that they usually run in under 3
seconds, I've decided the best way to fix this is to remove them from
the build data, rather than add the required steps to collect their
build times.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger/metrics: Move metric helpers to their own Bazel package.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use ledger/metrics.
* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.
Importing methods is harder than importing objects.
* metrics: Publish to Maven Central.
It looks like the change in Windows agent names has caused an issue:
because Windows agents are not always properly cleaned up on shutdown,
i.e. they do not always have time to tell Azure they are going away, and
because GCP likes to reuse the same names for machines in a group, we've
been seeing errors like:
```
ERROR: The running command stopped because the preference variable
"ErrorActionPreference" or common parameter is set to Stop: Pool 11
already contains an agent with name VSTS-WIN-3QCX.
```
recently. Today, only 2 out of our 6 agents have managed to register
with Azure. This PR should fix that.
ChaNGELOG_BEGIN
CHANGELOG_END
Looking at the behaviour of `succeededOrFailed`, it looks like it does
not do what we want at all: both steps now only run on failures. My
current hypothesis is that `write_ledger_dump` being skipped switches
the state of the last job to something that is neither success nor
failure.
It would be really nice if Azure had a way to detect cancellation. :(
CHANGELOG_BEGIN
CHANGELOG_END
This change adds a simple retry logic to the ledger websocket methods.
If the connection is closed, and the connection was previously healthy,
meaning it was live and running for more than 30 seconds, we try to
reconnect to the stream.
CHANGELOG_BEGIN
CHANGELOG_END
pgp.keyserver.io seems to have lost our key
somehow. pool.sks-keyservers.net is the pool that includes the MIT gpg
keyserver and others so hopefully it is somewhat reliable.
changelog_begin
changelog_end
