We change the color from red to yellow to inform about the possibility
to rebuild and also output the success message in green.
CHANGELOG_BEGIN
CHANGELOG_END
* Draw the rest of the Scala 2.13 owl
Not quite but pretty close and this switches us over from inclusions
to exclusions which makes it much easier to track.
Ledger API test tool should be fixed by #8821. Non-repudiation needs a
tiny bit of work since unwrapArray doesn’t work the same on 2.13 but
shouldn’t be hard to fix.
changelog_begin
changelog_end
* Fix ScriptService tests
Those tests were all dumb. They asserted on a fixed order while the
function to sort the things was broken so we ended up with the random
Map order which is unsurprisingly not the same.
This is easily fixed by fixing the sort function.
There is also a second issue with query not sorting.
changelog_begin
changelog_end
* Turns out if you fix one test the next one breaks
And clearly nobody ever tested this or give this a second thought.
changelog_begin
changelog_end
* Use cliopts.Http for OAuth 2.0 middleware
changelog_begin
- [OAuth 2.0 Middleware] You can now configure the address that the
middleware listens to using the ``--address`` flag.
The port that the middleware listens to is now configured using the
``--http-port`` flag, use 0 to dynamically choose a free port.
You can now configure a port file where the chosen port will be
written to using the ``--port-file`` flag.
changelog_end
* Add test-case for OAuth 2.0 middleware port file
* Don't forget to close source
* Fix integration test
* Update triggers/service/auth/src/main/scala/com/daml/auth/middleware/oauth2/Config.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* add livez endpoint to auth middleware
* Add OAuth 2.0 middleware to Daml SDK
* unhide trigger service auth flags
changelog_begin
- [Triggers] The trigger service now supports authorization through an
auth middleware. The feature is enabled using the `--auth` and
`--auth-callback` command-line flags. Please refer to the
Authorization chapter of the trigger service documentation for further
instructions.
- [OAuth 2.0 middleware] Daml Connect now includes an implementation of
the auth middleware API that supports OAuth 2.0 Authorization Code
Grant. Please refer to the Auth Middleware and OAuth 2.0 Auth
Middleware chapters of the documentation.
changelog_end
* drop early access flag on triggers
Daml triggers, the trigger service, and the auth middleware are no
longer marked as early access features.
changelog_begin
- [Triggers] Daml Triggers and the Trigger Service are no longer in
early access status.
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
To avoid prompting the user to set the PATH variable on CI, we check the
`CI` environment variable and do nothing if set.
CHANGELOG_BEGIN
CHANGELOG_END
* fix: project parties are not required
This fixes a bug in daml assistant. `daml deploy` would fail if there
were no project parties listed in the daml.yaml project config file. Now
project parties are not required anymore.
Fixes#8487.
CHANGELOG_BEGIN
CHANGELOG_END
* added an integration test
We use the new cache folder specified with `DAML_CACHE` defaulting to
`.cache/daml` as a new cache folder for the assistant to store available
versions etc.
This does not migrate an existing cache folder and just starts a new
one.
CHANGELOG_BEGIN
CHANGELOG_END
* telemetry: use an extra cache directory for telemetry
If the sdk's `.daml` directory is not writable, telemetry fails and
kills the IDE. We add a new assistant environment variable "DAML_CACHE".
If set, this directory is used for telemetry cache data.
Fixes#8396.
CHANGELOG_BEGIN
CHANGELOG_END
* try creating directory
* check for already existing machine id file
* set DAML_CACHE to read-only in assistant integration tests
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
The one thing that is still missing is making the generated Scala code
from the codegen compatible with Scala 2.13 so the examples are
excluded for now.
changelog_begin
changelog_end
* fix: navigator regression: create empty config file.
For backwards compatibility, `daml ledger navigator` needs to create an
empty `ui-backend.conf` file when invoced outside a project directory.
Otherwise, navigator will fail to start.
CHANGELOG_BEGIN
CHANGELOG_END
* integration test
* Replace many occurrences of DAML with Daml
* Update docs logo
* A few more CLI occurrences
CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END
* Fix some over-eager replacements
* A few mor occurrences in md files
* Address comments in *.proto files
* Change case in comments and strings in .ts files
* Revert changes to frozen proto files
* Also revert LF 1.11
* Update get-daml.sh
* Update windows installer
* Include .py files
* Include comments in .daml files
* More instances in the assistant CLI
* some more help texts
* Support DAML_PROJECT in daml codegen java
fixes#8406
We need to resolve the DAR path relative to the project
root. Otherwise, we’ll just fail to find the file if daml codegen java
is invoked from a different directory.
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* fmt lalala
changelog_begin
changelog_end
* fix another test
changelog_begin
changelog_end
* assistant: get the latest sdk version form docs.daml.com
We get the latest sdk version from docs.daml.com/latest instead of
github.com. Getting the latest release from github proved to be tricky
because github defines 'latest' by the tag timestamp.
CHANGELOG_BEGIN
CHANGELOG_END
* hlint
* navigator: ignore project parties flag
This adds a flag '--ignoreProjectParties' to navigator such that
parties specified in the daml.yaml file are ignored and the
`PartyManagementService` is queried for parties instead.
This is part of #6099.
CHANGELOG_BEGIN
CHANGELOG_END
* add flag to
* add flag to daml ledger navigator
* Update navigator/backend/src/main/scala/com/digitalasset/navigator/config/Arguments.scala
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* remove config file from daml ledger navigator command
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* hotreload: archive active contracts
Fixes#8162. This archives all active contracts on the ledger when a
hotreload is performed. A hidden command `reset` is added to `daml
ledger` to perform the reset against a running sandbox.
CHANGELOG_BEGIN
CHANGELOG_END
* don't reset when no parties are allocated
* hlint
* reuse tokenFor
* remove unused imports
* add applicationId to token
* remove readAs field from command
* use uuid as cmdId
* remove token from reset command
* batch cmds in size of 100
* remove comment about todo, remove readAs from token
Also disables auto-install on Windows, since installation-related
errors range from "simple error message" to "program hangs forever",
and users can bypass "daml install" by downloading the new installer.
changelog_begin
- [DAML Assistant] The update-check is disabled by default for
newer installations. In addition, auto-install is disabled by
default on Windows. See here for instructions on how to
configure these settings manually: https://docs.daml.com/tools/assistant.html#global-config-file-daml-config-yaml
changelog_end
* install script: check available disk space
This fixes#6993. We check that /tmp offers enough space to extract the
sdk tarball and fail otherwise. An alternative extraction directory can
be set by setting the TEMPDIR environment variable.
CHANGELOG_BEGIN
CHANGELOG_END
* don't remove user specified directory
CHANGELOG_BEGIN
CHANGELOG_END
* Update daml-assistant/get-daml.sh
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
These are basically identical to create-daml-app-tests-proj-name
except for the fact that they use the default project name and they
are super slow so we don’t want to run them twice.
changelog_begin
changelog_end
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.
This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.
Apologies for the giant PR, I don’t see a way to keep it smaller.
changelog_begin
changelog_end
* Dont look for daml.yaml in SDK install.
Unless you're running "daml install project".
Fixes#5720
changelog_begin
changelog_end
* use pattern synonym for raw install target
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.
* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.
* Bazel: Factor out Java/Scala protobuf class generation into a helper.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/archive: Use `proto_jars`.
* Bazel: Remove the visibility modifier from `proto_jars`.
It's too confusing. Just make everything public.
* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.
* Bazel: Add comments to the various parts of `proto_jars`.
* daml-assistant: Do unpleasant things with `location` in Bazel.
* daml ledger: add a max-inbound-message-size flag for grpc
This new flag allows to set the maximal accepted inbound message size
when using gRPC to interact with a legger.
CHANGELOG_BEGIN
CHANGELOG_END
* added a test
* daml ledger: implementation of ledger commands
Implementation of the remaining daml ledger commands via HTTP JSON
service endpoints instead of gRPC.
Fixes#4824 .
CHANGELOG_BEGIN
[daml assistant] `daml ledger` commands can now also be run against the JSON API instead of the gRPC API.
CHANGELOG_END
* assistant: prompt users to automatically update PATH
We ask the Max/Linux users on installation if we should update their
PATH environment variable to point to the assistant binary and
automatically do so when the answer is affirmative.
CHANGELOG_BEGIN
CHANGELOG_END
* set .profile insetad .bashrc
* add a case for sh
* fix tests
CHANGELOG_BEGIN
CHANGELOG_END
* use imports only on nix systems
* update windows updatePath function for YesNoAuto flag.
CHANGELOG_BEGIN
CHANGELOG_END
* assistant tests: Testing hot-reload
This adds a test to the assistant integration tests for testing the
hot-reload feature.
CHANGELOG_BEGIN
CHANGELOG_END
* hlint
We rerun the initialization script for every rebuild. In addition we fix
a potential bug that would lead to a wrong sandbox port when trying to
re-upload the package.
CHANGELOG_BEGIN
CHANGELOG_END
* daml ledger: Infrastructure for tests
This adds infrastructure for testing `daml ledger` and moves one test
from the assistant integration tests to the newly created tests.
CHANGELOG_BEGIN
CHANGELOG_END
* Update libs-haskell/test-utils/DA/Test/HttpJson.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Update libs-haskell/test-utils/DA/Test/HttpJson.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* small fixes
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Currently we catch the exception and then print out "DAR upload
succeeded.". That’s clearly not what is intended here.
changelog_begin
- [DAML Assistant] `daml ledger upload-dar` now exits with a non-zero
exit code on failures and no longer prints "DAR upload succeeded"
in error cases.
changelog_end
The `downloadAllReachablePackages` function, which is part of the
implementation of `featch-dar`, uses an associative list when a `Map`
seems more appropriate. More appropriate in both runtime complexity as
well as code complexity. This has caught my eye while reviewing some
code nearby and I thought I have a minute to spare to improve it.
This PR replaces the associative list with a `Map`.
CHANGELOG_BEGIN
CHANGELOG_END
* daml assistant: cleanup of daml ledger commands
This is a cleanup of the Helper/Ledger.hs module in preparation for
further implementations of `daml ledger` commands against the JSON API.
In particular the module Compiler/Fetch.hs is removed from daml-compiler
and its contents moved over the Helper/Ledger.hs. Some renaming and
reorganization is done for clarity.
CHANGELOG_BEGIN
CHANGELOG_END
* language: `daml ledger` commands against JSON API
This is the first step in offering the JSON API to run `daml ledger`
commands. First, only the `list-parties` command is implemented. The
other `daml ledger` commands will follow in separate PR's.
CHANGELOG_BEGIN
[Assistant] The `daml ledger list-parties` command can now query the
ledger via the HTTP JSON API instead the gRPC API for known parties on
the ledger.
CHANGELOG_END
This makes the `daml start` process listen for a key-press. When 'r' is
pressed, the package is re-build, the new package uploaded to the
in-memory sandbox and the codegen re-run.
Note: While we clean previous outputs of the code generation, there is
currently no way to drop previously uploaded packages from the in-memory
sandbox. However, it will take many re-uploads until this will become a
problem.
CHANGELOG_BEGIN
[daml start] You can now press 'r' (or 'r' + 'Enter' on Windows) in the
terminal where `daml start` is running to rebuild the DAR package and
generate JavaScript/Java/Scala bindings and upload the new package to
the sandbox. This frees the user from killing and restarting `daml
start`.
CHANGELOG_END
In preparation for the renaming, we are stripping the DAML prefix. It
is still called SDK and the sdk-version variable keeps existing.
changelog_begin
changelog_end
This fixes the parsing of the npm-scope field of the project config,
where a space is accidentally introduced.
CHANGELOG_BEGIN
[Codegen] Fix for an accidentally introduced space in the npm scope of
genreated JavaSript libraries when it was parsed from the daml.yaml
project config.
CHANGELOG_END
This executes the code generations specified in the daml.yaml
configuration file on every invocation of `daml start`
and hence frees the user of doing it manually.
CHANGELOG_BEGIN
- [DAML Assistant] The `daml start` now runs all the code generators
specified in the `daml.yaml` project configuration file under the
`codegen` stanza. This frees the user of doing so manually on every
change to the DAML model.
CHANGELOG_END
* Improve the user error message if the upload-dar command fails.
In particular we want to catch the case of a gRPC timeout exception which we know can occur if the DAR is too large. But it seems helpful to catch any exception, and shows what we caught along with the helpful message.
* use safe-exceptions
We deprecate specifying the template when calling `daml new` via a
positional argument, as in
```sh
daml new foo skeleton
```
The new syntax is
```sh
daml new foo --template skeleton
```
Whenever the former version is used, we now print a not that it is
deprecated and that the latter version is the recommended way.
CHANGELOG_BEGIN
[DAML Assistant]
- Deprecate specifying the template for `daml new` via a positional
argument.
CHANGELOG_END
This PR implements a part of the proposal from #7093.
Here packages are validated in the participant node before to be sent to the ledger.
CHANGELOG_BEGIN
- [Ledger-API] participant node validate Dar before uploading to the ledger.
This may increase upload time significantly.
CHANGELOG_END
* Add --navigator-port option in daml start.
Fixes#5777
changelog_begin
- [DAML SDK] Added a ``--navigator-port`` option in ``daml start``,
allowing you to specify the port for navigator's web server.
changelog_end
* Perform defaulting in optparse-applicative
Fixes#7381.
```
$ alias daml=~/.daml/sdk/0.0.0/daml/daml
$ daml version
DAML SDK versions:
0.0.0
1.4.0
1.5.0-snapshot.20200811.4959.0.bbc2fe56
1.5.0-snapshot.20200818.5027.0.1b33d374
1.5.0-snapshot.20200902.5118.0.2b3cf1b3 (default SDK version for new projects)
$ DAML_SDK_VERSION=invalid daml version
daml: Invalid value for environment variable DAML_SDK_VERSION.
context: Determining SDK version and path.
details: Invalid SDK version "invalid": Failed reading: takeWhile1
$ DAML_SDK_VERSION=1.4.4 daml version
DAML SDK versions:
0.0.0
1.4.0
1.4.4 (selected by env var DAML_SDK_VERSION, not installed)
1.5.0-snapshot.20200811.4959.0.bbc2fe56
1.5.0-snapshot.20200818.5027.0.1b33d374
1.5.0-snapshot.20200902.5118.0.2b3cf1b3 (default SDK version for new projects)
$ DAML_SDK_VERSION=1.4.0 daml version
DAML SDK versions:
0.0.0
1.4.0 (selected by env var DAML_SDK_VERSION)
1.5.0-snapshot.20200811.4959.0.bbc2fe56
1.5.0-snapshot.20200818.5027.0.1b33d374
1.5.0-snapshot.20200902.5118.0.2b3cf1b3 (default SDK version for new projects)
$ DAML_SDK_VERSION=1.5.0-snapshot.20200902.5118.0.2b3cf1b3 daml version
DAML SDK versions:
0.0.0
1.4.0
1.5.0-snapshot.20200811.4959.0.bbc2fe56
1.5.0-snapshot.20200818.5027.0.1b33d374
1.5.0-snapshot.20200902.5118.0.2b3cf1b3 (selected by env var DAML_SDK_VERSION, default SDK version for new projects)
$
```
CHANGELOG_BEGIN
CHANGELOG_END
Previously, it was filtered out by accident since damlc considered it
to be an old-style typeclass. This PR fixes this by adding a dummy
field.
This is primarily useful in DAML REPL since all DARs there are
importad as data-dependencies atm. It’s not actually all that useful
across SDK versions since you end up with multiple daml-script
libraries but at least within an SDK you can use it and don’t have to
think about whether your project is a dependency or data-dependency.
changelog_begin
changelog_end
NPM doesn’t actually need this (at least it didn’t for me locally and
hopefully CI agrees) to pick up changes and it emits a very
scary-looking warning if you do pass it.
changelog_begin
changelog_end
* Migrate skeleton to DAML Script
Also kills the separate setup module since it’s the same thing
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* Remove authentication service from trigger service
This rips out the integration with ref-ledger-authenticator and
together with that the weird http basic auth which doesn’t actually
work for ref-ledger-authenticator outside of its testing mode and also
includes some very sketchy stuff like storing user
passwords (encrypted but still).
While the exact details of the new approach towards auth are still not
completely clear, it is clear that it is going to be sufficiently
different from what we did before that starting from a clean slate is
easier.
changelog_begin
- [Trigger Service] Parties are now specified in request bodies as
opposed to via HTTP Basic auth. This is done in preparation for
running the trigger service against authenticated ledgers.
changelog_end
* Fix integration tests
changelog_begin
changelog_end
Part of #6977. This PR adds some assistant tests, but regression tests and documentation should be added before the issue is closed.
changelog_begin
changelog_end
* add -Ywarn-unused to all scalac options
* remove some unused arguments
* remove some unused definitions
* remove some unused variable names
* suppress some unused variable names
* changeExtension doesn't use baseName
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* work around no plugins in scenario interpreter perf tests
* remove many more unused things
* remove more unused things, restore some used things
* remove more unused things, restore a couple signature mistakes
* missed import
* unused argument
* remove more unused loggingContexts
* some unused code in triggers
* some unused code in sandbox and kvutils
* some unused code in repl-service and daml-script
* some unused code in bindings-rxjava tests
* some unused code in triggers runner
* more comments on silent usages
- suggested by @cocreature; thanks
* fix missing reference in TestCommands
* more unused in triggers
* more unused in sandbox
* more unused in daml-script
* more unused in ledger-client tests
* more unused in triggers
* more unused in kvutils
* more unused in daml-script
* more unused in sandbox
* remove unused in ledger-api-test-tool
* suppress final special case for codegen unused warnings
.../com/daml/sample/mymain/ContractIdNT.scala:24: warning: parameter value ev 0 in method ContractIdNT Value is never used
implicit def `ContractIdNT Value`[a_a1dk](implicit `ev 0`: ` lfdomainapi`.Value[a_a1dk]): ` lfdomainapi`.Value[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
^
.../com/daml/sample/mymain/ContractIdNT.scala:41: warning: parameter value eva_a1dk in method ContractIdNT LfEncodable is never used
implicit def `ContractIdNT LfEncodable`[a_a1dk](implicit eva_a1dk: ` lfdomainapi`.encoding.LfEncodable[a_a1dk]): ` lfdomainapi`.encoding.LfEncodable[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
^
* one more unused in daml-script
* special scaladoc rules may need silencer, too
* unused in compatibility/sandbox-migration
* more commas, a different way to `find`
- suggested by @remyhaemmerle-da; thanks
* Factor out tar/gzip reproducibility flags
* use mktgz in package-app
* Bazel managed tar/gzip
* Remove quiet = True
As stated in the comment this is no longer required with Bazel >= 3.0.
* Build package-app as a sh_binary
This way Bazel will manage the runtime dependencies tar, gzip, mktgz,
and patchelf.
package-app.sh changes directory so it needs to make sure that all paths
are absolute and that the runfiles tree/manifest location is forwarded
to programs called by package-app.sh.
* Avoid file path too long errors
* Fix readlink -f on MacOS
* Document abspath
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Extend `daml new` to accept template as an option
The two positional arguments keep confusing users so this PR changes
things to allow the template to be passed via `--template`. Using a
positional argument still works so this is not breaking.
I’ve updated all docs to use the less confusing syntax.
changelog_begin
- [DAML Assistant] You can now use ``daml new project-name
--template=template-name`` instead of ``daml new project-name
template-name``. The old CLI syntax continues to be supported.
changelog_end
* Update docs/source/getting-started/index.rst
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Upgrade nixpkgs revision
* Remove unused minio
It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.
* Update Bazel on Windows
changelog_begin
changelog_end
* Fix hlint warnings
The nixpkgs update implied an hlint update which enabled new warnings.
* Fix "Error applying patch"
Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.
* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681
Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.
```
git diff ':(exclude)WORKSPACE'
```
* Update rules_nixpkgs
* nixpkgs location expansion escaping
* Drop --noincompatible_windows_native_test_wrapper
* client_server_test using sh_inline_test
client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.
This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.
* daml_test using sh_inline_test
* daml_doc_test using sh_inline_test
* _daml_validate_test using sh_inline_test
* damlc_compile_test using sh_inline_test
* client_server_test find .exe on Windows
* Bump Windows cache for Bazel update
Remove `clean --expunge` after merge.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This limits the JVM max memory and initial memory in a few tests that
look like they might be using more than they have two and that run for
a long time so there is a high chance they end up running in parallel
with something else.
changelog_begin
changelog_end
* Move public code into daml-integration-api
CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
* Add a --timeout option to `daml ledger` commands
At least in the compatibility tests, we see timeouts fairly regularly
in the tests for fetch-dar. If we see those timeouts in the tests, I
see no reason why our users should never see that especially on
actually distributed ledgers and hardcoded timeouts are a bad idea
anyway so now they are configurable.
I’ve also changed the timeout for the fetch tests (those are the only
once I’ve seen timeout) to 120s.
changelog_begin
- [DAML Assistant] The `daml ledger` commands now accept a `--timeout`
option which can be used to change the timeout for API requests from
the default of 30s.
changelog_end
* Generate parties as uuids to avoid collisions
changelog_begin
changelog_end
* Match on Deadline Exceeded as well
changelog_begin
changelog_end
* Update compatibility/bazel_tools/test_utils/DA/Test/Util.hs
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Add some docs
changelog_begin
changelog_end
* Better version checks
changelog_begin
changelog_end
* fix imports
changelog_begin
changelog_end
* This is not how you exception
changelog_begin
changelog_end
* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Add `platform-version` field to `daml.yaml`
This PR adds the `platform-version` field to `daml.yaml`. Based on the
approach agreed upon in #6558, the logic for this all sits in
`daml-helper` and there are no changes to the assistant.
The details of how the logic work are in a comment so I’m not going to
repeat them here but the commands that are affected are:
- `daml sandbox`
- `daml sandbox-classic`
- `daml json-api`
- `daml start` (but only for sandbox and the JSON API, not for
Navigator or anything else)
For tests, I’ve added a test to the compat workspace that installs two
SDKs simultaneously and tries out various combinations and verifies
that we get the correct version. Open to other ideas for testing this
but that seemed like the most sensible option that actually tests what
we run.
changelog_begin
- [DAML Assistant] You can now specify the version of Sandbox and the
JSON API independently of your SDK version by setting
``platform-version`` in your ``daml.yaml``. This is useful if you
are deploying to a ledger that is running components from a
different SDK version. See
https://docs.daml.com/tools/assistant.html#project-config-file-daml-yaml
for details.
changelog_end
* Run platform-version tests
changelog_begin
changelog_end
* Fix tag globbing
changelog_begin
changelog_end
* fmt
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* Try to fix env vars
changelog_begin
changelog_end
* Remove hardcoded references to 1.2.0
changelog_begin
changelog_end
* Rephrase doc comment
changelog_begin
changelog_end
* get things to compile
changelog_begin
changelog_end
* maybe fix things for realz
changelog_begin
changelog_end
* Remove debugging output
changelog_begin
changelog_end
* Get angry at windows
changelog_begin
changelog_end
* why is windows
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* Respect project name in create-daml-app
Now `daml new foobar create-daml-app` creates a project called
`foobar` replacing the name everywhere. The tests now run twice once
with the original project name since that’s what we use in the GSG and
once with a modified one. I guess you could argue that only running
the second should be enough so I’m open to removing the first one.
fixes#6681
changelog_begin
- [DAML Assistant] `daml new foobar create-daml-app` now properly
respects the project name and creates a project called `foobar`
instead of hardcoding the name fo `create-daml-app`.
changelog_end
* .
changelog_begin
changelog_end
* Remove debugging output
changelog_begin
changelog_end
The default cache directories in yarn don’t work in the Bazel
sandbox. This results in Yarn creating tons of temp directories for
each test that are never cleared up.
This PR changes this to create actual temp directories that are
properly cleaned up.
changelog_begin
changelog_end
* Update rules_haskell
- Fixes the issue where a `sh_test` wrapping a `haskell_binary` couldn't
add runfiles as is the case with other Bazel rules.
CHANGELOG_BEGIN
CHANGELOG_END
* Save the runfiles environment at start-up
To work around the fact that `withProgName` overwrites `argv[0]`.
See https://gitlab.haskell.org/ghc/ghc/-/issues/18418.
* damlc_compile_test include damlc runfiles
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Buildifier now comes with a handy attachment to catch single `\`
characters inside strings and replace them with `\\` if the escape
sequence is invalid. Skylark/Python will do this at runtime anyway; this
just makes it clearer what the actual behavior is.
I needed to change `\` characters at the end of lines to `\\` manually
in order to stop Buildifier from simply concatenating the lines
together. Everything else was automatic.
CHANGELOG_BEGIN
CHANGELOG_END
This PR splits up the significantly too large `DA.Daml.Helper.Run`
module into modules grouped by the individual commands:
- `DA.Daml.Helper.Init` for `daml init`
- `DA.Daml.Helper.Ledger` for `daml ledger *`
- `DA.Daml.Helper.New` for `daml new`
- `DA.Daml.Helper.Start` for `daml start` and code for starting the
components that are part of that individually.
- `DA.Daml.Helper.Studio` for `daml studio`
- `DA.Daml.Helper.Util` as a kitchen-sink utilty moduel for things
that don’t really belong in one of the others.
This PR _only_ shuffles code around, there is no change in the
implementation.
changelog_begin
changelog_end
Returning the original relative path from withProjectRoot makes no
sense since we change the directory in there so this PR fixes this to
return the canonicalized project root.
fixes#6245
changelog_begin
- [DAML Compiler] damlc test --project-root now works with relative
paths as well.
changelog_end
This adds an empty template for the daml assistant. This is very helpful
for writing katacoda's, because otherwise you have to delete contained
source files in the `skeleton` project everytime as a first step. The
README in the `daml` directory is mainly there to get the daml directory
in the template.
CHANGELOG_BEGIN
CHANGELOG_END
* Make Bearer prefix optional in Haskell ledger bindings
This matches the change to the JVM bindings in 1.0.0
changelog_begin
- [DAML Assistant] The ``daml ledger`` commands no longer require the
``Bearer `` prefix in the access token file. This matches the behavior
of DAML Script and other SDK tools.
changelog_end
* Fix tasty crash
changelog_begin
changelog_end
* Update compatibility/bazel_tools/daml_ledger/Main.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* new --leak-passwords-firesheep-style option; functions to check forwarded protocol
* enforce https reverse-proxy in all JWT-accepting endpoints
* make HttpService.start take config record
* test that X-Forwarded-Proto or Forwarded is enforced
* use new start signature in daml-script tests
* use insecure http mode for ts codegen tests
* note on regex
* use insecure option in daml assistant integration tests
* log allowNonHttps setting
* add non-https option to more places in daml-assistant tests
* add non-https option to getting started guide
* rename --leak-passwords-firesheep-style to --allow-insecure-tokens
- per suggestion by @garyverhaegen-da, @hurryabit
CHANGELOG_BEGIN
- [JSON API] By default, checks that connections are made through a reverse-proxy
providing HTTPS, ensuring that JWT tokens don't leak. To disable this check,
such as for development, pass ``--allow-insecure-tokens``.
See `issue #5572 <https://github.com/digital-asset/daml/issues/5572>`_.
CHANGELOG_END
* daml start includes --allow-insecure-tokens by default
- as indicated by @cocreature
* Copy daml ledger test to compatibility
Specifically:
- `daml-assistant/daml-helper/test/DA/Daml/Helper/Test/Deployment.hs`
- `libs-haskell/test-utils/DA/Test/Sandbox.hs`
* Adapt daml ledger test for compatibility
Avoid dependencies on the daml repository and make the SDK and platform
components configurable.
* Add test certificates to compatibility
Taken from `@daml//ledger/test-common/test-certificates`.
* Define daml_ledger_test
CHANGELOG_BEGIN
CHANGELOG_END
* Add note on keeping daml ledger tests in sync
* ./fmt.sh
* Strictly require --sdk-version flag
Tasty doesn't allow options without a default value, so we default to
`error` to enforce a value being set.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Previously that resulted in us telling sandbox,navigator,… to connect
to port 0 which obviously does not work. I’ve changed the types a bit
to avoid accidentally passing on the cli argument to them.
changelog_begin
changelog_end
* Make DAML Triggers and DAML Script default to wall-clock-time
Now that sandbox defaults to wall-clock-time there is no reason why we
should not default in DAML triggers and DAML Script.
changelog_begin
- [DAML Triggers] ``daml trigger`` now defaults to wall clock time if
neither ``--wall-clock-time`` or ``--static-time`` is passed.
- [DAML Script] ``daml script`` now defaults to wall clock time if
neither ``--wall-clock-time`` or ``--static-time`` is passed.
changelog_end
* Make --static-time and --wall-clock-time exclusive
This PR gets yarn test running the Puppeteer end-to-end tests in the GSG integration test.
The first step of the test is adding the extra dependencies (Jest, Puppeteer and more). The GSG recommends a yarn add command, but this does not work against HEAD. This is because yarn add does not use resolutions in the parent package.json, and then complains about unknown versions 0.0.0 of the daml TS libaries. The solution here is to hack in the extra dependencies into the ui/package.json and then yarn install. This works, but it hard codes version numbers which we would need to maintain. I would like to be able to say version "latest", which is what yarn add would install.
The next step of the test is to copy the index.test.ts file and run yarn test in CI mode.
I've moved the GSG test to a new create-daml-app-tests target. It's marked "exclusive" in Bazel until we figure out how to avoid hardcoding port numbers. This is a bit tricky since the HTTP port is hardcoded in a couple of places in ui/package.json.
Finally, this PR gets the GSG testing docs to reference the code in the new templates/create-daml-app-test-resources folder.
changelog_begin
changelog_end
This PR adds an --output-file option to DAML Script that writes the
result of a DAML Script to a file and complements the --input-file option.
changelog_begin
- [DAML Script] ``daml script`` now has a `--output-file`` option that
can be used to specify a file the result of the script should be
written to. Similar to ``--input-file`` the result will be output in
the DAML-LF JSON encoding.
changelog_end
* Diff with messaging feature and some noise manually removed
* Bazel target to use patch file in other build targets
* Patch file as data dep for integration tests
* Attempt to patch and test messaging feature in create-daml-app test
changelog_begin
changelog_end
* Use exports_files instead of filegroup
* Remove file existence checks that don't make sense
* Add patch to dev_env and reference it from integration tests
* Include patch on windows for later
* Set up yarn env again after codegen
* Restore file check
* Fix typo in comment on util function
* Add Tasty steps to make process explicit
* Use messaging patch for code snippets in GSG
* Use messaging code from template instead of copy
* Remove copied message code
* Refactor script to copy template code with messaging patch
* No need to retry second yarn install (only local deps should be updated)
* Used `daml codegen java` instead of calling the codegen from maven
This should hopefully fix the issues with mismatched versions of
slf4j.
changelog_begin
changelog_end
* Move config to daml.yaml
* Remove alternative invocation via maven from docs
* Haskell: Add assertFileExists to DA.Test.Util
This PR adds a helper function `assertFileExists` that captures the
`doesFileExist ... >>= assertBool ...` pattern that is very common in
our Haskell test suites. It also adds the inverse
`assertFileDoesNotExist` function. Both functions are now used where
appropriate.
CHANGELOG_BEGIN
CHANGELOG_END
* Add directory dependency to test-util lib
CHANGELOG_BEGIN
CHANGELOG_END
Some of the dependencies of the `create-daml-app` seem to fail to
install without any good reason every now and then. We try to
circumvent this flakiness by running the `yarn install` step for the
UI three times before giving up.
CHANGELOG_BEGIN
CHANGELOG_END
There are multiple tests that setup an environment for yarn to find
our TypeScript libraries. This PR unifies this setup by providing a
single function to set it up.
CHANGELOG_BEGIN
CHANGELOG_END
Packages com.digitalasset.daml and com.daml have been unified under com.daml
Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.
CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
I've completely removed the possibility to call `daml codegen ts`. I'm
happy to add in back in a followup PR once I've seen that all our tests
pass without it existing.
CHANGELOG_BEGIN
CHANGELOG_END
* Add snapshot version support in daml-assistant.
changelog_begin
- [DAML SDK] ``daml version`` can now list the available snapshot
versions by passing the flag ``--snapshots=yes``.
- [DAML SDK] ``daml install latest`` can now include the latest snapshot version by passing the flag ``--snapshots=yes``.
changelog_end
* Keep backward-compatible --all behavior
* Fix comment
* Delete spurious punctuation
Previously we emitted 3 lines starting with a capslock `WARNING` which
seems to scare some people. Now we emit two lines without the
`WARNING` and point to the release notes since that seems better than
explaining to users how they upgrade without telling them what
changed.
It’s still two lines, I couldn’t come up with anything shorter that
seems reasonable. We might want to reduce the frequency at which this
is displayed as well but for now this should do.
changelog_begin
changelog_end
Change the output directory given to `daml codegen ts` from `daml-ts`
to `daml.js`. This is naming is in line with the fact that `daml2ts`
puts all generated packages into the `@daml.js` scope. A renaming of
`daml2ts` into `daml2js` is immiment.
This PR also changes a few small documentation issues the
search-and-replace has surfaced.
CHANGELOG_BEGIN
CHANGELOG_END
The test calls `daml new ... create-daml-app`, builds the DAR, runs
the TypeScript codegen, lints and builds the React app.
We should also run the tests for the React app but since that's most
likely tricky, I'll leave that for a followup PR so that we can land
at least some sanity checks now.
There's also a bit of cleanup to do around setting up a yarn workspace
to bring our TypeScript libraries into scope. Other test already solve
a similar problem in a way that is not general enough for this test.
I'll unify the setup in a followup PR as well.
CHANGELOG_BEGIN
CHANGELOG_END
- Move deployment tests (deployTest, fetchTest) out of integration-tests.
- Use DA.Test.Sandbox where appropriate.
- Split out code for useful test patterns: i.e. calling commands quietly, getFreePort.
changelog_begin
changelog_end
* Initial commit with create-daml-app master
* Include create-daml-app in build rule
* Make daml.yaml a template in version and project name
* Remove git attributes
* Remove license and azure config
* Remove scripts
* Don't overwrite config files in build rule
* Template version numbers in package.json, to be replaced by the assistant
* Rename to package.json.template
changelog_begin
changelog_end
* Add copyright headers
* Do template substitutions in all .template files
And don't special case daml new create-daml-app (so it treats it as a
regular template).
* Add create-daml-app to integration tests
* Remove WIP warning
* Move towards setup that works on head
* Make local copies of the TS libs in the templates tarball
* Hardcode project name for now
* Use isExtensionOf
* Remove service worker
* remove robots.txt (don't even know what it is)
* Revert "Make local copies of the TS libs in the templates tarball"
This reverts commit 1289581fb4a82af3ab534baf978a2c6ed895d538.
* Retemplatize TS lib versions. For head builds these will be installed using npm
* Remove daml/ledger from resolutions for daml-ts
* Comment about test secret
* Remove special create-daml-app assistant command and test that won't work anymore
* Remove redundant imports and export
* Remove old create-daml-app tests
* Remove yarn.lock
* Clean up integration test (just daml new and build atm)
* Add daml/ledger as a resolution for daml-ts
* Remove top level package.json
* Update daml.js version
* Use new import scheme for generated TS
* Update readme with new codegen and build steps
* Use start-navigator in daml.yaml
* Increase a couple of timeouts in tests (either sandbox or TS lib is a bit slower?)
* Update GSG intro with new build steps
* Remove daml2ts -p flag and --start-navigator flag from GSG instructions
* Don't use start-navigator flag in ui tests
* Temporary readme describing how to manually test the create-daml-app template
* Update code samples in app arch section of GSG
* Update code samples in testing doc
* Remove copied create-daml-app code
* Indent docs markers to be more subtle
* Update visible code in Messages (after) section
This needs to be kept up to date properly somehow.
* Update text to useLedger
* Restore code/ui-before copies until the Bazel magic is figured out
We need to make the template code a dependency in the Bazel rule as
otherwise we can't find the files in the docs build.
* Update create-daml-app/readme and make templates/readme more detailed
* Use jsx comments for docs markers so they don't show up in the app
This cause issues in combination with data-dependencies see #5330 and
we started warning if you don’t expose everything. This resulted in
quickstart-java producing a warning on every `daml build`.
changelog_begin
changelog_end
* Improve handling of exposed-modules with data-dependencies
Previously, we tried to rename all modules of a dependency via
--package. This fails if some of those modules are not exported. This
was trivial to hit as a user since the ``daml-trigger`` library made
use of this.
This PR adds a few things to improve the situation:
1. We only rename modules that are exposed. This fixes the issue if
you don’t actually reference a non-exposed module from your
data-dependency.
2. I’ve removed the exposed-modules from daml-trigger. I don’t think
they are essential here given that the module name has `Internal`
in the name and it’s too easy to have something that actually
references the non-exposed module since the types are reexported.
3. I’ve added documentation that mentions this issue.
4. I’ve added a warning if your exposed-modules are excluding some
modules. Maybe worth turning this into an error in the future.
changelog_begin
changelog_end
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
* Use com.daml as groupId for all artifacts
CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END
* Add 2 additional maven related checks to the release binary
1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId
* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
* Make new sandbox the default.
changelog_begin
- [DAML SDK] The new sandbox is now the default that runs with ``daml sandbox`` and ``daml start``. The command ``daml sandbox-next`` has been removed. The old sandbox can be invoked via ``daml sandbox-classic`` and ``daml start --sandbox-classic=yes``.
changelog_end
* Update descriptions.
* Change it to a switch
* Change switch help
* Recapitalize
* Move 'withEnv', call it from daml2ts tests
changelog_begin
changelog_end
* Fix withEnv call to ensure that TASTY_NUM_THREADs is set
withEnv replaces the whole environment so we need to set everything we
care about.
* withEnv replaces the whole environment so we need to set everything we
care about.
* Apparently applying the same fix has destabilized Windows
* Try even harder to get daml assistant tests passing on Windows again
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
The way we use `withEnv` it is really intended to set a few
environment variables in an already existing environment instead of
clearing everything before. This PR changes it to do only that.
changelog_begin
changelog_end
closes#5280
Those were originally used for `damlc migrate`. However, that command
no longer exists and the only call site of `runNew` sets those
arguments to an empty list.
changelog_begin
changelog_end
* New comamnd: daml ledger fetch-dar
`daml ledger fetch-dar [PID] [PATH]`
Download a `Package` and it's dependencies from a ledger, given a root `packageId`, and re-construct a valid `.dar` file. Addresses issue #5037.
The original package names are not reconstructed.
CHANGELOG_BEGIN
CHANGELOG_END
* address some comments
* fix spello
* attempt: recoverPackageName
* recover main package name & version
* Try to fix integration tests on Windows
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
There was a brief period of time where `daml build` did not work
outside of the assistant. When we fixed that we added it to the
integration tests since there was no other test suite that used damlc
as a binary (as opposed to using it as a library which runs through
different codepaths). However, in the meantime we have tons of tests
all over the place (e.g. the packaging tests) that call `damlc build`
outside of the assistant so these tests serve no purpose.
Also they are somewhat confusing since the point of the integration
tests is to test an installed SDK whereas these tests do not need an
installed SDK (that’s the whole point of those tests).
changelog_begin
changelog_end
This ensures that -l and -p work properly in the integration tests
since they no longer depend on the order.
There is lots of other crap to cleanup in those tests but I’m trying
to keep it to small steps.
changelog_begin
changelog_end
* Depend on LF version specific daml-libs
* daml-script.dar build multiple LF versions
CHANGELOG_BEGIN
[DAML Script] The `daml-script` library is now available in multiple LF
versions, namely 1.7, 1.8, and 1.dev.
CHANGELOG_END
* daml-trigger.dar build multiple LF versions
[DAML Triggers] The `daml-trigger` library is now available in multiple
LF versions, namely 1.7, 1.8, and 1.dev.
* Keep daml-script.dar available for tests
* Keep daml-trigger.dar available for tests
* daml-libs LF versions integration test
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Integrate create-daml-app into the assistant
fixes#4868
changelog_begin
- [DAML Assistant] Add a new ``daml create-daml-app`` command for creating a project based on
`create-daml-app <https://github.com/digital-asset/create-daml-app>`_.
changelog_end
* Try random things hoping to fix windows
* Try random things hoping to fix things on macos
* Allow controlling navigator startup in daml.yaml
While I’m not entirely convinced the default atm is right, making it
configurable seems like an easier solution than bikeshedding the default.
changelog_begin
- [DAML Assistant] You can now disable starting navigator as part of
``daml start`` in your ``daml.yaml`` file by adding ``start-navigator: false``.
changelog_end
* Update docs/source/tools/assistant.rst
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
* Use --start-navigator=true in the docs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Bump openssl
The previous one has stopped working for some reason :sadpanda:
changelog_begin
changelog_end
* daml-assistant: Add `--wall-clock-time` to the Sandbox Next test.
Missed this due to doing two things at once.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* daml-assistant: Add `daml sandbox-next`.
CHANGELOG_BEGIN
- [DAML Assistant] You can now run a pre-release version of Sandbox with
``daml sandbox-next`` so you can test it out and verify everything is
working as expected. Running this will launch Sandbox rebuilt on a
more modern architecture. An upcoming release of DAML will switch over
to the new implementation by default.
CHANGELOG_END
* daml-assistant: Explain that sandbox-next is experimental.
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* daml-assistant: Copy-pasta an integration test for `daml sandbox-next`.
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* sandbox: Fail to start if a time mode is not explicitly specified.
CHANGELOG_BEGIN
- [Sandbox] Sandbox is switching from Static Time mode to Wall Clock
Time mode as the default. To ensure that our users know about this,
for one version, there will be no default time mode. Instead, users
will have to explicitly select their preferred time mode by means of
the `--static-time` or `--wall-clock-time` switches. In the next
release, Wall Clock Time will become the default, and users who are
happy with the defaults will no longer need to specify the time mode.
CHANGELOG_END
* daml-script|triggers: Specify time mode when testing against Sandbox.
* daml-assistant: Default the Sandbox to wall clock time.
CHANGELOG_BEGIN
- [DAML Assistant] Initializing a new DAML project adds a switch to
``daml.yaml`` to ensure Sandbox can continue to start with ``daml
start``::
sandbox-options:
- --wall-clock-time
CHANGELOG_END
* docs: Update the DAML Script and Triggers docs to use Wall Clock time.
It's now what Sandbox will use by default when using `daml init`.
* docs: Change the Quickstart to run Sandbox in wall clock time.
This explains why the contract IDs may vary.
It also updates the manual release testing script to match.
We previously had 3 slightly different but consistently shitty logic
for handling this in the tests for daml-helper daml repl and the
Haskell ledger bindings. This PR introduces a module that is flexible
enough to capture all their needs and hopefully is somewhat less
shitty.
changelog_begin
changelog_end
* Share test certificates
This is primarily an attempt at making sure my contribution stats
remain negative but I think it’s a nice cleanup. The only difference
in the certs used by daml-helper which are now used everywhere is that
they use a different CN for the CA and the server. This is required to
make openssl happy (which is used by the daml-helper).
changelog_begin
changelog_end
* Fix script and trigger tests
This adds CLI parametrs for connecting via TLS following the scheme
used by navigator, extractor and `daml ledger`.
changelog_begin
- [DAML Script] Support TLS. Enable it by passing ``--tls``. You can
set certificates for client authentication via ``--pem`` and
``-crt`` and a custom root CA for validating the server certificate
via ``--cacrt``.
- [DAML Triggers - Experimental] Support TLS. Enable it by passing ``--tls``. You can
set certificates for client authentication via ``--pem`` and
``-crt`` and a custom root CA for validating the server certificate
via ``--cacrt``.
changelog_end
Currently sandbox only supports TLS if you also enable client
authentication. There is no reason for why this has to be the case and
for things like DABL we want TLS without client authentication so it’s
useful to be able to test this in sandbox. This PR introduces a
`--client-auth` flag that allows you to configure the behavior. The
default is the current one of requiring client authentication.
This PR does not yet update Java clients, however, the Haskell client
supports this already and is used to test this functionality.
I’ve also added a section in the documentation on TLS (there were no
docs at all so far).
changelog_begin
- [DAML Sandbox] When Sandbox is run with TLS enabled, you can now
configure the requirement for client authentication via
``--client-auth``. See
https://docs.daml.com/tools/sandbox.html#running-with-tls for more information.
changelog_end
changelog_begin
changelog_end
For now this is only used for the daml-helper tests. I’ll shuffle
things around and use it for all tests in a separate PR.
This PR adds TLS support to DAML helper both via client certs and
without (although the latter is not tested so far since atm this is
not supported by sandbox). The CLI options follow the scheme used by
navigator/extractor/… with the addition that you can just pass `--tls`
which will turn on TLS without custom root certs or client certs.
changelog_begin
- [DAML Assistant] You can now connect to ledger via TLS for ``daml
deploy`` and ``daml ledger`` commands. See
https://docs.daml.com/deploy/generic_ledger.html for more information.
changelog_end
* daml-assistant tests: fix flaky test on Windows
```
bazel test //daml-assistant:test --test_arg=--quickcheck-replay=425714
```
failed on Windows in the test-case:
```
tail . ascendents == ascendents . takeDirectory
```
In that case the two path components given two the test case shrink to
`p1 = "a"` and `p2 = "\\"`. Confusingly, on Windows
```
isRelative "\\" == True
```
while
```
"a" </> "\\" = "\\"
```
This is documented behaviour on Windows, see [1] and [2].
Using `p1 </> p2 /= p2` instead of `isRelative p2` works around this.
[1]: https://hackage.haskell.org/package/filepath-1.4.2.1/docs/System-FilePath-Posix.html#v:-60--47--62-
[2]: https://hackage.haskell.org/package/filepath-1.4.2.1/docs/System-FilePath-Posix.html#v:isRelative
* Fix ascendants test group name
`ascendants` is defined in `DA.Daml.Project.Util`, not
`DA.Daml.Assistant`.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
On MacOS using Java 13 (at least that was the only combination where
I’ve managed to reproduce this). Sandbox spits out errors of the
following form in ``daml start``
```
java.net.SocketException: Connection reset
```
This appears to be caused by the TCP requests that we send to detect
whether sandbox has started. Switching to using the ``--port-file``
option makes this issue go away.
fixes#4670
changelog_begin
- [DAML Assistant] In certain configurations ``daml start`` produced
a (harmless) ``Connection reset`` log message. This has now been
fixed.
changelog_end
changelog_begin
- [DAML Assistant] You can now specify options for
Sandbox/Navigator/the HTTP JSON API in ``daml.yaml`` via
``sandbox-options``/``navigator-options``/``json-api-options``. These
options will be picked up by running ``daml start``. This is
particularly useful for specifying ``--wall-clock-time`` and for
specifying a fixed ledger ID during development.
changelog_end
fixes#2993
This introduces a `HasSubmit` typeclass (following the naming scheme
of `HasCreate`, …) and instances for `Scenario` and `Script`. This
avoids the need to hide `submit` in every single DAML script.
changelog_begin
- [DAML Standard Library] ``submit`` and ``submitMustFail`` are now
overloaded so that they can be used in both scenarios and DAML script.
changelog_end
* Remove damlc migrate
``damlc migrate`` hasn’t worked for quite a while and we emitted a
warning for months so given that we don’t have plans to make it work
again in the near future, I think it does more harm than good to keep
it around.
changelog_begin
- [DAML Compiler] After being deprecated for a while the ``damlc
migrate`` command has now been removed. See
https://docs.daml.com/upgrade/ for up to date documentation
on model upgrades.
changelog_end
fixes#3704 (by removing the tests 😇)
* yeah the windows cache is once again broken \o/
* Revert "yeah the windows cache is once again broken \o/"
This reverts commit 38d7877aa4.
* Add init-script daml.yaml field
CHANGELOG_BEGIN
[DAML Script -- Experimental] You can now define an ``init-script`` in
the ``daml.yaml`` file. If present, that DAML script will be executed
to initialize the sandbox on ``daml start``. This can be used instead
of the ``scenario`` field.
CHANGELOG_END
* Add integration test for init-script
* Generate JWT token in tests
Addressing review comment
https://github.com/digital-asset/daml/pull/4685#discussion_r383835050
* Remove unnecessary daml calls
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
* Report assistant commands and errors via a logger.
CHANGELOG_BEGIN
CHANGELOG_END
* Hook up daml-assistant to a GCP logger.
* fix test case
* fix more tests
* Check opted in status in assistant.
* Anonimize args that have unexpected characters.
* More agressive anonimization
* add missing containers dependency
We were still using the old token format without the
https://daml.com/ledger-api key. This resulted in a warning on every
call to `daml start` which looks rather ugly. I’ve changed it to
construct the token using the JWT library instead of hardcoding it
which should make it easier to modify in the future
changelog_begin
changelog_end
