This PR accepts the --tls flag but rejects the flags to set certs
because those don’t really make sense here (or are at least not easily
settable).
Unfortunately no great way to test this without a huge amount of test
infrastructure to setup a custom TLS reverse proxy for tests.
If someone has an idea to add a reasonable test, I’m all ears.
changelog_begin
- [Daml Assistant] The `daml ledger` commands now accepts `--tls` in
combination with `--json-api` to access a JSON API behind a TLS
reverse proxy.
changelog_end
CHANGELOG_BEGIN
ledger-api - Command deduplication period can now be specified by setting `deduplication_offset` instead of `deduplication_time` (only valid for v2 WriteService). This change is backwards compatible.
CHANGELOG_END
* Propagate the enriched deduplicationPeriod instead of deduplication duration
* Update the Haskell bindings for the new deduplication period
* Calculate the deduplicateUntil using the new deduplication period for backward compat
* Use consistent naming for deduplication_period
* Cleanup command timeout extraction from deduplication period
* Add the required deduplication_offset to deduplication instead of deduplication_start
* Update haskell bindings to support deduplication_offset
* Add support for deduplication_offset in the ledger-api
* Remove the timestamp-based deduplication from our models to simplify upgrade for users
* Add optional conformance test for offset based deduplication
* Remove buf rule for FIELD_SAME_ONEOF as our change is backwards compatible
* Disable FIELD_SAME_ONEOF buf check for commands file
* Apply suggestions from code review
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Update comment for deduplication period
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* rewrite trigger docs to follow gsg
Per #10419 point 4, I've rewritten the Triggers section to build upon
the Getting Started Guide instead of inventing its own example.
Compared to #10395, this has a lot more explanations as this page must
now serve the dual purpose of being a possible "next step" from the GSG
and being the main reference page for triggers. It's also lost the "next
steps" section, which I think is a bit of a shame, but it doesn't really
make sense here.
There's also no easy way for people not interested in the GSG to follow
along; should we expose the "completed GSG" as a tempate?
CHANGELOG_BEGIN
CHANGELOG_END
* keep copy-trigger as a template
* fix copy-trigger project name
* make up gsg-trigger template
* remove awkward sentence, fix existing typo
* update code to use when{,Some}
* add to
* swap emitCommands and getCommandsInFlight
* typo
* insist on state-correction perspective
* fix copy-trigger tests
* add back copy-trigger to whitelist
* add gsg-trigger to whitelist
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
No reason to force the existence of a config file if you can start
without one.
fixes#10516
changelog_begin
- [Navigator] Navigator will now start with an empty config if no
config file exists and it is run outside of a project.
changelog_end
* Add optional submission id to commands.proto
This allows to propagate a submission id. If no id is submitted (the submission id is empty) then we generate a new submission id
CHANGELOG_BEGIN
Add optional submission_id to the commands.proto.
CHANGELOG_END
* Update haskell bindings to include the submission id
* Code review - rename submission id extractor
* Code review - update comment and remove braces from if block
* Fix braces
changelog_begin
changelog_end
- DAML -> daml
- Assistant -> assistant
The capitalization of the latter fixed one occurrence of a capitalized 'A'
which was inconsistent with the vast majority of uncapitalized 'a's.
* Parallelize daml-assistant integration tests.
This mainly involves avoiding changing the working directory or the environment while inside tests.
AFAIK the daml start tests can't be parallelized without starting up a sandbox instance each time, which seems bad. These tests could be separated from the rest...
The magic number "2" was picked via experimentation:
```
1 threads -> 306 s
2 threads -> 199 s
3 threads -> 198 s
4 threads -> 195 s
```
changelog_begin
changelog_end
* buildifier-fix
Each of these installations is about 1gb so it’s not that unlikely
that this plays a significant factor in our out of disk space errors.
Although I think we cleanup partially so I’m not sure how large the
part of the SDK installation is that gets leftover.
changelog_begin
changelog_end
I started looking into failures in the integration tests and this
seems by far the most prevalent one (maybe even the only one but that
needs more investigation). The issue here is that sometimes waiting
40s isn’t enough. On the other hand, sometimes we wait unnecessarily
long which is also bad.
The fix in this PR is to wait until we see the "reload complete"
message which should be exactly the right amount of time to wait.
changelog_begin
changelog_end
Currently we have the following dependency chain:
1. The compiler depends on the scenario service for daml test
2. The scenario service depends on Daml Script
3. Daml Script depends on the Sandbox code only for daml test-script
The last one can easily be split. The scenario service does not care
about this code.
This means that now if we change ledger code, at least not all
compiler tests are going to rerun.
Verified that this actually breaks the dependency fully via
```
bazel query 'somepath(//compiler/damlc/tests:packaging, //ledger/participant-integration-api/...)'
```
changelog_begin
changelog_end
* Block concurrent SDK installations.
Fixes#9612
changelog_begin
- [Daml Assistant] The assistant will now avoid
installing SDK versions concurrently, waiting
for the previous installation to finish before
starting the next installation (if still
necessary). This fixes a bug where the SDK
would become corrupted because two
installations were started concurrently.
changelog_end
* Remove filelock from stack-snapshot.yaml
* Update windows snapshot pin
* Add message if file lock is not immediately available.
* Set supported jdbc driver names at compile time
This is mainly to unblock the work on Oracle support in the Ledger API
but I think it’s a sensible thing in general. For the Ledger API,
moving the dependency to the top-level is apparently rather
tricky. Because the SDK bundles everything into a single megajar,
Sandbox depending on the oracle library does also result in the JSON
API and the trigger service will also have the oracle library in scope
and will support Oracle in CE which they should not.
This PR simply hardcodes the list of supported drivers to address
that. Not pretty but does the job.
changelog_begin
changelog_end
* format
* Address review comments
changelog_begin
changelog_end
* Don't duplicate export script option parsing
Forward all flags to the wrapped executable instead of handling them in
daml-helper with optparse-applicative.
changelog_begin
changelog_end
* Fix hlint warning
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* rename "dump" to "export"
* Add Daml export command to assistant
* Make daml export script a subcommand of daml export
* daml-assistant IT for daml export script
* Expose export as daml ledger command (hidden for now)
Add Haskell side parser for ledger export flags
changelog_begin
* [Daml export] New feature: Use ``daml ledger export script`` to
generate a Daml script that will reconstruct a given ledger state.
This is an early access feature.
changelog_end
* Update integration test
* Remove daml export - it's daml ledger export now
* integration-test set ledger port
The integration tests don't configure the port in the project config,
but on the command-line, and they don't use the default value.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* daml build: add a --access-token-file for remote dependencies
This adds a `ledger.access-token-file` field in the `daml.yaml` project
file and a `--access-token-file` flag to `daml build` to authorize
querying/fetching of remote dependencies.
CHANGELOG_BEGIN
[daml build] A new flag `--access-token-file` is added for the `daml
build` command. It allows the specify the path to an access token to
authenticate against the ledger API. This is needed if the project
depends on a remote Daml package hosted on such a ledger. Alternatively,
the path to the token can also be specified in the `daml.yaml` project
file under the `ledger.access-token-file` field.
CHANGELOG_END
* Expose tho Oracle support in the EE trigger service
This PR builds on the previous PR that did all the actual work on
Oracle support and exposes it in the enterprise edition. This PR only
releases the enterprise edition via the SDK tarball. I’ll add
artifactory publishing separately.
changelog_begin
changelog_end
* Update daml-assistant/daml-sdk/validate.sh
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Fixes#9196.
This adds a new command `daml package list` which fetches all
available packages from the ledger and prints package name/versions to
stdout.
CHANGELOG_BEGIN
[daml packages] A new `daml packages list` command has been added to
list packages deployed on a remote Daml ledger.
CHANGELOG_END
Fixes#9187.
Instead of `daml: Must be called from within a project` you will get now
`daml: This command needs to be either run from within a project or the argument --host needs to be specified.`
CHANGELOG_BEGIN
CHANGELOG_END
* remote pkgs: resolve package names in data dependencies via ledger
This implements part 3) of #8976.
This adds the ability to specify package names/versions in daml.yaml in
the data-dependencies stanza. They are being resolved via the project
ledger and a daml.lock lock file.
CHANGELOG_BEGIN
[remote dependencies] Package names and versions, as well as package
ID's are allowed in the `data-dependencies` list of `daml.yaml`. These
packages are fetched from the project ledger. The auto-generated
`daml.lock` file keeps track of the package name/version to package
ID's resolution and should be checked in to version control of the
project.
CHANGELOG_END
* added docs
* Update compiler/damlc/lib/DA/Cli/Damlc/DependencyDb.hs
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Update docs/source/daml/reference/packages.rst
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* suggestions
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
--sandbox-port has caused confusion several times and cannot be
specified in daml.yaml which is a bit annoying. There isn’t really any
good reason not to support --sandbox-option --port since we can pick
up the port via the port file either way.
This change is fully backwards compatible.
changelog_begin
- [Daml Assistant] The sandbox port can now also be configured via
`--sandbox-option=--port=12345` instead of `--sandbox-port`. Other
tools like Navigator, the JSON API and Daml Script will pick up the
modified port automatically.
changelog_end
* damlc pkg: use cache for already present packages
This only downloads packages from a remote ledger, if the package is not
already present in the dependency db as a normal dependency.
CHANGELOG_BEGIN
CHANGELOG_END
* damlc: Allow package IDs in data-dependencies.
This is the next step outlined in issue #8976. If package id's are
present in the `data-dependency` section of the daml.yaml file, we try
to fetch them (and their transitive dependencies) from the default
ledger of the project.
CHANGELOG_BEGIN
CHANGELOG_END
* Support fetching SDK EE tarball in the assistant
This PR adds an `artifactory-api-key` field to `daml-config.yaml`. If
this is set, we will download the SDK EE tarball if it exists, falling
back to GH releases on 404s (for old releases).
changelog_begin
- [Daml Assistant] The assistant now supports an `artifactory-api-key`
field in `daml-config.yaml`. If you have a license for Daml Connect
EE you can specify this and the assistant will automatically fetch
the EE edition which provides additional functionality.
changelog_end
* Hardcode first EE snapshot
changelog_begin
changelog_end
* Release EE SDK tarballs and installer
As before, no way of testing this. I’ll do a snapshot afterwards.
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* Rename EE artifacts
changelog_begin
changelog_end
* Move Daml Profiler to EE version of sandbox/sandbox-classic
This splits Sandbox targets into EE/CE targets and exposes the option
in the EE version. The option still exists in the CE option for now
until we have released EE artifacts to not break users that might know
about it without an alternative.
There is also a small test that makes sure that this actually works
since classpaths are dumb and it didn’t work at first.
changelog_begin
changelog_end
* Fix publish target
changelog_begin
changelog_end
* Publish transitive dep
changelog_begin
changelog_end
* I hate bash
changelog_begin
changelog_end
* Build SDK EE tarball
This sets up the infrastructure to build an SDK EE tarball and allows
for swapping out all files included in the tarball depending on the
edition. As an example, this includes the JSON API with (partial)
Oracle support in the EE tarball.
This PR does not yet address publishing this artifact to Artifactory.
I’ll tackle that in a separate PR.
changelog_begin
changelog_end
* Build in temp dir because Windows is stupid
changelog_begin
changelog_end
* directories are bad
changelog_begin
changelog_end
* Navigator resources are actually needed
changelog_begin
changelog_end
We change the color from red to yellow to inform about the possibility
to rebuild and also output the success message in green.
CHANGELOG_BEGIN
CHANGELOG_END
* Draw the rest of the Scala 2.13 owl
Not quite but pretty close and this switches us over from inclusions
to exclusions which makes it much easier to track.
Ledger API test tool should be fixed by #8821. Non-repudiation needs a
tiny bit of work since unwrapArray doesn’t work the same on 2.13 but
shouldn’t be hard to fix.
changelog_begin
changelog_end
* Fix ScriptService tests
Those tests were all dumb. They asserted on a fixed order while the
function to sort the things was broken so we ended up with the random
Map order which is unsurprisingly not the same.
This is easily fixed by fixing the sort function.
There is also a second issue with query not sorting.
changelog_begin
changelog_end
* Turns out if you fix one test the next one breaks
And clearly nobody ever tested this or give this a second thought.
changelog_begin
changelog_end
* Use cliopts.Http for OAuth 2.0 middleware
changelog_begin
- [OAuth 2.0 Middleware] You can now configure the address that the
middleware listens to using the ``--address`` flag.
The port that the middleware listens to is now configured using the
``--http-port`` flag, use 0 to dynamically choose a free port.
You can now configure a port file where the chosen port will be
written to using the ``--port-file`` flag.
changelog_end
* Add test-case for OAuth 2.0 middleware port file
* Don't forget to close source
* Fix integration test
* Update triggers/service/auth/src/main/scala/com/daml/auth/middleware/oauth2/Config.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* add livez endpoint to auth middleware
* Add OAuth 2.0 middleware to Daml SDK
* unhide trigger service auth flags
changelog_begin
- [Triggers] The trigger service now supports authorization through an
auth middleware. The feature is enabled using the `--auth` and
`--auth-callback` command-line flags. Please refer to the
Authorization chapter of the trigger service documentation for further
instructions.
- [OAuth 2.0 middleware] Daml Connect now includes an implementation of
the auth middleware API that supports OAuth 2.0 Authorization Code
Grant. Please refer to the Auth Middleware and OAuth 2.0 Auth
Middleware chapters of the documentation.
changelog_end
* drop early access flag on triggers
Daml triggers, the trigger service, and the auth middleware are no
longer marked as early access features.
changelog_begin
- [Triggers] Daml Triggers and the Trigger Service are no longer in
early access status.
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
To avoid prompting the user to set the PATH variable on CI, we check the
`CI` environment variable and do nothing if set.
CHANGELOG_BEGIN
CHANGELOG_END
* fix: project parties are not required
This fixes a bug in daml assistant. `daml deploy` would fail if there
were no project parties listed in the daml.yaml project config file. Now
project parties are not required anymore.
Fixes#8487.
CHANGELOG_BEGIN
CHANGELOG_END
* added an integration test
We use the new cache folder specified with `DAML_CACHE` defaulting to
`.cache/daml` as a new cache folder for the assistant to store available
versions etc.
This does not migrate an existing cache folder and just starts a new
one.
CHANGELOG_BEGIN
CHANGELOG_END
* telemetry: use an extra cache directory for telemetry
If the sdk's `.daml` directory is not writable, telemetry fails and
kills the IDE. We add a new assistant environment variable "DAML_CACHE".
If set, this directory is used for telemetry cache data.
Fixes#8396.
CHANGELOG_BEGIN
CHANGELOG_END
* try creating directory
* check for already existing machine id file
* set DAML_CACHE to read-only in assistant integration tests
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
The one thing that is still missing is making the generated Scala code
from the codegen compatible with Scala 2.13 so the examples are
excluded for now.
changelog_begin
changelog_end
* fix: navigator regression: create empty config file.
For backwards compatibility, `daml ledger navigator` needs to create an
empty `ui-backend.conf` file when invoced outside a project directory.
Otherwise, navigator will fail to start.
CHANGELOG_BEGIN
CHANGELOG_END
* integration test
* Replace many occurrences of DAML with Daml
* Update docs logo
* A few more CLI occurrences
CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END
* Fix some over-eager replacements
* A few mor occurrences in md files
* Address comments in *.proto files
* Change case in comments and strings in .ts files
* Revert changes to frozen proto files
* Also revert LF 1.11
* Update get-daml.sh
* Update windows installer
* Include .py files
* Include comments in .daml files
* More instances in the assistant CLI
* some more help texts
* Support DAML_PROJECT in daml codegen java
fixes#8406
We need to resolve the DAR path relative to the project
root. Otherwise, we’ll just fail to find the file if daml codegen java
is invoked from a different directory.
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* fmt lalala
changelog_begin
changelog_end
* fix another test
changelog_begin
changelog_end
* assistant: get the latest sdk version form docs.daml.com
We get the latest sdk version from docs.daml.com/latest instead of
github.com. Getting the latest release from github proved to be tricky
because github defines 'latest' by the tag timestamp.
CHANGELOG_BEGIN
CHANGELOG_END
* hlint
* navigator: ignore project parties flag
This adds a flag '--ignoreProjectParties' to navigator such that
parties specified in the daml.yaml file are ignored and the
`PartyManagementService` is queried for parties instead.
This is part of #6099.
CHANGELOG_BEGIN
CHANGELOG_END
* add flag to
* add flag to daml ledger navigator
* Update navigator/backend/src/main/scala/com/digitalasset/navigator/config/Arguments.scala
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* remove config file from daml ledger navigator command
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* hotreload: archive active contracts
Fixes#8162. This archives all active contracts on the ledger when a
hotreload is performed. A hidden command `reset` is added to `daml
ledger` to perform the reset against a running sandbox.
CHANGELOG_BEGIN
CHANGELOG_END
* don't reset when no parties are allocated
* hlint
* reuse tokenFor
* remove unused imports
* add applicationId to token
* remove readAs field from command
* use uuid as cmdId
* remove token from reset command
* batch cmds in size of 100
* remove comment about todo, remove readAs from token
Also disables auto-install on Windows, since installation-related
errors range from "simple error message" to "program hangs forever",
and users can bypass "daml install" by downloading the new installer.
changelog_begin
- [DAML Assistant] The update-check is disabled by default for
newer installations. In addition, auto-install is disabled by
default on Windows. See here for instructions on how to
configure these settings manually: https://docs.daml.com/tools/assistant.html#global-config-file-daml-config-yaml
changelog_end
