* Migrate skeleton to DAML Script
Also kills the separate setup module since it’s the same thing
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* Remove authentication service from trigger service
This rips out the integration with ref-ledger-authenticator and
together with that the weird http basic auth which doesn’t actually
work for ref-ledger-authenticator outside of its testing mode and also
includes some very sketchy stuff like storing user
passwords (encrypted but still).
While the exact details of the new approach towards auth are still not
completely clear, it is clear that it is going to be sufficiently
different from what we did before that starting from a clean slate is
easier.
changelog_begin
- [Trigger Service] Parties are now specified in request bodies as
opposed to via HTTP Basic auth. This is done in preparation for
running the trigger service against authenticated ledgers.
changelog_end
* Fix integration tests
changelog_begin
changelog_end
Part of #6977. This PR adds some assistant tests, but regression tests and documentation should be added before the issue is closed.
changelog_begin
changelog_end
* add -Ywarn-unused to all scalac options
* remove some unused arguments
* remove some unused definitions
* remove some unused variable names
* suppress some unused variable names
* changeExtension doesn't use baseName
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* work around no plugins in scenario interpreter perf tests
* remove many more unused things
* remove more unused things, restore some used things
* remove more unused things, restore a couple signature mistakes
* missed import
* unused argument
* remove more unused loggingContexts
* some unused code in triggers
* some unused code in sandbox and kvutils
* some unused code in repl-service and daml-script
* some unused code in bindings-rxjava tests
* some unused code in triggers runner
* more comments on silent usages
- suggested by @cocreature; thanks
* fix missing reference in TestCommands
* more unused in triggers
* more unused in sandbox
* more unused in daml-script
* more unused in ledger-client tests
* more unused in triggers
* more unused in kvutils
* more unused in daml-script
* more unused in sandbox
* remove unused in ledger-api-test-tool
* suppress final special case for codegen unused warnings
.../com/daml/sample/mymain/ContractIdNT.scala:24: warning: parameter value ev 0 in method ContractIdNT Value is never used
implicit def `ContractIdNT Value`[a_a1dk](implicit `ev 0`: ` lfdomainapi`.Value[a_a1dk]): ` lfdomainapi`.Value[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
^
.../com/daml/sample/mymain/ContractIdNT.scala:41: warning: parameter value eva_a1dk in method ContractIdNT LfEncodable is never used
implicit def `ContractIdNT LfEncodable`[a_a1dk](implicit eva_a1dk: ` lfdomainapi`.encoding.LfEncodable[a_a1dk]): ` lfdomainapi`.encoding.LfEncodable[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
^
* one more unused in daml-script
* special scaladoc rules may need silencer, too
* unused in compatibility/sandbox-migration
* more commas, a different way to `find`
- suggested by @remyhaemmerle-da; thanks
* Factor out tar/gzip reproducibility flags
* use mktgz in package-app
* Bazel managed tar/gzip
* Remove quiet = True
As stated in the comment this is no longer required with Bazel >= 3.0.
* Build package-app as a sh_binary
This way Bazel will manage the runtime dependencies tar, gzip, mktgz,
and patchelf.
package-app.sh changes directory so it needs to make sure that all paths
are absolute and that the runfiles tree/manifest location is forwarded
to programs called by package-app.sh.
* Avoid file path too long errors
* Fix readlink -f on MacOS
* Document abspath
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Extend `daml new` to accept template as an option
The two positional arguments keep confusing users so this PR changes
things to allow the template to be passed via `--template`. Using a
positional argument still works so this is not breaking.
I’ve updated all docs to use the less confusing syntax.
changelog_begin
- [DAML Assistant] You can now use ``daml new project-name
--template=template-name`` instead of ``daml new project-name
template-name``. The old CLI syntax continues to be supported.
changelog_end
* Update docs/source/getting-started/index.rst
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Upgrade nixpkgs revision
* Remove unused minio
It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.
* Update Bazel on Windows
changelog_begin
changelog_end
* Fix hlint warnings
The nixpkgs update implied an hlint update which enabled new warnings.
* Fix "Error applying patch"
Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.
* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681
Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.
```
git diff ':(exclude)WORKSPACE'
```
* Update rules_nixpkgs
* nixpkgs location expansion escaping
* Drop --noincompatible_windows_native_test_wrapper
* client_server_test using sh_inline_test
client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.
This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.
* daml_test using sh_inline_test
* daml_doc_test using sh_inline_test
* _daml_validate_test using sh_inline_test
* damlc_compile_test using sh_inline_test
* client_server_test find .exe on Windows
* Bump Windows cache for Bazel update
Remove `clean --expunge` after merge.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This limits the JVM max memory and initial memory in a few tests that
look like they might be using more than they have two and that run for
a long time so there is a high chance they end up running in parallel
with something else.
changelog_begin
changelog_end
* Move public code into daml-integration-api
CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
* Add a --timeout option to `daml ledger` commands
At least in the compatibility tests, we see timeouts fairly regularly
in the tests for fetch-dar. If we see those timeouts in the tests, I
see no reason why our users should never see that especially on
actually distributed ledgers and hardcoded timeouts are a bad idea
anyway so now they are configurable.
I’ve also changed the timeout for the fetch tests (those are the only
once I’ve seen timeout) to 120s.
changelog_begin
- [DAML Assistant] The `daml ledger` commands now accept a `--timeout`
option which can be used to change the timeout for API requests from
the default of 30s.
changelog_end
* Generate parties as uuids to avoid collisions
changelog_begin
changelog_end
* Match on Deadline Exceeded as well
changelog_begin
changelog_end
* Update compatibility/bazel_tools/test_utils/DA/Test/Util.hs
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Add some docs
changelog_begin
changelog_end
* Better version checks
changelog_begin
changelog_end
* fix imports
changelog_begin
changelog_end
* This is not how you exception
changelog_begin
changelog_end
* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Add `platform-version` field to `daml.yaml`
This PR adds the `platform-version` field to `daml.yaml`. Based on the
approach agreed upon in #6558, the logic for this all sits in
`daml-helper` and there are no changes to the assistant.
The details of how the logic work are in a comment so I’m not going to
repeat them here but the commands that are affected are:
- `daml sandbox`
- `daml sandbox-classic`
- `daml json-api`
- `daml start` (but only for sandbox and the JSON API, not for
Navigator or anything else)
For tests, I’ve added a test to the compat workspace that installs two
SDKs simultaneously and tries out various combinations and verifies
that we get the correct version. Open to other ideas for testing this
but that seemed like the most sensible option that actually tests what
we run.
changelog_begin
- [DAML Assistant] You can now specify the version of Sandbox and the
JSON API independently of your SDK version by setting
``platform-version`` in your ``daml.yaml``. This is useful if you
are deploying to a ledger that is running components from a
different SDK version. See
https://docs.daml.com/tools/assistant.html#project-config-file-daml-yaml
for details.
changelog_end
* Run platform-version tests
changelog_begin
changelog_end
* Fix tag globbing
changelog_begin
changelog_end
* fmt
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* Try to fix env vars
changelog_begin
changelog_end
* Remove hardcoded references to 1.2.0
changelog_begin
changelog_end
* Rephrase doc comment
changelog_begin
changelog_end
* get things to compile
changelog_begin
changelog_end
* maybe fix things for realz
changelog_begin
changelog_end
* Remove debugging output
changelog_begin
changelog_end
* Get angry at windows
changelog_begin
changelog_end
* why is windows
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* Respect project name in create-daml-app
Now `daml new foobar create-daml-app` creates a project called
`foobar` replacing the name everywhere. The tests now run twice once
with the original project name since that’s what we use in the GSG and
once with a modified one. I guess you could argue that only running
the second should be enough so I’m open to removing the first one.
fixes#6681
changelog_begin
- [DAML Assistant] `daml new foobar create-daml-app` now properly
respects the project name and creates a project called `foobar`
instead of hardcoding the name fo `create-daml-app`.
changelog_end
* .
changelog_begin
changelog_end
* Remove debugging output
changelog_begin
changelog_end
The default cache directories in yarn don’t work in the Bazel
sandbox. This results in Yarn creating tons of temp directories for
each test that are never cleared up.
This PR changes this to create actual temp directories that are
properly cleaned up.
changelog_begin
changelog_end
* Update rules_haskell
- Fixes the issue where a `sh_test` wrapping a `haskell_binary` couldn't
add runfiles as is the case with other Bazel rules.
CHANGELOG_BEGIN
CHANGELOG_END
* Save the runfiles environment at start-up
To work around the fact that `withProgName` overwrites `argv[0]`.
See https://gitlab.haskell.org/ghc/ghc/-/issues/18418.
* damlc_compile_test include damlc runfiles
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Buildifier now comes with a handy attachment to catch single `\`
characters inside strings and replace them with `\\` if the escape
sequence is invalid. Skylark/Python will do this at runtime anyway; this
just makes it clearer what the actual behavior is.
I needed to change `\` characters at the end of lines to `\\` manually
in order to stop Buildifier from simply concatenating the lines
together. Everything else was automatic.
CHANGELOG_BEGIN
CHANGELOG_END
This PR splits up the significantly too large `DA.Daml.Helper.Run`
module into modules grouped by the individual commands:
- `DA.Daml.Helper.Init` for `daml init`
- `DA.Daml.Helper.Ledger` for `daml ledger *`
- `DA.Daml.Helper.New` for `daml new`
- `DA.Daml.Helper.Start` for `daml start` and code for starting the
components that are part of that individually.
- `DA.Daml.Helper.Studio` for `daml studio`
- `DA.Daml.Helper.Util` as a kitchen-sink utilty moduel for things
that don’t really belong in one of the others.
This PR _only_ shuffles code around, there is no change in the
implementation.
changelog_begin
changelog_end
Returning the original relative path from withProjectRoot makes no
sense since we change the directory in there so this PR fixes this to
return the canonicalized project root.
fixes#6245
changelog_begin
- [DAML Compiler] damlc test --project-root now works with relative
paths as well.
changelog_end
This adds an empty template for the daml assistant. This is very helpful
for writing katacoda's, because otherwise you have to delete contained
source files in the `skeleton` project everytime as a first step. The
README in the `daml` directory is mainly there to get the daml directory
in the template.
CHANGELOG_BEGIN
CHANGELOG_END
* Make Bearer prefix optional in Haskell ledger bindings
This matches the change to the JVM bindings in 1.0.0
changelog_begin
- [DAML Assistant] The ``daml ledger`` commands no longer require the
``Bearer `` prefix in the access token file. This matches the behavior
of DAML Script and other SDK tools.
changelog_end
* Fix tasty crash
changelog_begin
changelog_end
* Update compatibility/bazel_tools/daml_ledger/Main.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* new --leak-passwords-firesheep-style option; functions to check forwarded protocol
* enforce https reverse-proxy in all JWT-accepting endpoints
* make HttpService.start take config record
* test that X-Forwarded-Proto or Forwarded is enforced
* use new start signature in daml-script tests
* use insecure http mode for ts codegen tests
* note on regex
* use insecure option in daml assistant integration tests
* log allowNonHttps setting
* add non-https option to more places in daml-assistant tests
* add non-https option to getting started guide
* rename --leak-passwords-firesheep-style to --allow-insecure-tokens
- per suggestion by @garyverhaegen-da, @hurryabit
CHANGELOG_BEGIN
- [JSON API] By default, checks that connections are made through a reverse-proxy
providing HTTPS, ensuring that JWT tokens don't leak. To disable this check,
such as for development, pass ``--allow-insecure-tokens``.
See `issue #5572 <https://github.com/digital-asset/daml/issues/5572>`_.
CHANGELOG_END
* daml start includes --allow-insecure-tokens by default
- as indicated by @cocreature
* Copy daml ledger test to compatibility
Specifically:
- `daml-assistant/daml-helper/test/DA/Daml/Helper/Test/Deployment.hs`
- `libs-haskell/test-utils/DA/Test/Sandbox.hs`
* Adapt daml ledger test for compatibility
Avoid dependencies on the daml repository and make the SDK and platform
components configurable.
* Add test certificates to compatibility
Taken from `@daml//ledger/test-common/test-certificates`.
* Define daml_ledger_test
CHANGELOG_BEGIN
CHANGELOG_END
* Add note on keeping daml ledger tests in sync
* ./fmt.sh
* Strictly require --sdk-version flag
Tasty doesn't allow options without a default value, so we default to
`error` to enforce a value being set.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Previously that resulted in us telling sandbox,navigator,… to connect
to port 0 which obviously does not work. I’ve changed the types a bit
to avoid accidentally passing on the cli argument to them.
changelog_begin
changelog_end
* Make DAML Triggers and DAML Script default to wall-clock-time
Now that sandbox defaults to wall-clock-time there is no reason why we
should not default in DAML triggers and DAML Script.
changelog_begin
- [DAML Triggers] ``daml trigger`` now defaults to wall clock time if
neither ``--wall-clock-time`` or ``--static-time`` is passed.
- [DAML Script] ``daml script`` now defaults to wall clock time if
neither ``--wall-clock-time`` or ``--static-time`` is passed.
changelog_end
* Make --static-time and --wall-clock-time exclusive
This PR gets yarn test running the Puppeteer end-to-end tests in the GSG integration test.
The first step of the test is adding the extra dependencies (Jest, Puppeteer and more). The GSG recommends a yarn add command, but this does not work against HEAD. This is because yarn add does not use resolutions in the parent package.json, and then complains about unknown versions 0.0.0 of the daml TS libaries. The solution here is to hack in the extra dependencies into the ui/package.json and then yarn install. This works, but it hard codes version numbers which we would need to maintain. I would like to be able to say version "latest", which is what yarn add would install.
The next step of the test is to copy the index.test.ts file and run yarn test in CI mode.
I've moved the GSG test to a new create-daml-app-tests target. It's marked "exclusive" in Bazel until we figure out how to avoid hardcoding port numbers. This is a bit tricky since the HTTP port is hardcoded in a couple of places in ui/package.json.
Finally, this PR gets the GSG testing docs to reference the code in the new templates/create-daml-app-test-resources folder.
changelog_begin
changelog_end
This PR adds an --output-file option to DAML Script that writes the
result of a DAML Script to a file and complements the --input-file option.
changelog_begin
- [DAML Script] ``daml script`` now has a `--output-file`` option that
can be used to specify a file the result of the script should be
written to. Similar to ``--input-file`` the result will be output in
the DAML-LF JSON encoding.
changelog_end
* Diff with messaging feature and some noise manually removed
* Bazel target to use patch file in other build targets
* Patch file as data dep for integration tests
* Attempt to patch and test messaging feature in create-daml-app test
changelog_begin
changelog_end
* Use exports_files instead of filegroup
* Remove file existence checks that don't make sense
* Add patch to dev_env and reference it from integration tests
* Include patch on windows for later
* Set up yarn env again after codegen
* Restore file check
* Fix typo in comment on util function
* Add Tasty steps to make process explicit
* Use messaging patch for code snippets in GSG
* Use messaging code from template instead of copy
* Remove copied message code
* Refactor script to copy template code with messaging patch
* No need to retry second yarn install (only local deps should be updated)
* Used `daml codegen java` instead of calling the codegen from maven
This should hopefully fix the issues with mismatched versions of
slf4j.
changelog_begin
changelog_end
* Move config to daml.yaml
* Remove alternative invocation via maven from docs
* Haskell: Add assertFileExists to DA.Test.Util
This PR adds a helper function `assertFileExists` that captures the
`doesFileExist ... >>= assertBool ...` pattern that is very common in
our Haskell test suites. It also adds the inverse
`assertFileDoesNotExist` function. Both functions are now used where
appropriate.
CHANGELOG_BEGIN
CHANGELOG_END
* Add directory dependency to test-util lib
CHANGELOG_BEGIN
CHANGELOG_END
Some of the dependencies of the `create-daml-app` seem to fail to
install without any good reason every now and then. We try to
circumvent this flakiness by running the `yarn install` step for the
UI three times before giving up.
CHANGELOG_BEGIN
CHANGELOG_END
There are multiple tests that setup an environment for yarn to find
our TypeScript libraries. This PR unifies this setup by providing a
single function to set it up.
CHANGELOG_BEGIN
CHANGELOG_END
Packages com.digitalasset.daml and com.daml have been unified under com.daml
Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.
CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
I've completely removed the possibility to call `daml codegen ts`. I'm
happy to add in back in a followup PR once I've seen that all our tests
pass without it existing.
CHANGELOG_BEGIN
CHANGELOG_END
* Add snapshot version support in daml-assistant.
changelog_begin
- [DAML SDK] ``daml version`` can now list the available snapshot
versions by passing the flag ``--snapshots=yes``.
- [DAML SDK] ``daml install latest`` can now include the latest snapshot version by passing the flag ``--snapshots=yes``.
changelog_end
* Keep backward-compatible --all behavior
* Fix comment
* Delete spurious punctuation
Previously we emitted 3 lines starting with a capslock `WARNING` which
seems to scare some people. Now we emit two lines without the
`WARNING` and point to the release notes since that seems better than
explaining to users how they upgrade without telling them what
changed.
It’s still two lines, I couldn’t come up with anything shorter that
seems reasonable. We might want to reduce the frequency at which this
is displayed as well but for now this should do.
changelog_begin
changelog_end
Change the output directory given to `daml codegen ts` from `daml-ts`
to `daml.js`. This is naming is in line with the fact that `daml2ts`
puts all generated packages into the `@daml.js` scope. A renaming of
`daml2ts` into `daml2js` is immiment.
This PR also changes a few small documentation issues the
search-and-replace has surfaced.
CHANGELOG_BEGIN
CHANGELOG_END
The test calls `daml new ... create-daml-app`, builds the DAR, runs
the TypeScript codegen, lints and builds the React app.
We should also run the tests for the React app but since that's most
likely tricky, I'll leave that for a followup PR so that we can land
at least some sanity checks now.
There's also a bit of cleanup to do around setting up a yarn workspace
to bring our TypeScript libraries into scope. Other test already solve
a similar problem in a way that is not general enough for this test.
I'll unify the setup in a followup PR as well.
CHANGELOG_BEGIN
CHANGELOG_END
- Move deployment tests (deployTest, fetchTest) out of integration-tests.
- Use DA.Test.Sandbox where appropriate.
- Split out code for useful test patterns: i.e. calling commands quietly, getFreePort.
changelog_begin
changelog_end
* Initial commit with create-daml-app master
* Include create-daml-app in build rule
* Make daml.yaml a template in version and project name
* Remove git attributes
* Remove license and azure config
* Remove scripts
* Don't overwrite config files in build rule
* Template version numbers in package.json, to be replaced by the assistant
* Rename to package.json.template
changelog_begin
changelog_end
* Add copyright headers
* Do template substitutions in all .template files
And don't special case daml new create-daml-app (so it treats it as a
regular template).
* Add create-daml-app to integration tests
* Remove WIP warning
* Move towards setup that works on head
* Make local copies of the TS libs in the templates tarball
* Hardcode project name for now
* Use isExtensionOf
* Remove service worker
* remove robots.txt (don't even know what it is)
* Revert "Make local copies of the TS libs in the templates tarball"
This reverts commit 1289581fb4a82af3ab534baf978a2c6ed895d538.
* Retemplatize TS lib versions. For head builds these will be installed using npm
* Remove daml/ledger from resolutions for daml-ts
* Comment about test secret
* Remove special create-daml-app assistant command and test that won't work anymore
* Remove redundant imports and export
* Remove old create-daml-app tests
* Remove yarn.lock
* Clean up integration test (just daml new and build atm)
* Add daml/ledger as a resolution for daml-ts
* Remove top level package.json
* Update daml.js version
* Use new import scheme for generated TS
* Update readme with new codegen and build steps
* Use start-navigator in daml.yaml
* Increase a couple of timeouts in tests (either sandbox or TS lib is a bit slower?)
* Update GSG intro with new build steps
* Remove daml2ts -p flag and --start-navigator flag from GSG instructions
* Don't use start-navigator flag in ui tests
* Temporary readme describing how to manually test the create-daml-app template
* Update code samples in app arch section of GSG
* Update code samples in testing doc
* Remove copied create-daml-app code
* Indent docs markers to be more subtle
* Update visible code in Messages (after) section
This needs to be kept up to date properly somehow.
* Update text to useLedger
* Restore code/ui-before copies until the Bazel magic is figured out
We need to make the template code a dependency in the Bazel rule as
otherwise we can't find the files in the docs build.
* Update create-daml-app/readme and make templates/readme more detailed
* Use jsx comments for docs markers so they don't show up in the app
This cause issues in combination with data-dependencies see #5330 and
we started warning if you don’t expose everything. This resulted in
quickstart-java producing a warning on every `daml build`.
changelog_begin
changelog_end
* Improve handling of exposed-modules with data-dependencies
Previously, we tried to rename all modules of a dependency via
--package. This fails if some of those modules are not exported. This
was trivial to hit as a user since the ``daml-trigger`` library made
use of this.
This PR adds a few things to improve the situation:
1. We only rename modules that are exposed. This fixes the issue if
you don’t actually reference a non-exposed module from your
data-dependency.
2. I’ve removed the exposed-modules from daml-trigger. I don’t think
they are essential here given that the module name has `Internal`
in the name and it’s too easy to have something that actually
references the non-exposed module since the types are reexported.
3. I’ve added documentation that mentions this issue.
4. I’ve added a warning if your exposed-modules are excluding some
modules. Maybe worth turning this into an error in the future.
changelog_begin
changelog_end
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
* Use com.daml as groupId for all artifacts
CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END
* Add 2 additional maven related checks to the release binary
1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId
* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
* Make new sandbox the default.
changelog_begin
- [DAML SDK] The new sandbox is now the default that runs with ``daml sandbox`` and ``daml start``. The command ``daml sandbox-next`` has been removed. The old sandbox can be invoked via ``daml sandbox-classic`` and ``daml start --sandbox-classic=yes``.
changelog_end
* Update descriptions.
* Change it to a switch
* Change switch help
* Recapitalize
* Move 'withEnv', call it from daml2ts tests
changelog_begin
changelog_end
* Fix withEnv call to ensure that TASTY_NUM_THREADs is set
withEnv replaces the whole environment so we need to set everything we
care about.
* withEnv replaces the whole environment so we need to set everything we
care about.
* Apparently applying the same fix has destabilized Windows
* Try even harder to get daml assistant tests passing on Windows again
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
The way we use `withEnv` it is really intended to set a few
environment variables in an already existing environment instead of
clearing everything before. This PR changes it to do only that.
changelog_begin
changelog_end
closes#5280
Those were originally used for `damlc migrate`. However, that command
no longer exists and the only call site of `runNew` sets those
arguments to an empty list.
changelog_begin
changelog_end
* New comamnd: daml ledger fetch-dar
`daml ledger fetch-dar [PID] [PATH]`
Download a `Package` and it's dependencies from a ledger, given a root `packageId`, and re-construct a valid `.dar` file. Addresses issue #5037.
The original package names are not reconstructed.
CHANGELOG_BEGIN
CHANGELOG_END
* address some comments
* fix spello
* attempt: recoverPackageName
* recover main package name & version
* Try to fix integration tests on Windows
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
There was a brief period of time where `daml build` did not work
outside of the assistant. When we fixed that we added it to the
integration tests since there was no other test suite that used damlc
as a binary (as opposed to using it as a library which runs through
different codepaths). However, in the meantime we have tons of tests
all over the place (e.g. the packaging tests) that call `damlc build`
outside of the assistant so these tests serve no purpose.
Also they are somewhat confusing since the point of the integration
tests is to test an installed SDK whereas these tests do not need an
installed SDK (that’s the whole point of those tests).
changelog_begin
changelog_end
This ensures that -l and -p work properly in the integration tests
since they no longer depend on the order.
There is lots of other crap to cleanup in those tests but I’m trying
to keep it to small steps.
changelog_begin
changelog_end
* Depend on LF version specific daml-libs
* daml-script.dar build multiple LF versions
CHANGELOG_BEGIN
[DAML Script] The `daml-script` library is now available in multiple LF
versions, namely 1.7, 1.8, and 1.dev.
CHANGELOG_END
* daml-trigger.dar build multiple LF versions
[DAML Triggers] The `daml-trigger` library is now available in multiple
LF versions, namely 1.7, 1.8, and 1.dev.
* Keep daml-script.dar available for tests
* Keep daml-trigger.dar available for tests
* daml-libs LF versions integration test
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Integrate create-daml-app into the assistant
fixes#4868
changelog_begin
- [DAML Assistant] Add a new ``daml create-daml-app`` command for creating a project based on
`create-daml-app <https://github.com/digital-asset/create-daml-app>`_.
changelog_end
* Try random things hoping to fix windows
* Try random things hoping to fix things on macos
* Allow controlling navigator startup in daml.yaml
While I’m not entirely convinced the default atm is right, making it
configurable seems like an easier solution than bikeshedding the default.
changelog_begin
- [DAML Assistant] You can now disable starting navigator as part of
``daml start`` in your ``daml.yaml`` file by adding ``start-navigator: false``.
changelog_end
* Update docs/source/tools/assistant.rst
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
* Use --start-navigator=true in the docs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Bump openssl
The previous one has stopped working for some reason :sadpanda:
changelog_begin
changelog_end
* daml-assistant: Add `--wall-clock-time` to the Sandbox Next test.
Missed this due to doing two things at once.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* daml-assistant: Add `daml sandbox-next`.
CHANGELOG_BEGIN
- [DAML Assistant] You can now run a pre-release version of Sandbox with
``daml sandbox-next`` so you can test it out and verify everything is
working as expected. Running this will launch Sandbox rebuilt on a
more modern architecture. An upcoming release of DAML will switch over
to the new implementation by default.
CHANGELOG_END
* daml-assistant: Explain that sandbox-next is experimental.
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* daml-assistant: Copy-pasta an integration test for `daml sandbox-next`.
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* sandbox: Fail to start if a time mode is not explicitly specified.
CHANGELOG_BEGIN
- [Sandbox] Sandbox is switching from Static Time mode to Wall Clock
Time mode as the default. To ensure that our users know about this,
for one version, there will be no default time mode. Instead, users
will have to explicitly select their preferred time mode by means of
the `--static-time` or `--wall-clock-time` switches. In the next
release, Wall Clock Time will become the default, and users who are
happy with the defaults will no longer need to specify the time mode.
CHANGELOG_END
* daml-script|triggers: Specify time mode when testing against Sandbox.
* daml-assistant: Default the Sandbox to wall clock time.
CHANGELOG_BEGIN
- [DAML Assistant] Initializing a new DAML project adds a switch to
``daml.yaml`` to ensure Sandbox can continue to start with ``daml
start``::
sandbox-options:
- --wall-clock-time
CHANGELOG_END
* docs: Update the DAML Script and Triggers docs to use Wall Clock time.
It's now what Sandbox will use by default when using `daml init`.
* docs: Change the Quickstart to run Sandbox in wall clock time.
This explains why the contract IDs may vary.
It also updates the manual release testing script to match.
We previously had 3 slightly different but consistently shitty logic
for handling this in the tests for daml-helper daml repl and the
Haskell ledger bindings. This PR introduces a module that is flexible
enough to capture all their needs and hopefully is somewhat less
shitty.
changelog_begin
changelog_end
* Share test certificates
This is primarily an attempt at making sure my contribution stats
remain negative but I think it’s a nice cleanup. The only difference
in the certs used by daml-helper which are now used everywhere is that
they use a different CN for the CA and the server. This is required to
make openssl happy (which is used by the daml-helper).
changelog_begin
changelog_end
* Fix script and trigger tests
This adds CLI parametrs for connecting via TLS following the scheme
used by navigator, extractor and `daml ledger`.
changelog_begin
- [DAML Script] Support TLS. Enable it by passing ``--tls``. You can
set certificates for client authentication via ``--pem`` and
``-crt`` and a custom root CA for validating the server certificate
via ``--cacrt``.
- [DAML Triggers - Experimental] Support TLS. Enable it by passing ``--tls``. You can
set certificates for client authentication via ``--pem`` and
``-crt`` and a custom root CA for validating the server certificate
via ``--cacrt``.
changelog_end
Currently sandbox only supports TLS if you also enable client
authentication. There is no reason for why this has to be the case and
for things like DABL we want TLS without client authentication so it’s
useful to be able to test this in sandbox. This PR introduces a
`--client-auth` flag that allows you to configure the behavior. The
default is the current one of requiring client authentication.
This PR does not yet update Java clients, however, the Haskell client
supports this already and is used to test this functionality.
I’ve also added a section in the documentation on TLS (there were no
docs at all so far).
changelog_begin
- [DAML Sandbox] When Sandbox is run with TLS enabled, you can now
configure the requirement for client authentication via
``--client-auth``. See
https://docs.daml.com/tools/sandbox.html#running-with-tls for more information.
changelog_end
changelog_begin
changelog_end
For now this is only used for the daml-helper tests. I’ll shuffle
things around and use it for all tests in a separate PR.
This PR adds TLS support to DAML helper both via client certs and
without (although the latter is not tested so far since atm this is
not supported by sandbox). The CLI options follow the scheme used by
navigator/extractor/… with the addition that you can just pass `--tls`
which will turn on TLS without custom root certs or client certs.
changelog_begin
- [DAML Assistant] You can now connect to ledger via TLS for ``daml
deploy`` and ``daml ledger`` commands. See
https://docs.daml.com/deploy/generic_ledger.html for more information.
changelog_end
* daml-assistant tests: fix flaky test on Windows
```
bazel test //daml-assistant:test --test_arg=--quickcheck-replay=425714
```
failed on Windows in the test-case:
```
tail . ascendents == ascendents . takeDirectory
```
In that case the two path components given two the test case shrink to
`p1 = "a"` and `p2 = "\\"`. Confusingly, on Windows
```
isRelative "\\" == True
```
while
```
"a" </> "\\" = "\\"
```
This is documented behaviour on Windows, see [1] and [2].
Using `p1 </> p2 /= p2` instead of `isRelative p2` works around this.
[1]: https://hackage.haskell.org/package/filepath-1.4.2.1/docs/System-FilePath-Posix.html#v:-60--47--62-
[2]: https://hackage.haskell.org/package/filepath-1.4.2.1/docs/System-FilePath-Posix.html#v:isRelative
* Fix ascendants test group name
`ascendants` is defined in `DA.Daml.Project.Util`, not
`DA.Daml.Assistant`.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
On MacOS using Java 13 (at least that was the only combination where
I’ve managed to reproduce this). Sandbox spits out errors of the
following form in ``daml start``
```
java.net.SocketException: Connection reset
```
This appears to be caused by the TCP requests that we send to detect
whether sandbox has started. Switching to using the ``--port-file``
option makes this issue go away.
fixes#4670
changelog_begin
- [DAML Assistant] In certain configurations ``daml start`` produced
a (harmless) ``Connection reset`` log message. This has now been
fixed.
changelog_end
changelog_begin
- [DAML Assistant] You can now specify options for
Sandbox/Navigator/the HTTP JSON API in ``daml.yaml`` via
``sandbox-options``/``navigator-options``/``json-api-options``. These
options will be picked up by running ``daml start``. This is
particularly useful for specifying ``--wall-clock-time`` and for
specifying a fixed ledger ID during development.
changelog_end
fixes#2993
This introduces a `HasSubmit` typeclass (following the naming scheme
of `HasCreate`, …) and instances for `Scenario` and `Script`. This
avoids the need to hide `submit` in every single DAML script.
changelog_begin
- [DAML Standard Library] ``submit`` and ``submitMustFail`` are now
overloaded so that they can be used in both scenarios and DAML script.
changelog_end
* Remove damlc migrate
``damlc migrate`` hasn’t worked for quite a while and we emitted a
warning for months so given that we don’t have plans to make it work
again in the near future, I think it does more harm than good to keep
it around.
changelog_begin
- [DAML Compiler] After being deprecated for a while the ``damlc
migrate`` command has now been removed. See
https://docs.daml.com/upgrade/ for up to date documentation
on model upgrades.
changelog_end
fixes#3704 (by removing the tests 😇)
* yeah the windows cache is once again broken \o/
* Revert "yeah the windows cache is once again broken \o/"
This reverts commit 38d7877aa4.
* Add init-script daml.yaml field
CHANGELOG_BEGIN
[DAML Script -- Experimental] You can now define an ``init-script`` in
the ``daml.yaml`` file. If present, that DAML script will be executed
to initialize the sandbox on ``daml start``. This can be used instead
of the ``scenario`` field.
CHANGELOG_END
* Add integration test for init-script
* Generate JWT token in tests
Addressing review comment
https://github.com/digital-asset/daml/pull/4685#discussion_r383835050
* Remove unnecessary daml calls
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
* Report assistant commands and errors via a logger.
CHANGELOG_BEGIN
CHANGELOG_END
* Hook up daml-assistant to a GCP logger.
* fix test case
* fix more tests
* Check opted in status in assistant.
* Anonimize args that have unexpected characters.
* More agressive anonimization
* add missing containers dependency
We were still using the old token format without the
https://daml.com/ledger-api key. This resulted in a warning on every
call to `daml start` which looks rather ugly. I’ve changed it to
construct the token using the JWT library instead of hardcoding it
which should make it easier to modify in the future
changelog_begin
changelog_end
* Enhanced completions for the assistant
changelog_begin
- [DAML Assistant] The assistant can now do completions for SDK
commands, e.g., ``daml ledger upl<TAB>`` will complete to ``daml
ledger upload-dar``.
changelog_end
* Apply suggestions from code review
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
It turns out that various environments (hello docker) start with a
closed stdin. This means that things like `daml sandbox` exit
immediately. Since users that care about the new behavior are likely
to know about it, we only enable it if a specific flag is given.
changelog_begin
- [DAML Assistant] The new behavior introduced in ``0.13.51`` to shut
down when stdin is closed is now disabled unless you explicitly
enable it by passing ``--shutdown-stdin-close``.
changelog_end
* Fall back to regular bash completions for command arguments
This addresses the first part of #4369 by falling back to the bash completions.
I sadly still haven’t figured out how we can stop to completing to a
directory if the command also exists as a directory.
changelog_begin
- [DAML Assistant] Bash and Zsh completions will now fall back to
regular file completions after the command argument.
changelog_end
* Shut up hlint
* Shut down when stdin is closed.
changelog_begin
- [DAML Assistant] The DAML assistant will now shut down long-running
processes like sandbox when stdin is
closed. This is mainly useful on Windows, where process APIs often
kill the process in a way that does not allow it to do any cleanup, in
particular, we cannot stop child processes.
changelog_end
For now, the logic for this is only in daml-helper which is the only
thing starting long-running processes (in particular sandbox). There
is a long inline-comment explaining why this is not on the assistant
itself.
fixes#4168
This test just checks that navigator starts and we get a 200
response. This is obviously not a very comprehensive test but it would
have caught the issues that prevented 0.13.44 an 0.13.45 from getting
published so it seems like a good idea to test this.
changelog_begin
changelog_end
* Add Zsh completions for the assistant
This is slightly more annoying for users since at least on Linux there
doesn’t seem to be a user-writable directory that is in `$fpath` by
default. I think on Zsh we could probably write them to
/usr/local/share/zsh/site-functions but I’d rather avoid platform
specific logic here. I would expect that Zsh users are usually
somewhat comfortable with modifying the config and this also matches
other installation instructions, e.g.,
https://github.com/zsh-users/zsh-completions#manual-installation.
On the plus side, the completions look significantly nicer in Zsh
since they include the description of commands.
CHANGELOG_BEGIN
- [DAML Assistant] Zsh completions for the DAML Assistant are now
installed as part of ``daml install``. To activate them you need to
add ``~/.daml/zsh`` to your ``$fpath``, e.g., by adding
``fpath=(~/.daml/zsh $fpath)`` to the beginning of your ``~/.zshrc``
before you call ``compinit``.
CHANGELOG_END
* Fix tests
* Update daml-assistant/src/DA/Daml/Assistant/Install/Completion.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
* Improve bash completions for daml-assistant.
* Install bash completion script automatically
* Better default logic for bash completions
* specifically -> explicitly
* Copyright headers
* Better hook logic and refactoring
* Handle non-standard installations more robustly.
* Handle CI env & add changelog note.
CHANGELOG_BEGIN
- [DAML Assistant] Bash completions for the DAML assistant are now
available via ``daml install``. These will be installed automatically
on Linux and Mac. If you use bash and have bash completions installed,
these bash completions let you use the tab key to autocomplete
many DAML Assistant commands, such as ``daml install`` and
``daml version``.
CHANGELOG_END
* Mention bash completion in assistant docs
* Remove promises
* Start on daml test-scripts
* Run all `Script a` as test cases
* LedgerClient: Expose PackageManagementClient
To enable DAR uploads
* Upload the DAR to the ledger
* Start sandbox if no ledger specified
* Format daml test-script
* Fix deprecation warning on ActorMaterializer
* Add test-case //daml-script/tests:test_daml_script_test_runner
* Add daml test-script command
CHANGELOG_BEGIN
- [DAML Script - Experimental] Allow running DAML scripts as test-cases.
Executing ``daml test-script --dar mydar.dar`` will execute all
definitions matching the type ``Script a`` as test-cases.
See `#3687 <https://github.com/digital-asset/daml/issues/3687>`__.
CHANGELOG_END
* daml-test-script enable logging
* Remove outdated TODO comment
* daml script-test More elaborate test-caseo
Compare to expected output and add failing test-case
* daml test-script Don't abort on test-failure
Before the test runner would abort on the first failed test-case. This
occasionally introduce additional test-failures if the sandbox was
torn down half-way through execution.
* ./fmt.sh
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
As mentioned in the inline comment, this matches the behavior on the
Scala side and given that most people configure their editors to
always add trailing newlines, this is quite convenient (especially
given that the error message is horrible if you do include the
newline).
CHANGELOG_BEGIN
- [Sandbox] Restore 0.13.38 logging behaviour.
- [Navigator] Restore 0.13.38 logging behaviour.
- [Extractor] Restore 0.13.38 logging behaviour.
- [Internals] As of 0.13.39, we merged a number of internal JAR files in
the SDK tarball to reduce its size. These jars used to be standalone
JARs you could invoke as e.g. ``java -jar sandbox.jar <args>``. As a
result of merging the jars, they lost their individual ``logback.xml``
configuration file. Although running the jars directly was (and is
still) not supported, note that you can now achieve the same behaviour
with e.g. ``java -Dlogback.configurationFile=sandbox-logback.xml -jar
daml-sdk.jar sandbox <args>``.
CHANGELOG_END
* daml assistant expected auth token in Bearer format
* Daml assistant does no validation of the auth token before passing in on to the ledger.
* clarify code with newtype Token
This is a first step towards making sure that the package ids for
types defined in daml-prim and daml-stdlib don’t change. This PR
mostly adds all the necessary infrastructure for that and moves
GHC.Types and GHC.Prim to make sure it works.
Until data-dependencies are really solid and we have verified that we
no longer have performance issues with an increasing number of Haskell
packages, we still include the source files in daml-prim and then just
rewrite the references.
We will also need to add tests that these packages really have stable
ids but I’ll leave that for separate PRs since this doesn’t make that
much sense anyway until all of the types have moved to stable
packages.
CHANGELOG_BEGIN
- [DAML Compiler] The modules GHC.Prim and GHC.Types from daml-prim
have been moved to separate packages.
CHANGELOG_END
This makes sure that navigator picks up things such as
`frontend-config.js` which is needed for custom tabs.
To get there, this PR does two things:
1. Change navigator such that it prefers a config file specified
explicitly via -c over the SDK config file.
2. Change daml-helper to pass in the config file via -c instead of
changing the directory
CHANGELOG_BEGIN
- [DAML Assistant] ``daml ledger navigator`` now loads
``frontend-config.js`` properly.
- [Navigator] Explicit config files passed via ``-c`` are preferred
over ``daml.yaml``.
CHANGELOG_END
* Error handling improvements,
partitioning the Soruce[Error \/ JsValue], folding all errors and
producing the final status and errors element based on the above fold
* Test case + fixes,
for some reason GeneratorDrivenPropertyChecks does not like AsyncFreeSpec
* changing the order of elements in the response,
would be better if we were checking the JSON objects not JSON strings.
* Start moving all the jars into a single jar.
* Fold navigator into daml-sdk jar
* include sandbox
* Remove unnecessary compileDeps.
CHANGELOG_BEGIN
- [DAML SDK] Reduced the size of the DAML SDK by about
60% uncompressed, 70% compressed, by deduplicating Scala
dependencies.
CHANGELOG_END
* update copyright header
* buildifier fix
This reenables the migration checks. The migration command now only
generates stumbs for conversions instead of using the formerly (generic)
upgrade template.
* Add authentication to Java identity client
Contributes to #3626
No changelog added right now, it will be added with the PR that
effectively closes the ticket.
Moves `LedgerCallCredentials` to a shared package.
Establishes the test framework for other Java clients.
* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731
* Address https://github.com/digital-asset/daml/pull/3630#discussion_r350663731
* Rebase and fix compilation errors
* Fix type of jar in artifacts.yaml
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Fix more errors emerged after rebase
* Add dependency for DAML assistant ITs
* Add missing dependency
* Fix wrong dependency
* fix release
We don't drop full support from the compiler yet but rather ban their use by
adding a check to the preprocessor. We'll remove the actual support as we go
along with fixing the upgrading story.
CHANGELOG_BEGIN
- [DAML Compiler] Make the experimental feature "generic templates"
unavailable. The current implementation is at odds with other, more important
language features still under development.
CHANGELOG_END
* support --access-token-file in SDK
* rename: HostAndPort -> LedgerArgs; HostAndPortFlags -> LedgerFlags
* integration test for --access-token-file
* doc & more help
* address comments: use FilePath
* adapt to renamed arg: --auth-jwt-hs256-unsafe
* avoid use of single quotes on command line
* Set Sdk-Version in DAR Manifest to compiler SDK version.
CHANGELOG_BEGIN
- [DAML Compiler] Bugfix: The Sdk-Version field in a DAR manifest file
now matches the SDK version of the compiler, not the sdk-version field
from daml.yaml. These are usually the same, but they could be different
if you set the DAML_SDK_VERSION environment variable before running
``daml init`` or ``daml build``.
CHANGELOG_END
* Fallback to daml.yaml if env var not set
* add yaml dependency
* Always require sdk-version, and emit warning on mismatch with env var
* More explicit about where override comes from
* Add packaging regression test
* Move packaging tests to a separate test suite
Originally, we had these as part of the integration test suite since
`bazel run damlc build` couldn’t locate `ghc-pkg` but that has been
fixed for a while.
Moving it to a separate test suite speeds things up as the integration
tests are rerun very often and also makes development much more
convenient since the new test suite supports `-p` properly to filter
to specific tests.
For now, I’ve left the upgrading tests as part of the integration
tests. I expect that we probably want to factor those out to another
test suite as well.
* fix warnings
* language: fix for tuple types in source generation
We correct the source generation from dalfs containing TupleN data
types. A test is added to the integration tests of daml assistant and
the `generate-src` command is improved to make this test possible.
* check that tuple type comes from DA.Types
* check that compilation succeeded
* pattern match golf
* check for tuples between 2 and 20
This change allows to only import `module A` from the instances package
if data dependencies are present instead of `module A` and `module
AInstances`.
* language: introduce data-imports
Right now the user experience for importing dalfs and dars from
different sdks is quiet confusing. This PR tries to solve this. We add
an additional field `data-imports` to daml.yaml. These imports can come
from different SDK's and we will generate interface files containing the
data types and their Template instances.
This also simplifies the migration command, as it now always imports the
respective packages as `data-imports`.
* language: cross sdk dalf/dar imports
The final piece for cross sdk imports. With this PR we can import the
data types of packages and dalfs that were created with different sdks.
This is done by generating interface files from dalfs and an 'instances'
package that contains the template instance definitions of template data
types. The instances itself are defined via the `external` keyword,
which is inlined to proper daml-lf instance definitions given in the
respective dalf package.
We test that cross sdk imports work by importing the `simple-dalf` in
the daml-assistant integation tests and running a scenario.
Currently we hide modules, for which we don't define an alias, when we
specify one alias for *one* module. With this change, all modules
without aliases are still exposed.
* language: refactoring of iface file generation and package db setup.
This is a refactoring of the damlc part that creates the package
database and the code generation for interface files. This is a
preparation for the cross sdk imports.
We also add an internal command to damlc to generate generic instances
code and use simple copying via {..} in the migration command. An
additional test checks that migration via generics still works.
* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/Upgrade.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* clearer description for generate-gen-src
* updated documentation
* correct copy/pasta mistake
* added a comment on different build options in migration command.
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* resolve dalf paths from dar manifest
* added a comment on different headers in upgrade modules.
* removed monoid instance for ExtractedDar
* Update bazel-common to fix javadoc issues
Specifically, to fix the following error
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
javadoc_library(name = 'rs-grpc-bridge_javadoc')
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```
* Define Maven deps using rules_jvm_external
* Pin artifacts
* Remove bazel-deps generated targets
* Remove bazel-deps
* Switch to rules_jvm_external targets
* update bazel documentation
* pom_file: There are no more bazel-deps targets
* BAZEL-JVM.md `maven_install` typo
* language: dalf imports and a test
This adds the possibility to directly import dalfs in a project. We test
that we can import the `simple-dalf` in the daml-assistant integation
tests. For now we only check that data type generation works, not yet
the template instance.
The following was fixed: When rewriting package self references, this
changes the hash of the package later on and leads to different package
hashes. Also we need to be careful to write the orignal binary
representation to this and not re-encode it because the encoding might
have changed with a different sdk.
* addressing moritz's comments.
* windows doesnt like bazel paths
* Add a first draft of documentation for DAML triggers
The API will still change in a bunch of ways but I’d rather get some
docs in place now and update them as we change things than not have
any docs at all.
* Fix path to daml.yaml
* s/bot/trigger/
* fix source code markers
* Fix tests
* Update docs/source/triggers/index.rst
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* language: fix: don't look for daml files in hidden directories
Fixes#3134. When locating daml source files, we need to make sure we're
not recursing into the .daml directory, because it contains source files
of the dependencies.
* Update daml-assistant/integration-tests/src/Main.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This moves the creation of a package database from given dalfs out of
the migrate command and into the init command. In particular, this makes
the process of creating a package database independent of the migrate
command.
It also changes the way this package database is created to be only
dependend on given dalf files.
* Flattening results of the `/contracts/search`,
returning a `ActiveContract` instead of `GetActiveContractsResponse`
* Fixing the test,
it is an empty array instead of empty GetActiveContractResponse object
* Updating examples
* Updating release notes
* Update unreleased.rst
Accepting suggested changes
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
* Resolving master merge conflicts
This should make `daml start` a bit more useful since you don’t have
to switch to starting all processes separately once you start
deviating from the defaults, e.g., I found myself wanting to specify a
custom ledger id during the hackathon. This is part 1 of ##2993.
The filename of the dar is not something that you should rely on as
evidenced by the fact that we have a -o option to change it to
something completely different.
* Use a consistant include dir for cwd
See https://github.com/digital-asset/ghcide/pull/114 for the actual
fix.
This PR just bumps ghcide and adds a regression test. I’ll change the
revision before merging, I just want to test CI for now.
fixes#2929
* Switch to proper ghcide revision
* writeIfacesAndHie no longer exists
* Add changelog entry
* Maybe I should try to compile code before committing but I don’t want to
* Fix ghcide exe
* language: support for automatic embeddings/projections in migation
This adds support to automatically generate embeddings/projections if an
optional field to a contract template is added. This allows generation
of migration contracts for these kind of updates.
* only project None entries
* added a test for optional field embedding/projection.
* filter out exception stack traces logged at debug
* Adding logback.xml for DAML Assistant release
* Fixing readme
* Fixing readme
* Use the config file in the assistant
* add Numeric.java
* ledger-api: rename `decimal` field to `numeric` in value protobuf
* Address Gerolf's comment
* ledger-api: add missing renammings
* ledger-api: relax syntax of numbers that can be sent as numerics
* extractor: fix
* leger-api: change format of number though ledger api
* daml-lf: fix numeric regexp
* ledger: fix tests
* language: compile everything in the source directory
This removes the need to specify a 'main'. Instead we 'source' in
daml.yaml should point to the source root directory.
* language: smoothing out the migration experience
Some improvements to make the migration process simpler when migrating
between packages with the same name, e.g. foo-1.0.0 -> foo-2.0.0. This
is the main use case.
* language: append the version to the output dar name by default.
We now by default output foo-1.0.0.dar instead just foo.dar. Also the
maven coordinate default naming got removed.
* fixing integration tests and quickstart.dar occurences
* Only change project root once
As described in #2449, calling withProjectRoot' twice breaks with
relative paths and is also just silly so this PR fixes this by
factoring out the actual logic from init from execInit which does the
project root thingy.
* Add a regression test
* Add link to deploy URL in daml ledger --help and daml deploy --help
* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
Despite, a lot of searching around, I couldn’t find any information on
why exactly quoting "code" causes issues but removing the quotes seems
to fix the issue ¯\_(ツ)_/¯
* Fix daml build for usernames with spaces
This PR fixes two issues caused by having spaces in your username:
1. On Windows, we need to quote the path to the daml binary in the
batch wrapper (quotes are not valid in usernames, so no need to worry
about escaping them).
2. Invoking ghc-pkg via callCommand broke since shells are
terrible. Luckily, we can easily get away with just using callProcess
here.
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* language: put hash in package directories
We put the package id of the main dalf of a package into the directory
names, where we store the files of that package in the package database.
This way we make sure that two equally named packages don't overwrite
their dependencies and files.
* Supporting producing sdist tarballs for the HS ledger bindings
The README.md has an explanation for how you can use this.
This should hopefully allow others to experiment with the bindings.
Currently, if you do `daml init foo` and there's a file `foo/Main.daml`,
you'd end up with a file `foo/daml.yaml` whose `source` entry is
`foo/Main.daml`when it should actually just be `Main.daml`.
This PR fixes this problem.
* daml deploy: first version
* fix build: replace callCommand with runProcess_
* abstract/share: doBuild, getDarPath
* hide deploy command for now
* make sandbox port configurable for both deploy & start daml-helper commands
fixes#2142
It turns out that typed-process has the behavior we want so rather
than rolling our own version of `withCreateProcess`, I just switched
to that.
extractTarball tries to restore the ownership stored in the
tarball. This goes wrong when the tarball is fetched from CI since the
uid/gid might not exist locally.
* Bazel: 0.24.0 -> 0.27.0
* Update rules_haskell for Bazel 0.27 compatibility
* Update bazel-deps and bazel-watcher
* Windows escape JVM flags
* load commands at top of .bzl file
Bazel 0.27 no longer allows load commands that are not at the beginning
of the file.
* Update Bazel rules
* subpackage boundary
* native is not defined in BUILD files
* yarn: @bazel/hide-bazel-files
Seems to be required since latest rules_nodejs version. Otherwise, yarn
fails with errors about existing BUILD or BUILD.bazel files.
* grpc-java plugin visibility
* Update fat_cc_library
* Nix Python3 toolchain
* Iteration over depset
* dev_env_package: Create symlinks one level deeper
To prevent symlinking the BUILD file as well. The nested BUILD file
confuses Bazel as of 0.27 and rules_nodejs cannot find the node
executable anymore.
* Update rules_nodejs
* Add managed_directories for node_modules
* hie-bios: Extract bazel-genfiles from bazel info
Bazel 0.27 changed the genfiles location which breaks the hie-core test
on macOS.
* update cc_wrapper to Bazel 0.27
* bazel info -> bazel info bazel-genfiles
* Fix typo in BUILD
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* language: upgades: generate upgade code from dalfs
This switches the `migrate` command to read dalf files instead source to
generate upgrade code. The dalf files are read, data types are converted
back to haskell source, and from this we can generate generic instances
again.
There are two known issues that we want to address next:
- Duplicate generic instance definitions will make the build fail.
I.e this only works for data declarations that have no generic
instances
- Data declarations of the form `data Foo = Foo ()` can not be
converted back to haskell source.
* Update rules_haskell and static GHC
Remove patches that have been upstreamed or are no longer required.
Update still required patches to match the new rules_haskell version.
Previously we patched rules_haskell to coerce GHC into using static
Haskell libraries in most places. In particular we moved hs-libraries
entries into extra-libraries entries in the package configuration files.
A much cleaner approach is to compile GHC with a static RTS, then GHC
will by itself choose to load static Haskell libraries.
* Remove haskell_cc_import
* da-hs-daml-cli -> daml-cli
* da-hs-damlc-app -> damlc-app
* windows: hanging GRPC FFI call problem resolution
* windows: fix visual test
* windows: enabled back again some of disabled daml-assitant tests
* marking daml_ghc_integration_test as large
language: upgrades: dont derive generics for data types having the instances already
We only derive generic instances on the fly for data types that don't
have them already.
* Add support for haskell_repl targets in da-ghci
* Change default target
* Revert newline loss.
* bazel fetch before bazel query
* Make a top level repl target the default.
* Add da_haskell_repl dependency in //BUILD
* Fix syntax error
* Fix bazel formatting...
* Rename DamlHelper modules to make //:repl work
* DamlHelper -> DamlHelper.Run
* Update the import in DamlHelper.Main
* Fix bazel rules again
* Update DamlHelper import in integration-tests
* language: generate module containing generic instances
The migrate command generates generic instances of data types and puts
them in a separate module. For now this only works if the data type
doesn't have a generic instance already, we'll deal with this case in
the next PR.
* Add --install-assistant option
* Better doc on iActivate
* Determine whether to install assistant based on new flag
* Fix logic and update install warning
* Remove --activate from install.sh
* Add missing ]
* Change deprecation warning.
* Add release notes for new option.
* Update help text for install-assistant
* Fix release notes
* language: feature: initial implementation of a 'migrate' command
We add a 'migrate' command to daml assistant that generates a project
that allows to migrate contract instances from package1 to package2.
This first version reads both package1 and package2 from source. As a
next step we read only the dalfs from package1, because it might have
been created with a different compiler.
* Simplify daml version output
* Make sure project version is really from daml.yaml
* Calculate latest version more thoroughly.
* Only show latest version if available
* Add test case for damlc test-files
* Separate damlc test-files from damlc test
* Fix rules_daml/daml.bzl daml_test
* damlc test-files --> damlc test --files
The project options are still relevant for the --files case, as it may
be necessary to change into the project root directory in order to
locate the project package database.
This was already possible before via the DAML_PROJECT environment
variable but for users that want to call damlc directly, e.g., via
damlc.jar a CLI flag can be more convenient.
* List all available versions.
* Add --all flag in daml version
* Save version list to cache
* Update version cacheing logic.
* Linting error
* PR revisions.
* Update release notes.
* Update daml-assistant/src/DAML/Assistant/Version.hs
* Update docs/source/support/release-notes.rst
Co-Authored-By: Beth Aitman <bethaitman@users.noreply.github.com>
* no var no problem
further refactor
introduced InfraState
a bit less vars
encapsulating closes
SandboxServer starts automatically
rebase fixup
collecting state into a single object
some cleanup
removing exposed materializer
LedgerBackend is closed in SandboxServer
changed ownership of Ledger
fixing perf tests
fixing some compile errors
formatting
removing unused method
fixing integration test to use correct dar file
fixing issue with PostgresFixture and SandboxResource
Fix integration tests on Windows
* fixing rebase artifacts
* Add an option to not modify PATH in daml install --activate
This is particularly useful in test suites where you install to a
temporary directory that should not be added to the registry.
* Use yes/no/auto
* Make check more robust
* Separate version logic out of DAML.Assistant.Env.
* Refactoring some of the exception handling.
* Update daml version command.
* Uncommit linting atrocity.
* Started working on daml init.
* Implement daml init.
* Nicer messages and nicer field generation.
* Cleaning up a duped definition.
* Review revisions
