Commit Graph

264 Commits

Author SHA1 Message Date
Moritz Kiefer
29ed16b4cc
Improve handling of exposed-modules with data-dependencies (#5330)
* Improve handling of exposed-modules with data-dependencies

Previously, we tried to rename all modules of a dependency via
--package. This fails if some of those modules are not exported. This
was trivial to hit as a user since the ``daml-trigger`` library made
use of this.

This PR adds a few things to improve the situation:

1. We only rename modules that are exposed. This fixes the issue if
   you don’t actually reference a non-exposed module from your
   data-dependency.
2. I’ve removed the exposed-modules from daml-trigger. I don’t think
   they are essential here given that the module name has `Internal`
   in the name and it’s too easy to have something that actually
   references the non-exposed module since the types are reexported.
3. I’ve added documentation that mentions this issue.
4. I’ve added a warning if your exposed-modules are excluding some
   modules. Maybe worth turning this into an error in the future.

changelog_begin
changelog_end

* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
2020-04-01 13:57:52 +02:00
Gerolf Seitz
329320bad9
Organize maven coordinates (#5272)
* Use com.daml as groupId for all artifacts

CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END

* Add 2 additional maven related checks to the release binary

1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId

* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
2020-04-01 11:41:18 +02:00
associahedron
dd8a3a3518
Make new sandbox the default. (#5295)
* Make new sandbox the default.

changelog_begin

- [DAML SDK] The new sandbox is now the default that runs with ``daml sandbox`` and ``daml start``. The command ``daml sandbox-next`` has been removed. The old sandbox can be invoked via ``daml sandbox-classic`` and ``daml start --sandbox-classic=yes``.

changelog_end

* Update descriptions.

* Change it to a switch

* Change switch help

* Recapitalize
2020-03-31 12:50:21 +00:00
Andreas Herrmann
89a9f5c7d2
tarball reproducibility (#5258)
* integration-tests reproducibility

* package-app reproducibility

* Make remaining tar czf reproducible

* package-app

CHANGELOG_BEGIN
CHANGELOG_END

* Reproducibility of remaing tar invocation

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-31 10:09:52 +02:00
Shayne Fletcher
2722e7ce95
Move 'withEnv', call it from daml2ts tests (#5276)
* Move 'withEnv', call it from daml2ts tests

changelog_begin
changelog_end

* Fix withEnv call to ensure that TASTY_NUM_THREADs is set

withEnv replaces the whole environment so we need to set everything we
care about.

* withEnv replaces the whole environment so we need to set everything we
care about.

* Apparently applying the same fix has destabilized Windows

* Try even harder to get daml assistant tests passing on Windows again

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-03-30 17:20:47 +00:00
Moritz Kiefer
a84311ac39
Change withEnv to not clear the whole env (#5281)
The way we use `withEnv` it is really intended to set a few
environment variables in an already existing environment instead of
clearing everything before. This PR changes it to do only that.

changelog_begin
changelog_end

closes #5280
2020-03-30 12:21:25 -04:00
Moritz Kiefer
19496482f6
Replace damlpatterns tarballs by a proper template (#5273)
There is no reason why patterns should be a tarball and the current
tarballs are broken anyway. Why do we even have 2 …

changelog_begin
changelog_end
2020-03-30 12:35:22 +00:00
Moritz Kiefer
a37d898e77
Remove dependency and data-dependency arguments from daml-helper (#5262)
Those were originally used for `damlc migrate`. However, that command
no longer exists and the only call site of `runNew` sets those
arguments to an empty list.

changelog_begin
changelog_end
2020-03-30 10:40:34 +02:00
Moritz Kiefer
1e296aa91f
Whitelist fetch-dar command in assistant telemetry (#5261)
changelog_begin
changelog_end
2020-03-30 10:39:55 +02:00
nickchapman-da
aa53c30de1
New command: daml ledger fetch-dar (#5225)
* New comamnd: daml ledger fetch-dar

`daml ledger fetch-dar [PID] [PATH]`

Download a `Package` and it's dependencies from a ledger, given a root `packageId`, and re-construct a valid `.dar` file. Addresses issue #5037.

The original package names are not reconstructed.

CHANGELOG_BEGIN
CHANGELOG_END

* address some comments

* fix spello

* attempt: recoverPackageName

* recover main package name & version

* Try to fix integration tests on Windows

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-03-27 18:04:14 +00:00
Moritz Kiefer
9a1ddc8fa4
Remove no assistant tests from integration tests (#5240)
There was a brief period of time where `daml build` did not work
outside of the assistant. When we fixed that we added it to the
integration tests since there was no other test suite that used damlc
as a binary (as opposed to using it as a library which runs through
different codepaths). However, in the meantime we have tons of tests
all over the place (e.g. the packaging tests) that call `damlc build`
outside of the assistant so these tests serve no purpose.
Also they are somewhat confusing since the point of the integration
tests is to test an installed SDK whereas these tests do not need an
installed SDK (that’s the whole point of those tests).

changelog_begin
changelog_end
2020-03-27 13:11:10 +01:00
Moritz Kiefer
3a7da97825
Move SDK installation to a tasty resource (#5238)
This ensures that -l and -p work properly in the integration tests
since they no longer depend on the order.

There is lots of other crap to cleanup in those tests but I’m trying
to keep it to small steps.

changelog_begin
changelog_end
2020-03-27 11:59:41 +01:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
Shayne Fletcher
8bb9f4b5c7
Don't write .eslintrc.json files (#5218)
* Factor out daml2ts utilities

changelog_begin
changelog_end

* Stop writing .eslintrc.json files into daml2ts packages

changelog_begin
changelog_end
2020-03-26 14:42:59 -04:00
Andreas Herrmann
9e5dff4109
Ship daml-script and daml-trigger libraries in multiple LF versions (#5192)
* Depend on LF version specific daml-libs

* daml-script.dar build multiple LF versions

CHANGELOG_BEGIN
[DAML Script] The `daml-script` library is now available in multiple LF
  versions, namely 1.7, 1.8, and 1.dev.
CHANGELOG_END

* daml-trigger.dar build multiple LF versions

[DAML Triggers] The `daml-trigger` library is now available in multiple
  LF versions, namely 1.7, 1.8, and 1.dev.

* Keep daml-script.dar available for tests

* Keep daml-trigger.dar available for tests

* daml-libs LF versions integration test

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-26 10:17:19 +01:00
Shayne Fletcher
24cbeddd35
Follow up to earlier PR (#5201)
changelog_begin
changelog_end
2020-03-26 05:16:16 -04:00
Shayne Fletcher
9351b7a7b9
Build js packages at code-generation time (#5171)
changelog_begin
changelog_end
2020-03-25 14:13:37 -04:00
Moritz Kiefer
37dc2f29d3
Integrate create-daml-app into the assistant (#5152)
* Integrate create-daml-app into the assistant

fixes #4868

changelog_begin

- [DAML Assistant] Add a new ``daml create-daml-app`` command for creating a project based on
  `create-daml-app <https://github.com/digital-asset/create-daml-app>`_.

changelog_end

* Try random things hoping to fix windows

* Try random things hoping to fix things on macos
2020-03-24 18:29:40 +01:00
Moritz Kiefer
1942efba23
Allow controlling navigator startup in daml.yaml (#5155)
* Allow controlling navigator startup in daml.yaml

While I’m not entirely convinced the default atm is right, making it
configurable seems like an easier solution than bikeshedding the default.

changelog_begin

- [DAML Assistant] You can now disable starting navigator as part of
  ``daml start`` in your ``daml.yaml`` file by adding ``start-navigator: false``.

changelog_end

* Update docs/source/tools/assistant.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Use --start-navigator=true in the docs

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-03-24 14:04:26 +01:00
Moritz Kiefer
42b4441159
Bump openssl (#5054)
* Bump openssl

The previous one has stopped working for some reason :sadpanda:

changelog_begin
changelog_end

* daml-assistant: Add `--wall-clock-time` to the Sandbox Next test.

Missed this due to doing two things at once.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-03-18 10:24:05 +01:00
Samir Talwar
7510d0d404
daml-assistant: Add daml sandbox-next. (#5045)
* daml-assistant: Add `daml sandbox-next`.

CHANGELOG_BEGIN
- [DAML Assistant] You can now run a pre-release version of Sandbox with
  ``daml sandbox-next`` so you can test it out and verify everything is
  working as expected. Running this will launch Sandbox rebuilt on a
  more modern architecture. An upcoming release of DAML will switch over
  to the new implementation by default.
CHANGELOG_END

* daml-assistant: Explain that sandbox-next is experimental.

Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* daml-assistant: Copy-pasta an integration test for `daml sandbox-next`.

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-03-18 08:25:08 +00:00
Samir Talwar
cbeeb5aafc
sandbox: Fail to start if a time mode is not explicitly specified. (#5033)
* sandbox: Fail to start if a time mode is not explicitly specified.

CHANGELOG_BEGIN
- [Sandbox] Sandbox is switching from Static Time mode to Wall Clock
  Time mode as the default. To ensure that our users know about this,
  for one version, there will be no default time mode. Instead, users
  will have to explicitly select their preferred time mode by means of
  the `--static-time` or `--wall-clock-time` switches. In the next
  release, Wall Clock Time will become the default, and users who are
  happy with the defaults will no longer need to specify the time mode.
CHANGELOG_END

* daml-script|triggers: Specify time mode when testing against Sandbox.

* daml-assistant: Default the Sandbox to wall clock time.

CHANGELOG_BEGIN
- [DAML Assistant] Initializing a new DAML project adds a switch to
  ``daml.yaml`` to ensure Sandbox can continue to start with ``daml
  start``::

      sandbox-options:
        - --wall-clock-time
CHANGELOG_END

* docs: Update the DAML Script and Triggers docs to use Wall Clock time.

It's now what Sandbox will use by default when using `daml init`.

* docs: Change the Quickstart to run Sandbox in wall clock time.

This explains why the contract IDs may vary.

It also updates the manual release testing script to match.
2020-03-18 08:25:03 +00:00
Stefano Baghino
1f0534ea4f
Spin-off BuildInfo into its own (micro-)library (#5004)
* Spin-off BuildInfo into its own (micro-)library

CHANGELOG_BEGIN
CHANGELOG_END

* Fix dependencies

* Remove unused dependency
2020-03-16 09:04:00 +01:00
Moritz Kiefer
4e99f18613
Introduce a DA.Test.Sandbox module for managing sandbox in tasty (#4986)
We previously had 3 slightly different but consistently shitty logic
for handling this in the tests for daml-helper daml repl and the
Haskell ledger bindings. This PR introduces a module that is flexible
enough to capture all their needs and hopefully is somewhat less
shitty.

changelog_begin
changelog_end
2020-03-13 15:35:03 +01:00
Moritz Kiefer
b3a5c3b28d
Share test certificates (#4982)
* Share test certificates

This is primarily an attempt at making sure my contribution stats
remain negative but I think it’s a nice cleanup. The only difference
in the certs used by daml-helper which are now used everywhere is that
they use a different CN for the CA and the server. This is required to
make openssl happy (which is used by the daml-helper).

changelog_begin
changelog_end

* Fix script and trigger tests
2020-03-13 12:12:34 +01:00
Moritz Kiefer
96f62418c5
Support TLS in DAML script and DAML triggers (#4971)
This adds CLI parametrs for connecting via TLS following the scheme
used by navigator, extractor and `daml ledger`.

changelog_begin

- [DAML Script] Support TLS. Enable it by passing ``--tls``. You can
  set certificates for client authentication via ``--pem`` and
  ``-crt`` and a custom root CA for validating the server certificate
  via ``--cacrt``.

- [DAML Triggers - Experimental] Support TLS. Enable it by passing ``--tls``. You can
  set certificates for client authentication via ``--pem`` and
  ``-crt`` and a custom root CA for validating the server certificate
  via ``--cacrt``.

changelog_end
2020-03-13 09:54:44 +01:00
Moritz Kiefer
edd73384c4
Make client authentication in Sandbox configurable (#4965)
Currently sandbox only supports TLS if you also enable client
authentication. There is no reason for why this has to be the case and
for things like DABL we want TLS without client authentication so it’s
useful to be able to test this in sandbox. This PR introduces a
`--client-auth` flag that allows you to configure the behavior. The
default is the current one of requiring client authentication.

This PR does not yet update Java clients, however, the Haskell client
supports this already and is used to test this functionality.

I’ve also added a section in the documentation on TLS (there were no
docs at all so far).

changelog_begin

- [DAML Sandbox] When Sandbox is run with TLS enabled, you can now
  configure the requirement for client authentication via
  ``--client-auth``. See
  https://docs.daml.com/tools/sandbox.html#running-with-tls for more information.

changelog_end
2020-03-12 16:04:46 +01:00
Moritz Kiefer
e0d652df91
Generate ssl certs in a genrule instead of checking them in (#4950)
changelog_begin
changelog_end

For now this is only used for the daml-helper tests. I’ll shuffle
things around and use it for all tests in a separate PR.
2020-03-12 10:36:40 +01:00
Moritz Kiefer
ed5886051d
Support TLS in daml-helper (#4943)
This PR adds TLS support to DAML helper both via client certs and
without (although the latter is not tested so far since atm this is
not supported by sandbox). The CLI options follow the scheme used by
navigator/extractor/… with the addition that you can just pass `--tls`
which will turn on TLS without custom root certs or client certs.

changelog_begin

- [DAML Assistant] You can now connect to ledger via TLS for ``daml
  deploy`` and ``daml ledger`` commands. See
  https://docs.daml.com/deploy/generic_ledger.html for more information.

changelog_end
2020-03-11 17:58:05 +01:00
Andreas Herrmann
f3f39a23a3
daml-assistant tests: fix flaky test on Windows (#4945)
* daml-assistant tests: fix flaky test on Windows

```
bazel test //daml-assistant:test --test_arg=--quickcheck-replay=425714
```
failed on Windows in the test-case:
```
tail . ascendents == ascendents . takeDirectory
```
In that case the two path components given two the test case shrink to
`p1 = "a"` and `p2 = "\\"`. Confusingly, on Windows
```
isRelative "\\" == True
```
while
```
"a" </> "\\" = "\\"
```
This is documented behaviour on Windows, see [1] and [2].

Using `p1 </> p2 /= p2` instead of `isRelative p2` works around this.

[1]: https://hackage.haskell.org/package/filepath-1.4.2.1/docs/System-FilePath-Posix.html#v:-60--47--62-
[2]: https://hackage.haskell.org/package/filepath-1.4.2.1/docs/System-FilePath-Posix.html#v:isRelative

* Fix ascendants test group name

`ascendants` is defined in `DA.Daml.Project.Util`, not
`DA.Daml.Assistant`.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-11 16:04:31 +00:00
Moritz Kiefer
d381603475
Detect whether sandbox has started using --port-file (#4880)
On MacOS using Java 13 (at least that was the only combination where
I’ve managed to reproduce this). Sandbox spits out errors of the
following form in ``daml start``

```
java.net.SocketException: Connection reset
```

This appears to be caused by the TCP requests that we send to detect
whether sandbox has started. Switching to using the ``--port-file``
option makes this issue go away.

fixes #4670

changelog_begin

- [DAML Assistant] In certain configurations ``daml start`` produced
  a (harmless) ``Connection reset`` log message. This has now been
  fixed.

changelog_end
2020-03-06 16:18:43 +00:00
Moritz Kiefer
d8afe7a4cc
Support specifying sandbox/navigator/json-api/script options in daml.yaml (#4877)
changelog_begin

- [DAML Assistant] You can now specify options for
  Sandbox/Navigator/the HTTP JSON API in ``daml.yaml`` via
  ``sandbox-options``/``navigator-options``/``json-api-options``. These
  options will be picked up by running ``daml start``. This is
  particularly useful for specifying ``--wall-clock-time`` and for
  specifying a fixed ledger ID during development.

changelog_end

fixes #2993
2020-03-06 14:52:45 +01:00
Moritz Kiefer
0a6be2b341
Overload submit so that it doesn’t collide with DAML script (#4831)
This introduces a `HasSubmit` typeclass (following the naming scheme
of `HasCreate`, …) and instances for `Scenario` and `Script`. This
avoids the need to hide `submit` in every single DAML script.

changelog_begin

- [DAML Standard Library] ``submit`` and ``submitMustFail`` are now
  overloaded so that they can be used in both scenarios and DAML script.

changelog_end
2020-03-05 15:43:35 +01:00
mergify[bot]
cb1395e923
Remove damlc migrate (#4816)
* Remove damlc migrate

``damlc migrate`` hasn’t worked for quite a while and we emitted a
warning for months so given that we don’t have plans to make it work
again in the near future, I think it does more harm than good to keep
it around.

changelog_begin

- [DAML Compiler] After being deprecated for a while the ``damlc
  migrate`` command has now been removed. See
  https://docs.daml.com/upgrade/ for up to date documentation
  on model upgrades.

changelog_end

fixes #3704 (by removing the tests 😇)

* yeah the windows cache is once again broken \o/

* Revert "yeah the windows cache is once again broken \o/"

This reverts commit 38d7877aa4.
2020-03-04 20:36:48 +00:00
Shayne Fletcher
48595f774d
daml2ts : Generate a package of each DALF (#4799)
* daml2ts : generate one package per dalf
2020-03-04 14:51:40 -05:00
nickchapman-da
47fa9eb0ca
Add validate-dar to the assistant whitelist. (#4766)
changelog_begin
changelog_end
2020-03-02 11:57:24 +00:00
Andreas Herrmann
2dbb5282ca
Add init-script field to daml.yaml (#4685)
* Add init-script daml.yaml field

CHANGELOG_BEGIN
[DAML Script -- Experimental] You can now define an ``init-script`` in
  the ``daml.yaml`` file. If present, that DAML script will be executed
  to initialize the sandbox on ``daml start``. This can be used instead
  of the ``scenario`` field.
CHANGELOG_END

* Add integration test for init-script

* Generate JWT token in tests

Addressing review comment
https://github.com/digital-asset/daml/pull/4685#discussion_r383835050

* Remove unnecessary daml calls

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-02-26 09:30:09 +00:00
Shayne Fletcher
b1a3be5950
copyDirectory, mvDirectory - daml-project-config -> da-hs-base (#4697)
changelog_begin
changelog_end
2020-02-25 14:57:06 -05:00
Gary Verhaegen
5a117dc358
introduce new release process (#4513)
Context
=======

After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.

This PR is a proposal for a new release process designed under the
following constraints:

- Reuse as much as possible of the existing infrastructure, to minimize
  effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
  not meant for general public consumption, but can still be used by savvy
  users without jumping through too many extra hoops (ideally just
  swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
  release status, with as few changes as possible in-between, so we can be
  confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
  releases, to avoid discovering non-transient problems while trying to
  promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
  keep the same approval process for SOC2 auditability.

The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:

```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```

This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.

Version numbers
===============

Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:

- Semantic versioning (which we chose as the version format for the SDK
  version number) allows for "prerelease" version numbers as well as
  "metadata"; an example of a complete version string would be
  `1.2.3-nightly.201+server12.43`. The "main" part of the version string
  always has to have 3 numbers separated by dots; the "prerelease"
  (after the `-` but before the `+`) and the "metadata" (after the `+`)
  parts are optional and, if present, must consist of one or more segments
  separated by dots, where a segment can be either a number or an
  alphanumeric string. In terms of ordering, metadata is irrelevant and
  any version with a prerelease string is before the corresponding "main"
  version string alone. Amongst prereleases, segments are compared in
  order with purely numeric ones compared as numbers and mixed ones
  compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
  which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
  `-`, or a transition between a number and a letter. Version strings
  are compared element-wise, with numeric segments being compared as
  numbers. Alphabetic segments are treated specially if they happen to be
  one of a handful of magic words (such as "alpha", "beta" or "snapshot"
  for example) which count as "qualifiers"; a version string with a
  qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
  which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
  special ordering amongst qualifiers. Other alphabetic segments are
  compared alphabetically and count as being "after" their prefix
  (`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
  separated by `.`, plus an optional (though deprecated) alphanumeric
  "version tag" separated by a `-`. I could not find any official
  documentation on ordering for the version tag; numeric segments are
  compared as numbers.
- npm uses semantic versioning so that is covered already.

After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:

```
0.13.53-alpha.20200214.3025.ef5d32b7
```

where the components are, respectively:

- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
  master.
- `3025`: the number of commits in master up to the release commit
  (included). Because we have a linear, append-only master branch, this
  uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
  not strictly speaking necessary, but makes it a lot more convenient to
  identify the commit.

The main downsides of this format are:

1. It is not a valid format for GHC packages. We do not publish GHC
  packages from the SDK (so far we have instead opted to release our
  Haskell code as separate packages entirely), so this should not be an
  issue. However, our SDK version currently leaks to `ghc-pkg` as the
  version string for the stdlib (and prim) packages. This PR addresses
  that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
  would see the above version number as `0.13.53.20200214.3025`, which
  should be enough to uniquely identify it. Note that, as far as I could
  find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
  it makes it more scary. However, I have been told that this may be
  long enough to cause issues on Windows by pushing us past the max path
  size limitation of that "OS". I suggest we try it and see what
  happens.

The upsides are:

- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
  they have the same date, allowing us to release same-day patches if
  needed. (Note: that is, commits that were made on the same day; the
  release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
  immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
  GHC).

Alternatives I considered:

- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
  be accepted by all schemes, but will not sort as expected under semantic
  versioning (though Maven will be fine). I have no idea how it will sort
  under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
  This is not valid semantic versioning and is therefore rejected by
  npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
  what is generally done in the Java world, but we then lose track of what
  version is actually in use and I'm concerned about bug reports. This
  would also not let us publish to the main Maven repo (at least not more
  than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
  version formats. However, it would not clearly indicate to humans that
  it is not meant as a stable version, and would sort differently under
  semantic versioning (which counts it as a prerelease, i.e. before
  `0.13.53`) than under maven (which counts it as a patch, so after
  `0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
  number of prereleases in-between `0.13.52` and the next. This is
  currently the fallback plan if Windows path length causes issues. It
  would be less convenient to map releases to commits, but it could still
  be done via querying the history of the `LATEST` file.

Release notes
=============

> Note: We have decided not to have release notes for snapshot releases.

Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.

But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.

The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.

> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.

New release process
===================

Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.

Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:

1. Before running the release step, CI will checkout the commit
  specified in the LATEST file. This requires separating the release
  step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
  that gets the version to build from an environment variable, with a
  default of `0.0.0` to remain consistent with the current `daml-head`
  behaviour.

Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.

The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.

I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 17:01:23 +01:00
associahedron
d9cb5099de
A first draft at telemetry for daml-assistant. (#4506)
* Report assistant commands and errors via a logger.

CHANGELOG_BEGIN
CHANGELOG_END

* Hook up daml-assistant to a GCP logger.

* fix test case

* fix more tests

* Check opted in status in assistant.

* Anonimize args that have unexpected characters.

* More agressive anonimization

* add missing containers dependency
2020-02-14 15:46:25 +00:00
Moritz Kiefer
56db27e50e
Use the new token format in daml-helper (#4486)
We were still using the old token format without the
https://daml.com/ledger-api key. This resulted in a warning on every
call to `daml start` which looks rather ugly. I’ve changed it to
construct the token using the JWT library instead of hardcoding it
which should make it easier to modify in the future

changelog_begin
changelog_end
2020-02-12 12:53:42 +01:00
Leonid Shlyapnikov
fb6d8afd1f
Rename JSON API endpoints, add version prefix (#4440)
* Rename JSON API endpoints, add version prefix. Combining two breaking changes.

CHANGELOG_BEGIN

[JSON API - Experimental] Rename JSON API endpoints. See #4289 and #3145
    /command/create => /v1/create
    /command/exercise => /v1/exercise
    /contracts/lookup => /v1/fetch
    /contracts/search => /v1/query
    /contracts/searchForever => /v1/stream/query
    /contracts/lookupForever => /v1/stream/fetch
    /parties => /v1/parties

CHANGELOG_END

* minor update

* fix typo in the logging

* experimental disclosure update

* experimental disclosure update

* experimental disclosure update
2020-02-06 16:30:47 -05:00
Moritz Kiefer
fc87953ed4
Enhanced completions for the assistant (#4420)
* Enhanced completions for the assistant

changelog_begin

- [DAML Assistant] The assistant can now do completions for SDK
  commands, e.g., ``daml ledger upl<TAB>`` will complete to ``daml
  ledger upload-dar``.

changelog_end

* Apply suggestions from code review

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
2020-02-06 15:16:12 +01:00
Moritz Kiefer
9af40f55fd
Only shut down when stdin is closed when a flag is passed (#4419)
It turns out that various environments (hello docker) start with a
closed stdin. This means that things like `daml sandbox` exit
immediately. Since users that care about the new behavior are likely
to know about it, we only enable it if a specific flag is given.

changelog_begin

- [DAML Assistant] The new behavior introduced in ``0.13.51`` to shut
  down when stdin is closed is now disabled unless you explicitly
  enable it by passing ``--shutdown-stdin-close``.

changelog_end
2020-02-06 11:31:49 +01:00
Moritz Kiefer
0d07210a3f
Fall back to regular bash completions for command arguments (#4375)
* Fall back to regular bash completions for command arguments

This addresses the first part of #4369 by falling back to the bash completions.

I sadly still haven’t figured out how we can stop to completing to a
directory if the command also exists as a directory.

changelog_begin

- [DAML Assistant] Bash and Zsh completions will now fall back to
  regular file completions after the command argument.

changelog_end

* Shut up hlint
2020-02-04 15:45:53 +01:00
Moritz Kiefer
5a1a2301c6
Shut down when stdin is closed. (#4365)
* Shut down when stdin is closed.

changelog_begin

- [DAML Assistant] The DAML assistant will now shut down long-running
  processes like sandbox when stdin is
  closed. This is mainly useful on Windows, where process APIs often
  kill the process in a way that does not allow it to do any cleanup, in
  particular, we cannot stop child processes.

changelog_end

For now, the logic for this is only in daml-helper which is the only
thing starting long-running processes (in particular sandbox). There
is a long inline-comment explaining why this is not on the assistant
itself.

fixes #4168
2020-02-04 14:34:34 +01:00
Shayne Fletcher
0e53ec0a2b Integrate daml2ts into the assistant (#4252)
changelog_begin
- Generate TypeScript bindings to DAML with `daml codegen ts DAR -o OUTDIR`
changelog_end
2020-01-28 22:03:54 +00:00
Moritz Kiefer
20804a4e6b
Add a basic integration test for navigator (#4174)
This test just checks that navigator starts and we get a 200
response. This is obviously not a very comprehensive test but it would
have caught the issues that prevented 0.13.44 an 0.13.45 from getting
published so it seems like a good idea to test this.

changelog_begin
changelog_end
2020-01-23 11:14:18 +01:00
Moritz Kiefer
d1449e78c7
Add Zsh completions for the assistant (#3980)
* Add Zsh completions for the assistant

This is slightly more annoying for users since at least on Linux there
doesn’t seem to be a user-writable directory that is in `$fpath` by
default. I think on Zsh we could probably write them to
/usr/local/share/zsh/site-functions but I’d rather avoid platform
specific logic here. I would expect that Zsh users are usually
somewhat comfortable with modifying the config and this also matches
other installation instructions, e.g.,
https://github.com/zsh-users/zsh-completions#manual-installation.

On the plus side, the completions look significantly nicer in Zsh
since they include the description of commands.

CHANGELOG_BEGIN
- [DAML Assistant] Zsh completions for the DAML Assistant are now
  installed as part of ``daml install``. To activate them you need to
  add ``~/.daml/zsh`` to your ``$fpath``, e.g., by adding
  ``fpath=(~/.daml/zsh $fpath)`` to the beginning of your ``~/.zshrc``
  before you call ``compinit``.
CHANGELOG_END

* Fix tests

* Update daml-assistant/src/DA/Daml/Assistant/Install/Completion.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
2020-01-08 14:58:13 +01:00
Martin Huschenbett
a043b09375
Add akka slf4j library as dependency of super fat daml-sdk.jar (#3965) 2020-01-07 17:19:42 +01:00