Commit Graph

114 Commits

Author SHA1 Message Date
Remy
15354c3256
DAML-LF: remove submitter is in maintainer check (#5611)
CHANGELOG_BEGIN
CHANGELOG_END
2020-04-23 16:10:39 +02:00
Remy
7bdde70840
Speedy Compiler: clean up. (#5649)
CHANGELOG_BEGIN
CHANGELOG_END
2020-04-22 16:09:54 +02:00
Moritz Kiefer
d8db88e7e1
Make DAML Triggers and DAML Script default to wall-clock-time (#5632)
* Make DAML Triggers and DAML Script default to wall-clock-time

Now that sandbox defaults to wall-clock-time there is no reason why we
should not default in DAML triggers and DAML Script.

changelog_begin

- [DAML Triggers] ``daml trigger`` now defaults to wall clock time if
  neither ``--wall-clock-time`` or ``--static-time`` is passed.

- [DAML Script] ``daml script`` now defaults to wall clock time if
  neither ``--wall-clock-time`` or ``--static-time`` is passed.

changelog_end

* Make --static-time and --wall-clock-time exclusive
2020-04-20 14:42:19 +00:00
Remy
9704a39bae
Engine: refactor PartialTransaction context (#5578)
* Engine: refactor PartialTransaction context

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-16 14:39:12 +02:00
Remy
d91da3be51
Engine: refactoring preprocessing. (#5434)
* Engine: refactor preprocessing

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-09 20:29:03 +02:00
Moritz Kiefer
3bbd7719bb
Move remaining trigger tests over to scalatest (#5494)
This PR ports the remaining trigger tests to scalatest and kills the
whole client_server_test based infrastructure 🎉

changelog_begin
changelog_end
2020-04-08 17:28:34 +02:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Moritz Kiefer
b17095e5cc
Move more trigger tests to scala-test (#5384)
* Move more trigger tests to scala tests

This PR moves more tests of triggers over to the Scala test suite, in
particular:

- The existing tests there abstract over the time mode and are
  instantiated once for wallclock mode and once for static time mode.
- I’ve added tests for TLS and Auth.
- I’ve removed the TLS and Auth tests outside of scala-test since they
  are now redundant.
- I’ve added the time tests to the scala-tests stuff since with the
  new ledger time model, that’s necessary to actually trigger a
  failure if you get static time vs wallclock time wrong (MRT and LET
  no longer exist).

I haven’t yet moved all the func tests over, I’ll do that separately
and then we can kill the old tests completely.

changelog_begin
changelog_end

* Factor out test utils into a library
2020-04-03 09:58:58 +00:00
Moritz Kiefer
29ed16b4cc
Improve handling of exposed-modules with data-dependencies (#5330)
* Improve handling of exposed-modules with data-dependencies

Previously, we tried to rename all modules of a dependency via
--package. This fails if some of those modules are not exported. This
was trivial to hit as a user since the ``daml-trigger`` library made
use of this.

This PR adds a few things to improve the situation:

1. We only rename modules that are exposed. This fixes the issue if
   you don’t actually reference a non-exposed module from your
   data-dependency.
2. I’ve removed the exposed-modules from daml-trigger. I don’t think
   they are essential here given that the module name has `Internal`
   in the name and it’s too easy to have something that actually
   references the non-exposed module since the types are reexported.
3. I’ve added documentation that mentions this issue.
4. I’ve added a warning if your exposed-modules are excluding some
   modules. Maybe worth turning this into an error in the future.

changelog_begin
changelog_end

* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
2020-04-01 13:57:52 +02:00
Gerolf Seitz
329320bad9
Organize maven coordinates (#5272)
* Use com.daml as groupId for all artifacts

CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END

* Add 2 additional maven related checks to the release binary

1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId

* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
2020-04-01 11:41:18 +02:00
Moritz Kiefer
da5daf3239
Turn script over JSON API tests into a proper test suite (#5278)
This replaces the rather horrible previous setup of having a custom
test runner that spawns 3 separate JVM processes by a single scalatest
test suite that starts sandbox and the JSON API in process.

changelog_begin
changelog_end
2020-03-31 11:55:29 +02:00
Moritz Kiefer
d70fa63a21
Add a proper scalatest test suite for triggers (#5250)
This PR adds a new test suite for DAML triggers based on scala test
rather than the client_server_test macro + a custom main. This seems
much nicer than the client_server_test (we get a lot of useful stuff
from scalatest, e.g., useful output of assertion failures, things
don’t blow up after the first test failure, …).

This PR only ports over a small fraction of the tests to make review
easier. The plan is then to port over everything and kill off the
existing test stuff once everything is ported over.

changelog_begin
changelog_end
2020-03-30 11:42:09 +02:00
Remy
1b37f6c482
DAML-LF: redesign absolute contract ids (#5207)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 19:07:42 +01:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
Andreas Herrmann
9e5dff4109
Ship daml-script and daml-trigger libraries in multiple LF versions (#5192)
* Depend on LF version specific daml-libs

* daml-script.dar build multiple LF versions

CHANGELOG_BEGIN
[DAML Script] The `daml-script` library is now available in multiple LF
  versions, namely 1.7, 1.8, and 1.dev.
CHANGELOG_END

* daml-trigger.dar build multiple LF versions

[DAML Triggers] The `daml-trigger` library is now available in multiple
  LF versions, namely 1.7, 1.8, and 1.dev.

* Keep daml-script.dar available for tests

* Keep daml-trigger.dar available for tests

* daml-libs LF versions integration test

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-26 10:17:19 +01:00
Robert Autenrieth
a3ddde3a9d
Remove MRT and LET (#5172)
Contributes to #4194.
Closes #4231.
Closes #5022.
CHANGELOG_BEGIN
- [Ledger API] The protobuf fields ledger_effective_time and maximum_record_time have been removed from
  command submission. These fields were previously deprecated following the introduction
  of a new ledger time model. See issue `#4194 <https://github.com/digital-asset/daml/issues/4194>`__.
[Java Bindings] removed the usage of ledgerEffectiveTime and
maximumRecordTime, and instead added minLedgerTimeAbsolute and
minLedgerTimeRelative in CommandSubmissionClient and CommandClient
CHANGELOG_END
2020-03-25 16:50:27 +01:00
Robert Autenrieth
3f597aae16
New ledger time (#5100)
* Tighten result type

Command execution can't result in a sequencer error

* New helper method for extracting used contracts

* New error clause

* Add a DAO query for the maximum time of contracts

* Implement algorithm for finding ledger time

CHANGELOG_BEGIN
CHANGELOG_END

* fixup ledgerTimeHelper

* Use new ledger time algorithm

* Mark LET/MRT as deprecated

CHANGELOG_BEGIN
- [Ledger API] DAML ledgers have switched to a new ledger time model.
  The ledger_effective_time and maximum_record_time fields of command submission are deprecated,
  the ledger time of transactions is instead set automatically by the ledger API server.
  Ledger time is no longer strictly monotonically increasing, but only follows causal monotonicity:
  ledger time of transactions is greater than or equal to the ledger time of any used contract.
  See `#4345 <https://github.com/digital-asset/daml/issues/4345>`__.
CHANGELOG_END

* Add ledger time skew check

* Remove command updater

LET/MRT are now deprecated, this class is now useless

* Remove old time model validator

* Switch to new time model check: kvutils

* Switch to new time model check: in-memory ledger

* Switch to new time model check: SqlLedger

* Use initial ledger config

* Ignore user provided LET

* Use TimeProvider in submission services

* Use deduplication_time in daml-script runner

- Also remove unnecessary command completion output of CommandTracker.
- Remove usage of maximum record time in CommandTracker.

* Use arbitrary default value for deduplication time

* Use built-in Instant ordering

* Remove obsolete test

* Remove obsolete test: CommandStaticTimeIT

* Refactor test: TransactionMRTCompliance

* Disable test: CommandTrackerFlow timeout

* thread maxDeduplicationTime through to CommandTracker

* Improve test

* Refactor command client configuration

* Deduplication time should always use UTC

* Add missing method in TimedIndexService after rebase

* Put more details into the deduplication error response.

* Use system time for command dedup submittedAt.

* Use explicit UTC time source in command validator

* Revert CommandTracker[Flow] to previous completion-recovering-behavior

* Adapt scala client command config to new config params

Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-03-25 09:28:56 +01:00
Moritz Kiefer
a50b7b7dbd
Hide "Slf4jlogger started" log statement (#5163)
This just looks ugly and doesn’t help users at all.

changelog_begin
changelog_end
2020-03-24 17:06:14 +01:00
Moritz Kiefer
13816c75b3
Support uploading DARs to the trigger service (#5084)
* Support uploading DARs to the trigger service

This PR adds a new `upload_dar` endpoint that accepts a DAR as a
multi-part form request and adds it to the list of compiled packages.

I’ve also made the DAR passed in on startup optional now given the new
endpoint.

There is no endpoint for deleting a DAR so far but there is none on
the ledger API either so I think this not particularly urgent.

changelog_begin
changelog_end

* Address review comments
2020-03-19 13:02:04 +01:00
Remy
c2070170b3
Engine: add rank to Enum and Variant SValue (#5048)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 14:37:44 +01:00
Moritz Kiefer
bd6db5be68
Handle failures during trigger initialization synchronously (#5046)
Previously the http endpoint for starting a trigger would always
return immediately. Based on the recent refactorings, we now do the
non-IO trigger initialization synchronously and return a failed http
status code with an error message.

This also refactors the code to only have one (mutable) set of
compiled packages which is a prerequisite for dynamic package uploads.

changelog_begin
changelog_end
2020-03-18 10:23:41 +01:00
Samir Talwar
cbeeb5aafc
sandbox: Fail to start if a time mode is not explicitly specified. (#5033)
* sandbox: Fail to start if a time mode is not explicitly specified.

CHANGELOG_BEGIN
- [Sandbox] Sandbox is switching from Static Time mode to Wall Clock
  Time mode as the default. To ensure that our users know about this,
  for one version, there will be no default time mode. Instead, users
  will have to explicitly select their preferred time mode by means of
  the `--static-time` or `--wall-clock-time` switches. In the next
  release, Wall Clock Time will become the default, and users who are
  happy with the defaults will no longer need to specify the time mode.
CHANGELOG_END

* daml-script|triggers: Specify time mode when testing against Sandbox.

* daml-assistant: Default the Sandbox to wall clock time.

CHANGELOG_BEGIN
- [DAML Assistant] Initializing a new DAML project adds a switch to
  ``daml.yaml`` to ensure Sandbox can continue to start with ``daml
  start``::

      sandbox-options:
        - --wall-clock-time
CHANGELOG_END

* docs: Update the DAML Script and Triggers docs to use Wall Clock time.

It's now what Sandbox will use by default when using `daml init`.

* docs: Change the Quickstart to run Sandbox in wall clock time.

This explains why the contract IDs may vary.

It also updates the manual release testing script to match.
2020-03-18 08:25:03 +00:00
Moritz Kiefer
fb86152f8c
Move non-IO initialization out of trigger runner (#5042)
Previously parts of the initialization, in particular, the code for
finding the filter and the heartbeat were part of the Runner. This led
to an akward API and didn’t really make any sense.

Now all of this code is part of a pure `Trigger.fromIdentifier`
method and the runner only takes care of actually running the
ledger. This could also be useful for the trigger service where we
might want to synchronously call `getIdentifier` so users get some
indication if there request even points to a valid trigger
directly. However, this is not tackled by this PR.

changelog_begin
changelog_end
2020-03-17 15:26:58 +01:00
Moritz Kiefer
cc2ca20336
Make trigger runner specific to a given trigger (#5038)
Previously the runner class was in a weird state where it was specific
to a DAR but not to an individual trigger. This meant that you had to
pass around a fair bit of state which got a bit awkward. This PR
addresses this by making the trigger class specific to the trigger.

It also now accepts `CompiledPackages` instead of a DAR which should
make it easier in the trigger service to support dynamic package
uploads.

changelog_begin
changelog_end
2020-03-17 11:51:06 +01:00
Moritz Kiefer
9aa68cac4f
Fix detection of package ids for triggers (#5016)
Previously we assumed that the module name was globally unique in the
DAR which is definitely not guaranteed. Now we instead detect the
package id of the trigger library based on the type of the trigger we
are running which doesn’t fall apart if there are multiple versions of
the trigger library.

I’ve also removed the check for the package id of the trigger library
since I’d like the trigger runner to be backwarts compatible from now on (we
didn’t break that in a while).

This is slightly ugly since the Runner class is currently not specific
to a single trigger but only the individual methods are aware of the
specific trigger identifier. I’ll refactor this in a separate PR.

changelog_begin
changelog_end
2020-03-16 16:13:42 +01:00
Moritz Kiefer
b3a5c3b28d
Share test certificates (#4982)
* Share test certificates

This is primarily an attempt at making sure my contribution stats
remain negative but I think it’s a nice cleanup. The only difference
in the certs used by daml-helper which are now used everywhere is that
they use a different CN for the CA and the server. This is required to
make openssl happy (which is used by the daml-helper).

changelog_begin
changelog_end

* Fix script and trigger tests
2020-03-13 12:12:34 +01:00
Moritz Kiefer
96f62418c5
Support TLS in DAML script and DAML triggers (#4971)
This adds CLI parametrs for connecting via TLS following the scheme
used by navigator, extractor and `daml ledger`.

changelog_begin

- [DAML Script] Support TLS. Enable it by passing ``--tls``. You can
  set certificates for client authentication via ``--pem`` and
  ``-crt`` and a custom root CA for validating the server certificate
  via ``--cacrt``.

- [DAML Triggers - Experimental] Support TLS. Enable it by passing ``--tls``. You can
  set certificates for client authentication via ``--pem`` and
  ``-crt`` and a custom root CA for validating the server certificate
  via ``--cacrt``.

changelog_end
2020-03-13 09:54:44 +01:00
Samir Talwar
a99156252d
libs-scala/ports: Wrap socket ports in a type, Port. (#4784)
* libs-scala/ports: Wrap socket ports in a type, `Port`.

* sandbox: Use `Port` for the API server port, and propagate.

CHANGELOG_BEGIN
CHANGELOG_END

* extractor: Use `Port` for the server port.

* ports: Make Port a compile-time class only.

* ports: Allow port 0; it can be specified by a user.

* ports: Publish to Maven Central.
2020-03-03 08:59:15 +00:00
Shayne Fletcher
66dd112960
Remove pragma 'daml 1.2' (#4702)
changelog_begin
- The pragma 'daml 1.2' is now optional.
changelog_end
2020-02-26 13:17:45 -05:00
Gary Verhaegen
5a117dc358
introduce new release process (#4513)
Context
=======

After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.

This PR is a proposal for a new release process designed under the
following constraints:

- Reuse as much as possible of the existing infrastructure, to minimize
  effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
  not meant for general public consumption, but can still be used by savvy
  users without jumping through too many extra hoops (ideally just
  swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
  release status, with as few changes as possible in-between, so we can be
  confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
  releases, to avoid discovering non-transient problems while trying to
  promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
  keep the same approval process for SOC2 auditability.

The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:

```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```

This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.

Version numbers
===============

Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:

- Semantic versioning (which we chose as the version format for the SDK
  version number) allows for "prerelease" version numbers as well as
  "metadata"; an example of a complete version string would be
  `1.2.3-nightly.201+server12.43`. The "main" part of the version string
  always has to have 3 numbers separated by dots; the "prerelease"
  (after the `-` but before the `+`) and the "metadata" (after the `+`)
  parts are optional and, if present, must consist of one or more segments
  separated by dots, where a segment can be either a number or an
  alphanumeric string. In terms of ordering, metadata is irrelevant and
  any version with a prerelease string is before the corresponding "main"
  version string alone. Amongst prereleases, segments are compared in
  order with purely numeric ones compared as numbers and mixed ones
  compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
  which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
  `-`, or a transition between a number and a letter. Version strings
  are compared element-wise, with numeric segments being compared as
  numbers. Alphabetic segments are treated specially if they happen to be
  one of a handful of magic words (such as "alpha", "beta" or "snapshot"
  for example) which count as "qualifiers"; a version string with a
  qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
  which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
  special ordering amongst qualifiers. Other alphabetic segments are
  compared alphabetically and count as being "after" their prefix
  (`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
  separated by `.`, plus an optional (though deprecated) alphanumeric
  "version tag" separated by a `-`. I could not find any official
  documentation on ordering for the version tag; numeric segments are
  compared as numbers.
- npm uses semantic versioning so that is covered already.

After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:

```
0.13.53-alpha.20200214.3025.ef5d32b7
```

where the components are, respectively:

- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
  master.
- `3025`: the number of commits in master up to the release commit
  (included). Because we have a linear, append-only master branch, this
  uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
  not strictly speaking necessary, but makes it a lot more convenient to
  identify the commit.

The main downsides of this format are:

1. It is not a valid format for GHC packages. We do not publish GHC
  packages from the SDK (so far we have instead opted to release our
  Haskell code as separate packages entirely), so this should not be an
  issue. However, our SDK version currently leaks to `ghc-pkg` as the
  version string for the stdlib (and prim) packages. This PR addresses
  that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
  would see the above version number as `0.13.53.20200214.3025`, which
  should be enough to uniquely identify it. Note that, as far as I could
  find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
  it makes it more scary. However, I have been told that this may be
  long enough to cause issues on Windows by pushing us past the max path
  size limitation of that "OS". I suggest we try it and see what
  happens.

The upsides are:

- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
  they have the same date, allowing us to release same-day patches if
  needed. (Note: that is, commits that were made on the same day; the
  release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
  immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
  GHC).

Alternatives I considered:

- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
  be accepted by all schemes, but will not sort as expected under semantic
  versioning (though Maven will be fine). I have no idea how it will sort
  under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
  This is not valid semantic versioning and is therefore rejected by
  npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
  what is generally done in the Java world, but we then lose track of what
  version is actually in use and I'm concerned about bug reports. This
  would also not let us publish to the main Maven repo (at least not more
  than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
  version formats. However, it would not clearly indicate to humans that
  it is not meant as a stable version, and would sort differently under
  semantic versioning (which counts it as a prerelease, i.e. before
  `0.13.53`) than under maven (which counts it as a patch, so after
  `0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
  number of prereleases in-between `0.13.52` and the next. This is
  currently the fallback plan if Windows path length causes issues. It
  would be less convenient to map releases to commits, but it could still
  be done via querying the history of the `LATEST` file.

Release notes
=============

> Note: We have decided not to have release notes for snapshot releases.

Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.

But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.

The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.

> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.

New release process
===================

Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.

Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:

1. Before running the release step, CI will checkout the commit
  specified in the LATEST file. This requires separating the release
  step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
  that gets the version to build from an environment variable, with a
  default of `0.0.0` to remain consistent with the current `daml-head`
  behaviour.

Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.

The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.

I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 17:01:23 +01:00
Samir Talwar
f231ebc1b3
Sandbox-Next: Re-establish static time support. (#4581)
* kvutils: Extract a committer from the uses of `SubmissionValidator`.

This makes the clock injectable too.

* kvutils: Provide logging contexts in the `Runner`.

* sandbox: Remove the `StaticAllowBackwards` time provider type.

It's not used anywhere.

* sandbox: Fix warnings in CliSpec.

* sandbox: Ensure that we cannot specify both static and wall-clock time.

* sandbox-next: Crash if wall clock time is not specified.

* sandbox-next: Document more known issues in the new Sandbox.

* sandbox: Add a Clock (and some tests) to TimeServiceBackend.

* sandbox-next: Support static time.

CHANGELOG_BEGIN
- [Sandbox Next] Re-establish static time mode.
CHANGELOG_END

* ledger-on-(memory|sql): Expect a `() => Instant`, not a `Clock`.
2020-02-18 17:03:15 +00:00
Moritz Kiefer
c64919cd58
Support --help in daml trigger/daml script/daml script-test (#4534)
fixes #4529
fixes #4530
fixes #4531

changelog_begin
changelog_end
2020-02-17 09:09:01 +01:00
Samir Talwar
a589f4af0b
sandbox: Move more resource acquisition into the owner. (#4501)
* sandbox: Move more resource acquisition into the `owner`.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Reimplement SandboxClientResource as a resources.Resource.

* codegen: Use resources in TestUtil.

* sandbox: Manage PostgreSQL in tests with ResourceOwners.
2020-02-14 13:52:45 +00:00
Moritz Kiefer
e7b8cdba89
Require explicit time mode for DAML script and DAML trigger (#4484)
This should provide a better migration path for people that still rely
on static time by forcing them to make this explicit. Given that both
DAML script and DAML triggers are still experimental, I’m not marking
this as a breaking change

changelog_begin

- [DAML Script - Experimental] The time mode must now always be
  specified explicitly. Use ``--static-time`` to recover the previous
  default time mode.

- [DAML Triggers - Experimental] The time mode must now always be
  specified explicitly. Use ``--static-time`` to recover the previous
  default time mode.

changelog_end
2020-02-12 11:16:25 +01:00
Moritz Kiefer
8590ff8274
Document all arguments of the trigger rule (#4448)
Previously we were missing the time.

changelog_begin
changelog_end
2020-02-07 18:20:04 +01:00
Gary Verhaegen
47bd131f15
add copyright headers to yml files (#4407)
We seem to have forgotten about them in the copyright scripts.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-06 12:54:07 +01:00
Remy
5fa173ea19
Engine: make Transaction/Node/VersionnedValue mapContractId LF private (#4364)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-05 16:25:39 +00:00
Miklos
dc0bca5a2f
Fix missing interpolators (#4372)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-04 17:09:49 +01:00
Moritz Kiefer
6453a73968
Add first prototype of triggers as a service (TaaS) (#4287)
* Add first prototype of triggers as a service (TaaS)

This is an extremely basic version of the trigger as a service thingy.

Right now, it supports spawning triggers and stopping them but nothing
else.

There is a very simple test to check that it’s not completely broken.

changelog_begin
changelog_end

* Apply suggestions from code review

Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* remove debugging output

* remove leftover import

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2020-01-31 12:53:45 +01:00
Andreas Herrmann
2e08e23703
Daml.Trigger.Assert for trigger testing API (#4264)
* Daml.Trigger.Assert for trigger testing API

Requires extracting part of Daml.Trigger into Daml.Trigger.Internal to
get access to internal data constructors and functionality

CHANGELOG_BEGIN
- [DAML Trigger - Experimental] Trigger testing functionality is now
available in the module Daml.Trigger.Assert.
CHANGELOG_END

* Set exposed-modules to hide Daml.Trigger.Internal

* API docs for Daml.Trigger.Assert

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-29 16:57:00 +00:00
Andreas Herrmann
74984559e8 Test triggers in scenarios (#4233)
* Bazel test for trigger scenario

* daml-triggers: Allow testing trigger rules in scenarios

CHANGELOG_BEGIN
- [DAML Triggers - Experimental] DAML triggers can now be tested in
scenarios. Specifically, a trigger's ``rule`` can be executed in a
scenario and assertions performed on the emitted commands.
CHANGELOG_END

* Allow assertions on create commands

CHANGELOG_BEGIN
* [DAML stdlib] Add `CanAbort` instance for `Either Text`.
CHANGELOG_END

* Add convenience to construct ACS for testRule

* Add assertions for exercise and exerciseByKey

* fix assert message

* Test assertExercise(ByKey)Cmd

* unpackCommands --> flattenCommands

* Add API documentation

* Document that command ids start from "0"

* generalise command assertions to CanAbortm

* export ACSBuilder type

* Better haddocs for trigger command assertions

* explicit let

* ./fmt.sh

* Fix runfiles on Windows

* Add reference to Bazel issue

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-28 11:50:46 +00:00
Remy
b6e848b576
Engine: Add node and contract Discriminator
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-22 18:49:30 +01:00
Moritz Kiefer
955978b894
Version the daml-trigger and daml-script DARs using the SDk version (#4148)
This makes much more sense than setting it to a fixed 0.0.1 version.

changelog_begin
changelog_end
2020-01-21 21:30:34 +01:00
Andreas Herrmann
3ec35be3de
Handle MHeartbeat in runTrigger (#4022)
* daml triggers: Set -Werror

* Handle MHeartbeat message in runTrigger

This was missing when heartbeat support was added.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-13 14:56:21 +01:00
Andreas Herrmann
2b64ea3ca4 Implement DAML trigger heartbeat (#4011)
* Implement heartbeat messages in trigger runner.

* Add heartbeat to Daml.Trigger

CHANGELOG_BEGIN
- [DAML Triggers - Experimental] DAML triggers can now configure a heartbeat message to be sent at regular time interval.
CHANGELOG_END

* Add DAML trigger heartbeat test-case

* ./fmts.h

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-10 16:04:47 +00:00
Moritz Kiefer
27fd561af6 Fix trigger time tests (#4002)
These have failed quite a few times on Windows and occasionally also
on MacOS.

This test, first fixes a small issue where the tests were actually
using the times from completions instead of only the timings from
creations. (that technically shouldn’t be an issue but it’s at least
confusing since the error claims to test creations).

In addition to that, this PR changes the condition to allow for the
times to be equal since especially on Windows we don’t seem to have a
very high resolution and the tests are remarkably quick so sometimes
the times can be identical.

I’ve slightly rephrased the condition since I got confused by the fact
that we test for the negated condition.

changelog_begin
changelog_end
2020-01-09 16:53:43 +00:00
Gerolf Seitz
0438bd5ec2
Log DAML trace messages (#3983)
Fixes #28.

CHANGELOG_BEGIN
[Sandbox] DAML trace logs (trace, traceRaw, traceId) are now logged via the regular logging system (slf4j+logback) at interpretation time via the logger ``daml.tracelog`` at DEBUG level.
CHANGELOG_END
2020-01-08 17:14:50 +01:00
Gary Verhaegen
878429e3bf
update copyright notices to 2020 (#3939)
copyright update 2020

* update template
* run script: `dade-copyright-headers update .`
* update script
* manual adjustments
* exclude frozen proto files from further header checks (by adding NO_AUTO_COPYRIGHT files)
2020-01-02 21:21:13 +01:00
Brian Healey
9f13a2fb10 Upgrade to Akka 2.6.1, akka-http 10.1.11 and Scala 2.12.10 (#3903)
* Upgrade to Akka 2.6.1, akka-http 10.1.11 and Scala 2.12.10

Akka 2.6.1 Upgrade Changes
- Materializer in place of ActorMaterializer
- Source.future instead of Source.fromFuture
- The Scheduler.schedule method has been deprecated in favor of selecting scheduleWithFixedDelay or scheduleAtFixedRate
- onDownstreamFinish(cause: Throwable)
- ActorAttributes.supervisionStrategy(...) in place of ActorMaterializerSettings.withSupervisionStrategy

See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html

* Akka 2.6.1 Upgrade Changes
- onDownstreamFinish(cause: Throwable)

See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html

* code review: remove unnecessary supervision strategy
2019-12-19 18:00:59 +00:00
Andreas Herrmann
06cd1f14a2
Expose timestamp in DAML triggers (#3858)
* Add time to Trigger update function

CHANGELOG_BEGIN

- [DAML Triggers - Experimental] Expose timestamp in triggers.
  See `#3612 <https://github.com/digital-asset/daml/issues/3612>`__.

CHANGELOG_END

* Add triggers time test

* Update trigger docs
2019-12-17 11:12:05 +01:00