Commit Graph

52 Commits

Author SHA1 Message Date
Moritz Kiefer
212bc9fdf3
Drop more controller can from the docs (#13215)
* Drop more controller can from the docs

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2022-03-09 11:35:54 +01:00
Gary Verhaegen
dfa648f585
hunt down DAML better (#13195)
Process:

- `git ls-files -z | xargs -0 -n 100 sed -i --follow-symlinks 's/DAML/Daml/g'`
- `git add -p`
- `git restore -p`
- Check there is no unstaged change left.

To review:

- Check for false positives by carefully reviewing the diff in this PR.
- Check for false negatives with `git grep DAML`.
- Quicker check for fals positives:

```
git grep DAML | grep -v migration | grep -v DAML_
```

Fixes #13190

Note: This is the "second half" of #13191, which failed to cover all the
remaining DAMLs because of:

```
$ git ls-files | grep "'"
compiler/damlc/tests/daml-test-files/MangledScenario'.daml
```

CHANGELOG_BEGIN
CHANGELOG_END
2022-03-08 17:04:58 +01:00
Moritz Kiefer
8e32842f57
Switch docs for upgrade automation to Canton Sandbox (#12908)
changelog_begin
changelog_end
2022-02-14 14:08:32 +00:00
Moritz Kiefer
a30c5c2ebc
Switch upgrade docs to Canton sandbox (#12904)
* Switch upgrade docs to Canton sandbox

Mainly requires explicitly allocating parties & users and dealing with
Navigator being annoying.

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2022-02-14 13:55:53 +01:00
Moisés Ackerman
dcd721da99
Drop scenarios (#12484)
* Remove scenarios from user examples

* Remove scenarios from tests

* implicitly enable scenario service for doctests

changelog_begin
changelog_end
2022-01-24 18:42:42 +01:00
Moritz Kiefer
ce624915d5
Fix damlc warnings & compile flags (#12404)
`--ghc-option=-Werror` turns warnings into errors, `--ghc-option=-Wwarn`
undoes exactly that so this clearly was not doing anything sensible.

changelog_begin
changelog_end
2022-01-13 18:32:31 +00:00
Gary Verhaegen
d2e2c21684
update copyright headers (#12240)
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.

I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-03 16:36:51 +00:00
Moisés Ackerman
ba0c6c9841
Set --enable-scenarios to False by default (#12156)
* Set --enable-scenarios to False by default

changelog_begin
changelog_end

* Enable scenarios for shake test

* Add --enable-scenarios=yes for //compiler/lsp-tests

* Add enable_scenarios arg to rules_daml/daml.bzl helpers

* daml_compile
* daml_build_test
* daml_test

* Add enable_scenarios to ledger/test-common helper da_scala_dar_resources_library

* Add --enable-scenarios=yes for //ledger/test-common

* Remove unused scenario in //ledger/test-common PingPong test

* Add --enable-scenarios=yes for //daml-lf/tests

* Add --enable-scenarios=yes for //language-support/java tests

* Add --enable-scenarios=yes for group-chat example

* Add --enable-scenarios for //ledger/sandbox-perf LargeTransaction test

* Add --enable-scenarios=yes for //docs tests

* Add --enable-scenarios=yes for //daml-lf/scenario-interpreter tests

* format bazel

* Add --enable-scenario=yes for 'daml_doc_test's

* Add --enable-scenarios=yes for DamlDocTestIntegration

* Add --enable-scenarios=yes for DamlcTest

* Add --enable-scenarios=yes for Test/DataDependencies

* Add --enable-scenarios=yes for daml-ghc-deterministic.sh

* Add --enable-scenarios=yes for Test.IncrementalBuilds

* Add --enable-scenarios=yes for tests:memory-bond-trading and tests:memory-examples

* Add --enable-scenarios=yes for daml-assistant/integratio-tests
2021-12-20 19:31:06 +01:00
Moritz Kiefer
16135e6f7c
Limit supported input versions in damlc to >= LF 1.8 (#11905)
* Limit supported input versions in damlc to >= LF 1.8

1.8 was the version that introduced type synonyms, we really don’t
gain much by dropping more since data-dependencies mainly depends on
that. and this provides for a very natural upgrade path for users
where pretty much everyone should be able to upgrade directly to SDK
2.0 without having to go through intermediate versions.

changelog_begin

- [Daml Compiler] The supported input LF versions for
  data-dependencies are now limited to LF 1.8 and newer.

changelog_end

* fix some tests

changelog_begin
changelog_end

* Drop export 1.6 tests

changelog_begin
changelog_end

* Drop daml2js support for LF < 1.8

changelog_begin

- [Daml2js] DARs with LF version < 1.8 are no longer supported.

changelog_end

* .

changelog_begin
changelog_end

* bash is bad, stop using it

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-11-30 17:24:08 +01:00
Moritz Kiefer
5785bbfcfd
Drop LF < 1.14 from supported damlc output versions (#11701)
* Drop LF < 1.14 from supported damlc output versions

fixes #11319

We keep test coverage by depending on the most recent snapshot which
still has 1.14 support.

changelog_begin

- [Daml Compiler] Damlc can only produce Daml-LF 1.14 or
  newer. Passing aynthing older to `--target` is an error. If you
  need to produce older versions, use an older SDK.

changelog_end

* Switch around legacy_compiler_lf_versions

changelog_begin
changelog_end

* drop since-lf

changelog_begin
changelog_end
2021-11-22 16:20:17 +00:00
Moritz Kiefer
c098d75621
Add a --log-level flag to damlc (#11514)
This also allows us to quite down our builds a bit which is nice.

changelog_begin
changelog_end
2021-11-03 14:22:57 +01:00
Robin Krom
0c1878530b
test: test for platform independent dars (#10535)
We add a CI test to check that dars don't depend on the underlying
operating system where the dar is build.

CHANGELOG_BEGIN
CHANGELOG_END
2021-08-17 18:59:12 +02:00
Moritz Kiefer
4fd1cf7c28
Fix doctests (#9225)
Haven’t tracked down when they broke. They used to work at some point
but atm they are very broken and don’t test anything.

This PR fixes the bash mess to parse the files correctly and fixes two
minor issues that creeped in by virtue of the doctests being broken.

changelog_begin
changelog_end
2021-03-24 12:56:37 +01:00
Moritz Kiefer
f6effb228a
Add a compile-only damlc target (#9164)
This untangles the dependency structure a bit so that //daml-lf no
longer ends up depending on daml script and sandbox and similar crap
which should improve build times in general.

changelog_begin
changelog_end
2021-03-17 10:31:37 +01:00
Moritz Kiefer
5d88c08832
Bump ghcide (#9041)
* Bump ghcide

Includes https://github.com/digital-asset/daml-ghcide/pull/13 meaning
we can now remove the hacks for missing signatures

changelog_begin

- [Daml Compiler] Fix a bug where passing `--ghc-option=-Werror` also
  produced errors for warnings produced by `-Wmissing-signatures` even
  if the user did not explicitly enable this.

changelog_end

* Bump to merged commit

changelog_begin
changelog_end
2021-03-05 18:17:28 +01:00
Moritz Kiefer
0281b442b8
-Werror all the things (#9027)
We used to do this for some packages but it broke cpp. I don’t
actually know why it doesn’t do that anymore but I’ll gladly accept
that fact and turn it on everywhere.

changelog_begin
changelog_end
2021-03-04 19:08:59 +01:00
Remy
d92f2c7003
Ledger: refactor bazel packaging of tedger test tool (#8894)
improve previous generalization from #8695

- use lf version instead keyword (like 'stable', 'latest', 'dev') to
  tag actual target.  This will allow two keywords to map to the same
  versions without doing twice the compilation/test work.

- use alias to map keywords tag target to versioned tag target.

- move package manage dar to test_commong.

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-22 11:41:19 +01:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
nickchapman-da
6a745ed1fa
Support choice observers in 1.dev (#7922)
* Adapt to new desugaring for choice observers.

update hash of ghc patch.

changelog_begin
changelog_end

update ghc patch to final version

update stack-snapshot hashes for ghc-lib(-parser)

update stackage_snapshot.json, following `bazel run @stackage-unpinned//:pin`

expose Optional constructors for desugared code to use

adapt LFConversion to expect a 4-tuple for a desugared choice def/sig

update LFConversion for choice-observers

first example using new choice observer syntax.

fix scala type checker to have correct scoping rules for choice-observers

remove comment from example which says it is broken

improve variable names

improve tests for choice-observer clause

only test choice-observers SINCE 1.dev

add jq queries for choice observeres

make positive statement in jq test which checks choice obserers are present

test behaviour of choice observers

squash me

typo

* test more choice-observer divulgence

* Update documention for choice observers.

changelog_begin
Support choice observers in 1.dev
changelog_end

* fix docs build

* fix daml docs choice-observers example

* address comments: rewording text

* annotate choices observers as early-access in documention

* split out documentation code-snippets which require --target=1.dev

* final tweaks to documentation text
2020-11-18 19:51:15 +00:00
Bernhard Elsner
af2607cfe5
Intro to DAML Chapter 8 (#7506)
* Some minor improvements to the DAML Intro docs

CHANGELOG_BEGIN
CHANGELOG_END

* Intro to DAML Chapter 8

CHANGELOG_BEGIN
CHANGELOG_END

* Incorporate feedback

* Update docs/source/daml/intro/8_Upgrading.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Remove wrong statements about interface files

* Complete rework to focus on dependencies, not upgrades

* Copyright headers and feedback

* Update toctree

* Get rid of warnings

* Fix template

* Fix tests

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-09-29 16:34:21 +00:00
Andreas Herrmann
6c8964fb3b
java-quickstart replace scenarios by DAML Script (#7183)
* java-quickstart replace scenarios by DAML Script

This replaces the scenarios in the java quickstart project by DAML
script test cases.

The following changes were required
* Replace `getParty` by `allocateParty`.
* Replace `create` and `exercise` by `createCmd` and `exerciseCmd`.
* Replace the pattern of `submit ... do create ...; exercise ...`
  by `submit ... do createAndExerciseCmd ... ...`.
* Replace instances of `submit ... do fetch ...` by `query ... ...`.
  To that end I've added the following helper function.
  ```
  queryIou : Party -> ContractId Iou -> Optional Iou
  ```

changelog_begin
changelog_end

* Replace queryIou by FetchIou choice

changelog_begin
changelog_end

* Add comment explaining fetch choice

Addressing review comment
https://github.com/digital-asset/daml/pull/7183#discussion_r478918758

* Support dependencies in daml_test rule

Cherry-picked from https://github.com/digital-asset/daml/pull/7264/files
Renamed `dependencies` as `deps` to stick to Bazel conventions.

* Fix quickstart `daml_test` target

* fmt

* Quote deps in daml.yaml

To avoid issues with `:` in Windows paths.

* //docs:quickstart-java - use temp dir

This target repeatedly failed to build on Windows with
```
ERROR: D:/a/1/s/docs/BUILD.bazel:538:8: declared output
'docs/quickstart-java.tar.gz' was not created by genrule. This is
probably because the genrule actually didn't create this output, or
because the output was a directory and the genrule was run remotely
(note that only the contents of declared file outputs are copied from
genrules run remotely)
```
The genrule was operating in the current working directory and builds on
Windows are not sandboxed. Conflicting files in the current working
directory could cause this to fail.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-28 16:19:45 +00:00
Andreas Herrmann
4b1438276c
Update Bazel 2.1.0 --> 3.3.1 (#6761)
* Upgrade nixpkgs revision

* Remove unused minio

It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.

* Update Bazel on Windows

changelog_begin
changelog_end

* Fix hlint warnings

The nixpkgs update implied an hlint update which enabled new warnings.

* Fix "Error applying patch"

Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.

* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681

Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.

```
git diff ':(exclude)WORKSPACE'
```

* Update rules_nixpkgs

* nixpkgs location expansion escaping

* Drop --noincompatible_windows_native_test_wrapper

* client_server_test using sh_inline_test

client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.

This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.

* daml_test using sh_inline_test

* daml_doc_test using sh_inline_test

* _daml_validate_test using sh_inline_test

* damlc_compile_test using sh_inline_test

* client_server_test find .exe on Windows

* Bump Windows cache for Bazel update

Remove `clean --expunge` after merge.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-23 09:46:04 +02:00
Andreas Herrmann
0544323dfc
upgrade rules_haskell (#6284)
* upgrade rules_haskell

CHANGELOG_BEGIN
CHANGELOG_END

* Use autogenerated stack_snapshot tools

* ghcide import-dirs flags

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-06-10 17:21:31 +02:00
nickchapman-da
9e5ae15e6d
Bazel rule to inspect dar file when compiling daml (#6067)
This change extends the bazel rule for `compile_daml` to add a stage which will pretty-print the generated .dar file, suitable for human inspection. The generated file is named with a `.dar.pp` suffix, and will only be generated if explicitly requested as a build target or listed as a dependency.

Make use of the new rule by demanding CollectAuthority.dar.pp when the perf benchmark is built.

changelog_begin
changelog_end
2020-05-21 11:06:51 +00:00
Moritz Kiefer
8c25bc4de5
Fix find: ./local-spawn-runner not found issue (#5793)
CWD will be set to the same execroot for all targets on Windows. While
this will contain the things we are searching for it contains a whole
bunch of other stuff and in particular it can also change during the
execution of `find`. This resulted in errors with temporary files such
as the local-spawn-runner-* stuff that appear and disappear while
find is running.

This PR switches it to a tmp dir which works around this issue and
makes more sense anyway since we clearly don’t want to search in the
whole execroot.

changelog_begin
changelog_end
2020-04-30 15:09:04 +02:00
Moritz Kiefer
b488b3530b
Fix sdk-versions in daml_build rule (#5479)
I thought we already were patching those because the versions of all
other packages are hardcoded but apparently we just got lucky so far
because we got it consistently wrong. The dependency on the trigger
and script libraries resulted in them being inconsistently wrong since
those do set the correct SDK version so things fell over.

changelog_begin
changelog_end
2020-04-07 20:17:55 +00:00
Moritz Kiefer
ccacddbbc6
Add documentation for upgrading using DAML script and triggers (#5465)
* Add documentation for upgrading using DAML script and triggers

This PR adds documentation for how you can automate the upgrade
process from the Coin example using DAML Script and DAML Triggers.

Having written this, I think we might want to add some integration
tests for this but it’s rather complex to setup since it ties together
so many pieces so I’d like to leave it out for now. We do at least
test that the code compiles and we all knows that code that compiles
never has bugs.

changelog_begin
changelog_end

* Apply suggestions from code review

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
2020-04-07 14:36:40 +02:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
nickchapman-da
d81caa9568
rework daml_compile bazel rule (#5070)
Avoiding `damlc compile/package` commands (which we would like to deprecate), and replace with plain `damlc build` together with a post dar->dalf extraction step in the couple of places where we actually want the .dalf for testing.

changelog_begin
changelog_end
2020-03-18 23:06:55 +00:00
nickchapman-da
944a23e302
New bazel rule -- daml_build_test -- to build a DAML project from its daml.yaml config, and setup a bazel test rule to validate the resulting .dar using daml validate-dar. (#4891)
Dependencies on other DAML projects are declared with the `dar_dict` attribute of the build rule. This attribute also declares the names by which the `.dar` files are known in the client project, corresponding to the references in the `daml.yaml` config.

The new rule is used build & test the upgrade documentation example code.

changelog_begin
changelog_end
2020-03-10 14:23:23 +00:00
Gary Verhaegen
5a117dc358
introduce new release process (#4513)
Context
=======

After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.

This PR is a proposal for a new release process designed under the
following constraints:

- Reuse as much as possible of the existing infrastructure, to minimize
  effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
  not meant for general public consumption, but can still be used by savvy
  users without jumping through too many extra hoops (ideally just
  swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
  release status, with as few changes as possible in-between, so we can be
  confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
  releases, to avoid discovering non-transient problems while trying to
  promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
  keep the same approval process for SOC2 auditability.

The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:

```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```

This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.

Version numbers
===============

Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:

- Semantic versioning (which we chose as the version format for the SDK
  version number) allows for "prerelease" version numbers as well as
  "metadata"; an example of a complete version string would be
  `1.2.3-nightly.201+server12.43`. The "main" part of the version string
  always has to have 3 numbers separated by dots; the "prerelease"
  (after the `-` but before the `+`) and the "metadata" (after the `+`)
  parts are optional and, if present, must consist of one or more segments
  separated by dots, where a segment can be either a number or an
  alphanumeric string. In terms of ordering, metadata is irrelevant and
  any version with a prerelease string is before the corresponding "main"
  version string alone. Amongst prereleases, segments are compared in
  order with purely numeric ones compared as numbers and mixed ones
  compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
  which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
  `-`, or a transition between a number and a letter. Version strings
  are compared element-wise, with numeric segments being compared as
  numbers. Alphabetic segments are treated specially if they happen to be
  one of a handful of magic words (such as "alpha", "beta" or "snapshot"
  for example) which count as "qualifiers"; a version string with a
  qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
  which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
  special ordering amongst qualifiers. Other alphabetic segments are
  compared alphabetically and count as being "after" their prefix
  (`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
  separated by `.`, plus an optional (though deprecated) alphanumeric
  "version tag" separated by a `-`. I could not find any official
  documentation on ordering for the version tag; numeric segments are
  compared as numbers.
- npm uses semantic versioning so that is covered already.

After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:

```
0.13.53-alpha.20200214.3025.ef5d32b7
```

where the components are, respectively:

- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
  master.
- `3025`: the number of commits in master up to the release commit
  (included). Because we have a linear, append-only master branch, this
  uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
  not strictly speaking necessary, but makes it a lot more convenient to
  identify the commit.

The main downsides of this format are:

1. It is not a valid format for GHC packages. We do not publish GHC
  packages from the SDK (so far we have instead opted to release our
  Haskell code as separate packages entirely), so this should not be an
  issue. However, our SDK version currently leaks to `ghc-pkg` as the
  version string for the stdlib (and prim) packages. This PR addresses
  that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
  would see the above version number as `0.13.53.20200214.3025`, which
  should be enough to uniquely identify it. Note that, as far as I could
  find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
  it makes it more scary. However, I have been told that this may be
  long enough to cause issues on Windows by pushing us past the max path
  size limitation of that "OS". I suggest we try it and see what
  happens.

The upsides are:

- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
  they have the same date, allowing us to release same-day patches if
  needed. (Note: that is, commits that were made on the same day; the
  release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
  immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
  GHC).

Alternatives I considered:

- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
  be accepted by all schemes, but will not sort as expected under semantic
  versioning (though Maven will be fine). I have no idea how it will sort
  under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
  This is not valid semantic versioning and is therefore rejected by
  npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
  what is generally done in the Java world, but we then lose track of what
  version is actually in use and I'm concerned about bug reports. This
  would also not let us publish to the main Maven repo (at least not more
  than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
  version formats. However, it would not clearly indicate to humans that
  it is not meant as a stable version, and would sort differently under
  semantic versioning (which counts it as a prerelease, i.e. before
  `0.13.53`) than under maven (which counts it as a patch, so after
  `0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
  number of prereleases in-between `0.13.52` and the next. This is
  currently the fallback plan if Windows path length causes issues. It
  would be less convenient to map releases to commits, but it could still
  be done via querying the history of the `LATEST` file.

Release notes
=============

> Note: We have decided not to have release notes for snapshot releases.

Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.

But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.

The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.

> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.

New release process
===================

Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.

Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:

1. Before running the release step, CI will checkout the commit
  specified in the LATEST file. This requires separating the release
  step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
  that gets the version to build from an environment variable, with a
  default of `0.0.0` to remain consistent with the current `daml-head`
  behaviour.

Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.

The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.

I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 17:01:23 +01:00
Gary Verhaegen
878429e3bf
update copyright notices to 2020 (#3939)
copyright update 2020

* update template
* run script: `dade-copyright-headers update .`
* update script
* manual adjustments
* exclude frozen proto files from further header checks (by adding NO_AUTO_COPYRIGHT files)
2020-01-02 21:21:13 +01:00
Martin Huschenbett
ab07b05fcb Clean up rules_daml (the bazel rules for DAML) (#3495)
There are rules that are used nowhere and seem like old cruft.
2019-11-18 08:12:47 +00:00
Andreas Herrmann
c994703c0c Update rules_haskell (#3473)
* Update rules_haskell & rules_nixpkgs

* Define dadew POSIX toolchain on Windows

* Build hpp with stack and Cabal

* Replace Hazel hpp by @stackage hpp

* replace backslash by forward slash

* Cabal wrapper exclude bindist includes

* ghci-grpc patch fix missing argument

* Switch to rules_haskell master
2019-11-15 10:50:12 +00:00
Andreas Herrmann
33e47828e3
Bazel 1.1 (#3249)
* bazel: 0.28.1 --> 1.1.0

* bazel-watcher sha256

* Fix missing line in patch

* proto_source_root --> strip_import_prefix

See https://github.com/bazelbuild/bazel/issues/7153 for details.

* Update rules_nixpkgs

Required to avoid errors of the form
```
ERROR: An error occurred during the fetch of repository 'node_nix':
   parameter 'sep' may not be specified by name, for call to method split(sep, maxsplit = None) of 'string'
```

and
```
ERROR: An error occurred during the fetch of repository 'node_nix':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
		_execute_or_fail(repository_ctx, <3 more arguments>)
	File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
		fail(<1 more arguments>)

Cannot build Nix attribute 'nodejs'.
Command: [/Users/runner/.nix-profile/bin/nix-build, /private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/node_nix/nix/bazel.nix, "-A", "nodejs", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
src/main/tools/process-tools.cc:173: "setitimer": Invalid argument
```

* Update rules_scala

* .proto has been removed, use [ProtoInfo] instead

See
https://docs.bazel.build/versions/1.1.0/be/protocol-buffer.html#proto_library

* python3_nix add nix_file attribute

To avoid the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/BUILD:66:1: //:nix_python3_runtime depends on @python3_nix//:bin/python in repository @python3_nix which failed to fetch. no such package '@python3_nix//': Traceback (most recent call last):
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
                _execute_or_fail(repository_ctx, <3 more arguments>)
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
                fail(<1 more arguments>)

Cannot build Nix attribute 'python3'.
Command: [/home/aj/.nix-profile/bin/nix-build, "-E", "import <nixpkgs> { config = {}; overlays = []; }", "-A", "python3", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
error: anonymous function at /home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs.nix:3:1 called with unexpected argument 'config', at (string):1:1
```

* rules_haskell unnamed string.split(_, maxsplit = _)

The keyword argument may no longer be named.

* string.replace(_, _, maxsplit = _) may not be named

* Move proto sources from deps to data

Fixes

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/daml-lf/archive/BUILD.bazel:150:1: in deps attribute of scala_test rule //daml-lf/archive:daml_lf_archive_reader_tests_test_suite_src_test_scala_com_digitalasset_daml_lf_archive_DecodeV1Spec.scala: '//daml-lf/archive:daml_lf_1.6_archive_proto_srcs' does not have mandatory providers: 'JavaInfo'. Since this rule was created by the macro 'da_scala_test_suite', the error might have been caused by the macro implementation
```

* Define sha256 for haskell_ghc__paths

Bazel 1.1.0 fails on missing hashes.

* Disable --incompatible_windows_native_test_wrapper

* //compiler/daml-extension don't modify sources

Modifying sources in-place can cause issues on Windows, where build
actions are not sandboxed and changes on sources can affect other build
steps.

* bazel-genfiles --> bazel-bin

The bazel-genfiles symlink has been removed since Bazel 1.0.
See https://github.com/bazelbuild/bazel/issues/8651

* Mark dev_env_tool repository rule as configure

See
https://docs.bazel.build/versions/1.1.0/skylark/lib/globals.html#repository_rule

* Move data deps into data attribute

* Mark dev_env_tool as local = True

* Manually fetch @makensis_dev_env
2019-11-11 10:06:03 +01:00
Moritz Kiefer
69da1d6379 Implement AnyTemplate in DAML (#2947)
* Implement AnyTemplate in DAML

* Fix doctest path

* Shuffle around CPP

* Do not hide anything

* Hide it again

* Clean build

* Enable caching again

* debug windows crap

* more tests

* reinstantiate full tests
2019-09-19 08:42:55 +00:00
Moritz Kiefer
9992bdbd2f
Support cpp in doctests (#2957)
* Support cpp in doctests

* debugging lalalala

* More debugging
2019-09-18 17:36:42 +02:00
Robin Krom
aa1e95173a
language: check dflags for errors (#2748)
* language: check dflags for errors

We add a check when we build the dflags for cases that will lead to a
failed build and emmit a clearer error message. Currently this only
includes a check, to see whether the current installed unit id is also
imported as a package from the package database.

* exclude ghc-prim from check

* exclude code generation from dflag check

* add an internal option to turn dflags check off
2019-09-06 21:24:46 +02:00
Gary Verhaegen
99ea93168d
update copyright notices (#2499) 2019-08-13 17:23:03 +01:00
Robin Krom
829cb4cef5
Daml stdlib with versions (#2479)
* language: build daml_stdlib with a version in the package name.

* versioned daml-stdlib in damlc

* reformat bazel files

* fix daml_doc_test
2019-08-12 16:05:28 +02:00
Moritz Kiefer
5aa3cba949
Add a damlc doctest command and test the standard library (#2157)
There is lots of room for improvements here but I think this is a good
first step. The 3 main things that could be improved imho are:

- Rewrite source locations to point to the original file rather than
  the generated module

- Provide some way to declare things like imports or more general,
  setup code that is added to the generated module.

- Prettier/more helpful output during a run, e.g., print the list of
  successful tests.
2019-07-16 15:51:01 +02:00
Moritz Kiefer
bb3a98bf61
Move code in daml-tools outside of daml-foundations (#2033) 2019-07-08 11:40:48 +02:00
Andreas Herrmann
93d8e01db2
Update rules_haskell and static GHC (#1515)
* Update rules_haskell and static GHC

Remove patches that have been upstreamed or are no longer required.
Update still required patches to match the new rules_haskell version.

Previously we patched rules_haskell to coerce GHC into using static
Haskell libraries in most places. In particular we moved hs-libraries
entries into extra-libraries entries in the package configuration files.
A much cleaner approach is to compile GHC with a static RTS, then GHC
will by itself choose to load static Haskell libraries.

* Remove haskell_cc_import

* da-hs-daml-cli -> daml-cli

* da-hs-damlc-app -> damlc-app
2019-07-01 13:26:19 +02:00
Michał Majcherski
2afefb7883
windows: root build (#1894)
* windows: root build

* windows: fixed haskell bindings tests

* windows: disable client_server_test test

* windows: marking daml_test flaky due to #1907

* windows: removing da-hs-damlc-app run from build.ps1

* windows: disable hie-core alias of currently disabled target
2019-06-28 12:55:31 +02:00
Andreas Herrmann
27ba8a77c8
daml_test: Take damlc runfiles into account (#1607)
Closes #1593
2019-06-12 17:18:45 +02:00
Andreas Herrmann
1a80106753
damlc test --files (#1409)
* Add test case for damlc test-files

* Separate damlc test-files from damlc test

* Fix rules_daml/daml.bzl daml_test

* damlc test-files --> damlc test --files

The project options are still relevant for the --files case, as it may
be necessary to change into the project root directory in order to
locate the project package database.
2019-05-28 15:36:42 +02:00
Leonid Shlyapnikov
f31980c18f Add Scala codegen bazel rule: dar_to_scala (#1202)
* Removing unused/broken daml.bzl rules, adding Scala codegen rule

Scala codegen rule: dar_to_scala follows the same approach
as dar_to_java with a few differences:
- dar_to_scala supports multiple dars as an input
- dar_to_scala does not try to compile the generated scala (separation
  of responsibilities)

* Using dar_to_scala to compile quickstart-scala example

* Fixing formatting

* Add dependency to examples-quickstart-scala-bin

So if example does not compile, it can't be published.

* Fixing the path to the jar executable

* Changing scala codegen rule to rely on zipper instead of jar

JDK's jar creates srcjar with timestamped files, zipper doesn't
timestamp files. This means we can create reproducible/deterministic/
cacheable srcjars

* Addressing code review comments, wrong var name
2019-05-20 14:13:30 +00:00
moritzkiefer-da
dff9f17ac6 Remove damli completely (#481)
The only command that we still had in damli which afaik isn’t
something anyone uses (and if it is, we should move it do damlc).
2019-04-15 14:47:59 +02:00
moritzkiefer-da
f6c6dd6a9a
Accept multiple files in damlc test (#436)
* Accept multiple files in damlc test

Since damlc test also runs tests in transitive dependencies, this can
be significantly faster than running "damlc test" individually on
a set of files as you will end up recompiling and rerunning tests
multiple times if those files depend on each other.

For //docs:daml-ref-daml-test This is roughly a 10x improvement from
~70s to ~7s.
2019-04-12 16:21:34 +02:00
gleber
aa70c7f64e
Enforce consistent formatting of BUILD files. (#412)
* Add buildifier targets.

The tool allows to check and format BUILD files in the repo.

To check if files are well formatted, run:

    bazel run //:buildifier

To fix badly-formatted files run:

    bazel run //:buildifier-fix

* Cleanup dade-copyright-headers formatting.

* Fix dade-copyright-headers on files with just the copyright.

* Run buildifier automatically on CI via 'fmt.sh'.

* Reformat all BUILD files with buildifier.

Excludes autogenerated Bazel files.
2019-04-12 13:10:16 +02:00