Commit Graph

193 Commits

Author SHA1 Message Date
Moritz Kiefer
7063e30b76
Configure python toolchain via nixpkgs_python_configure (#5702)
changelog_begin
changelog_end
2020-04-23 17:30:32 +02:00
Rohan Jacob-Rao
7fafe6a08a
GSG: test messaging feature and remove copied code snippets (#5507)
* Diff with messaging feature and some noise manually removed

* Bazel target to use patch file in other build targets

* Patch file as data dep for integration tests

* Attempt to patch and test messaging feature in create-daml-app test

changelog_begin
changelog_end

* Use exports_files instead of filegroup

* Remove file existence checks that don't make sense

* Add patch to dev_env and reference it from integration tests

* Include patch on windows for later

* Set up yarn env again after codegen

* Restore file check

* Fix typo in comment on util function

* Add Tasty steps to make process explicit

* Use messaging patch for code snippets in GSG

* Use messaging code from template instead of copy

* Remove copied message code

* Refactor script to copy template code with messaging patch

* No need to retry second yarn install (only local deps should be updated)
2020-04-11 19:53:44 +00:00
Andreas Herrmann
f0929f2d04
Consistently replace zip by bazel_tools zipper (#5215)
* sdk-release-tarball: zip is unused

* daml-lf/archive: Use bazel_tools zipper

@bazel_tools//tools/zip:zipper avoids timestamps and sources of
indeterminism when creating an archive, so that the result is
reproducible.

CHANGELOG_BEGIN
CHANGELOG_END

* Remove unused @zip_dev_env

zip is fully replaced by @bazel_tools//tools/zip:zipper.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-26 17:38:03 +01:00
Moritz Kiefer
c004c90fcb
Remove skydoc (#5208)
Nobody uses it, it’s deprecated anyway.

changelog_begin
changelog_end
2020-03-26 11:14:34 +00:00
Oliver Seeliger
3df5a6ec5b
Upgrade canton to 0.11.0 (#5144)
in turn based on 13.56-snapshot-20200318

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 18:06:38 +00:00
Moritz Kiefer
0471b26d70
Get openssl from msys2 (#5063)
* Get openssl from msys2

changelog_begin
changelog_end

* Try to fix opensl config location
2020-03-18 13:09:03 +01:00
Moritz Kiefer
e0d652df91
Generate ssl certs in a genrule instead of checking them in (#4950)
changelog_begin
changelog_end

For now this is only used for the daml-helper tests. I’ll shuffle
things around and use it for all tests in a separate PR.
2020-03-12 10:36:40 +01:00
Andreas Herrmann
7a2bf268fe
ld -s --> ld -unexported_symbols_list=* on macOS (#4838)
On MacOS the `-s` linker flag (`--strip-all`) is obsolete. Instead, one
should pass `-unexported_symbols_list=*` to strip all symbols.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-03-05 16:56:01 +01:00
Moritz Kiefer
d455267e58
Reduce the number of GHCs in our nix closure (#4729)
This reduces the number of GHCs to 2 on Linux (regular and DWARF) and
1 on macOS. Given that each derivation is > 1 GB this should hopefully
help a bit.

changelog_begin
changelog_end
2020-02-26 20:00:07 +01:00
Gary Verhaegen
5a117dc358
introduce new release process (#4513)
Context
=======

After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.

This PR is a proposal for a new release process designed under the
following constraints:

- Reuse as much as possible of the existing infrastructure, to minimize
  effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
  not meant for general public consumption, but can still be used by savvy
  users without jumping through too many extra hoops (ideally just
  swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
  release status, with as few changes as possible in-between, so we can be
  confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
  releases, to avoid discovering non-transient problems while trying to
  promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
  keep the same approval process for SOC2 auditability.

The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:

```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```

This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.

Version numbers
===============

Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:

- Semantic versioning (which we chose as the version format for the SDK
  version number) allows for "prerelease" version numbers as well as
  "metadata"; an example of a complete version string would be
  `1.2.3-nightly.201+server12.43`. The "main" part of the version string
  always has to have 3 numbers separated by dots; the "prerelease"
  (after the `-` but before the `+`) and the "metadata" (after the `+`)
  parts are optional and, if present, must consist of one or more segments
  separated by dots, where a segment can be either a number or an
  alphanumeric string. In terms of ordering, metadata is irrelevant and
  any version with a prerelease string is before the corresponding "main"
  version string alone. Amongst prereleases, segments are compared in
  order with purely numeric ones compared as numbers and mixed ones
  compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
  which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
  `-`, or a transition between a number and a letter. Version strings
  are compared element-wise, with numeric segments being compared as
  numbers. Alphabetic segments are treated specially if they happen to be
  one of a handful of magic words (such as "alpha", "beta" or "snapshot"
  for example) which count as "qualifiers"; a version string with a
  qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
  which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
  special ordering amongst qualifiers. Other alphabetic segments are
  compared alphabetically and count as being "after" their prefix
  (`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
  separated by `.`, plus an optional (though deprecated) alphanumeric
  "version tag" separated by a `-`. I could not find any official
  documentation on ordering for the version tag; numeric segments are
  compared as numbers.
- npm uses semantic versioning so that is covered already.

After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:

```
0.13.53-alpha.20200214.3025.ef5d32b7
```

where the components are, respectively:

- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
  master.
- `3025`: the number of commits in master up to the release commit
  (included). Because we have a linear, append-only master branch, this
  uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
  not strictly speaking necessary, but makes it a lot more convenient to
  identify the commit.

The main downsides of this format are:

1. It is not a valid format for GHC packages. We do not publish GHC
  packages from the SDK (so far we have instead opted to release our
  Haskell code as separate packages entirely), so this should not be an
  issue. However, our SDK version currently leaks to `ghc-pkg` as the
  version string for the stdlib (and prim) packages. This PR addresses
  that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
  would see the above version number as `0.13.53.20200214.3025`, which
  should be enough to uniquely identify it. Note that, as far as I could
  find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
  it makes it more scary. However, I have been told that this may be
  long enough to cause issues on Windows by pushing us past the max path
  size limitation of that "OS". I suggest we try it and see what
  happens.

The upsides are:

- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
  they have the same date, allowing us to release same-day patches if
  needed. (Note: that is, commits that were made on the same day; the
  release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
  immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
  GHC).

Alternatives I considered:

- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
  be accepted by all schemes, but will not sort as expected under semantic
  versioning (though Maven will be fine). I have no idea how it will sort
  under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
  This is not valid semantic versioning and is therefore rejected by
  npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
  what is generally done in the Java world, but we then lose track of what
  version is actually in use and I'm concerned about bug reports. This
  would also not let us publish to the main Maven repo (at least not more
  than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
  version formats. However, it would not clearly indicate to humans that
  it is not meant as a stable version, and would sort differently under
  semantic versioning (which counts it as a prerelease, i.e. before
  `0.13.53`) than under maven (which counts it as a patch, so after
  `0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
  number of prereleases in-between `0.13.52` and the next. This is
  currently the fallback plan if Windows path length causes issues. It
  would be less convenient to map releases to commits, but it could still
  be done via querying the history of the `LATEST` file.

Release notes
=============

> Note: We have decided not to have release notes for snapshot releases.

Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.

But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.

The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.

> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.

New release process
===================

Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.

Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:

1. Before running the release step, CI will checkout the commit
  specified in the LATEST file. This requires separating the release
  step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
  that gets the version to build from an environment variable, with a
  default of `0.0.0` to remain consistent with the current `daml-head`
  behaviour.

Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.

The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.

I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 17:01:23 +01:00
Samir Talwar
3df3190122
ledger-api-test-tool-on-canton: Upgrade Canton to v0.9.0. (#4684)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 12:23:42 +00:00
Andreas Herrmann
2dbd51bf29
language-support/ts generate package in commonjs format (#4380)
* Build commonjs format for npm packaging

CHANGELOG_BEGIN
CHANGELOG_END

* Custom commonjs typescript library rule

The npm packages generated by rules_nodejs' native ts_library rule use
the UMD package format. This breaks webpack which attempts to determine
dependencies by static code analysis and fails on UMD. To avoid this we
call `tsc` directly to ensure generation of commonjs modules.

* Enable module mapping on ts_commonjs_library

* Replace ts_library by da_ts_library

* Add dummy typescript/index.bzl on Windows

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-02-04 18:50:56 +01:00
Samir Talwar
0d5ae27e04 ledger-api-test-tool-on-canton: Upgrade to Canton v0.8.0. (#4244)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 15:10:05 +00:00
Samir Talwar
6c1d931c05
ledger-api-test-tool-on-canton: Upgrade to Canton v0.7.0 and cl… (#4170)
* ledger-api-test-tool-on-canton: Upgrade Canton to v0.7.0.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-api-test-tool-on-canton: Use the health endpoint instead of `nc`.

Netty prints a warning when kicking it with `nc -z` from Nix's
netcat-gnu. Not other netcats though… don't know why.

* ledger-api-test-tool-on-canton: Let IntelliJ format the shell script.

* ledger-api-test-tool-on-canton: Stop Canton with SIGINT.

For some reason, Canton doesn't print out a load of errors if it's
stopped with SIGINT (Ctrl+C). SIGTERM, on the other hand…

* ledger-api-test-tool-on-canton: Write Canton logs to STDOUT.

Now the logs are no longer noisy, we can just pump them to STDOUT. This
should make debugging any transient issues on CI a lot easier.
2020-01-23 09:04:16 +01:00
Andreas Herrmann
032ed82254
language-support/ts yarn install (#4159)
* Remove language-support/ts/packages/yarn.lock

That file is for local development exclusively.

CHANGELOG_BEGIN
CHANGELOG_END

* yarn args --frozen-lockfile

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-22 17:28:14 +01:00
Moritz Kiefer
9b529ad8cb
Cleanup typescript hacks for Windows (#4167)
changelog_begin
changelog_end
2020-01-22 17:19:45 +01:00
Moritz Kiefer
970d9891fd
Disable all the TS stuff on Windows (#4163)
* Disable all the TS stuff on Windows

changelog_begin
changelog_end

* disable jest explicitly

* more disabling

* :sadpanda:

* Replace @language_support_ts_deps on Windows

Provides dummy content so that `load` commands are still valid on
Windows without `yarn_install`.

* disable daml-ledger-fetch on windows

* shut up buildifier

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-22 16:07:20 +01:00
Robin Krom
81b8897fc8
added a package.json to work with yarn workspaces (#4147)
* added a package.json to work with yarn workspaces

This adds a package.json files on top of our typescript libraries so
that we can develop locally via yarn workspaces. The package.json that
describes the bazel managed dependencies is moved into a subfolder.

CHANGELOG_BEGIN
CHANGELOG_END

* updated bazelignore

* SDK_VERSION -> SDKVERSION
2020-01-21 22:23:51 +01:00
Andreas Herrmann
6cf5458e30
Avoid intermediate nodejs_dev_env (#4135)
* Remove unused npm_nix

* Use node_nix directly on Unix

rules_nodejs requires a vendored node to contain the node distribution
in a subdirectory of the provided external repository. With the default
node Nix package this is not the case. The nodejs_dev_env workspace used
an intermediate repository rule to fulfill these requirements.
Occasionally, this caused flakiness on CI.

This change fulfills this requirement on the Nix side, so that Bazel can
consume the provided Nix derivation directly.

CHANGELOG_BEGIN
CHANGELOG_END

* Avoid collision with dev-env tools like license-checker

* buildifier-fix

* Add comment motivating use of `@node_nix` on Unix.

* Document npm-cli patch

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-21 17:38:11 +01:00
Robin Krom
ad9325275a
language: put sdk versions into package.json (#4122)
* language: put sdk versions into package.json

The typescript library versions of our support libraries are now given
by the sdk version.

CHANGELOG_BEGIN
CHANGELOG_END

* removed local field

* better placeholders

* consistent SDK_VERSION

* sed sdkversion in test script
2020-01-21 15:41:40 +01:00
Robin Krom
704d576ecd Bazelify daml json types (#4110)
* language: bazel rules for daml-json-types/daml-ledger-fetch

This moves the daml-json-types/daml-ledger-fetch libraries out of the
tests directory and builds them with bazel. We'll rename these libraries
in a follow up PR.

CHANGELOG_BEGIN
CHANGELOG_END

* Update deps.bzl

Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* updated package.json

* rename nodejs patch

* update yarn.lock

* update @bazel/bazel dependency

* wrong typescript version in toplevel package.json

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2020-01-20 15:02:33 +00:00
Robin Krom
0a26591849
upgrading to newest nodejs_rules (#4057)
* upgrading to newest nodejs_rules

CHANGELOG_BEGIN
CHANGELOG_END

* addressing andreas comments
2020-01-16 15:55:32 +01:00
Samir Talwar
34d3abb457 ledger-api-test-tool-on-canton: Upgrade Canton to v0.6.0. (#4068)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-16 11:54:21 +00:00
Andreas Herrmann
91f5dd573f stack update (#3935)
* Remove manual stack update

* Update rules_haskell

* rules_haskell_worker_dependencies after bazel-haskell-deps

* Update rules_haskell Windows patch

* make cabal haddock optional

* Don't generate Haddocks on stack_snapshot

Fails with ghc-lib and takes more time to build.

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-06 10:24:15 +00:00
Oliver Seeliger
b967a7d8c6 Test tool flakiness take2 (#3843)
* Ledger api test tool flakiness fix part 2: Add "eventually"

to synchronize use of waitForParties

* Upgrade to Canton 0.5.2
2019-12-13 13:37:02 +00:00
Samir Talwar
90416744e4 sandbox: Re-add the Docker images with a public base image. (#3814)
* Revert "Remove docker bazel target for sandbox (#3802)"

This reverts commit 48a381ca49.

* sandbox: Use the public `openjdk` Docker image rather than GCR's.

Turns out you need to authenticate with Google Container Registry
regardless of whether the image is public or not. We don't want people
to have to bother logging in to GCR just to get Bazel working.
2019-12-11 16:58:08 +00:00
Samir Talwar
1033e48cb4
Ledger: Documentation for health checks. (#3779)
* ledger: Document the health checks.

* sandbox: Build a Docker image.

* sandbox: Create a sample Kubernetes YAML file.

* sandbox: Add health probes to the sample Kubernetes configuration file.

Startup and liveness are tested with a simple TCP connection to port
6865. Readiness checks are done with `grpc-health-probe`, which is added
to the Sandbox container image.

* sandbox: Link to kubernetes.yaml in the README and provide a disclaimer.

// changelog removed as it's not actually relevant to users

* sandbox: Don't try and build `sandbox-image-base` on Windows.

* Apply suggestions from code review

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2019-12-09 12:43:20 +01:00
Oliver Seeliger
ca47e815f1
Upgrade Canton to 0.5.0 with flaky test fix (#3724)
This release contains a canton flakiness fix to prevent the
occurrence of `Disputed: Failed to select first domain: The
following parti(es) are not available on any connected domain`
during SemanticTest runs.
2019-12-04 10:17:14 +01:00
Moritz Kiefer
f0a111ca1a
Get grpc from nix on unix (#3632)
* Get grpc from nix on unix

The one from Bazel seems to cause linking issues when trying to run
things in GHCi. I’ve spent some time trying to use rules_foreign_cc to
build gRPC using CMake but decided that for my own sanity it’s better
to not pursue that further.

* Address review comments

* Add missing module load

* Cleanup GHCI_SCRIPT

* use the correct file ending on macos

* Import is_linux

* Switch back to grpc-1.23

The newer version seems to cause issues in combination with the java libraries.

* Try to fix package_app on macos

* more debugging

* Maybe this is not necessary, we will never know

* linkers are the worst

* Remove debugging output again

* readd rpaths

* treat libdispatch specially

* remove hack

* more fooling around

* lalala
2019-11-26 18:47:39 +01:00
Andreas Herrmann
f1f41b6e3e Windows: Take stack from dadew (#3596)
rules_haskell looks for stack in PATH. On Windows it is provided by
dadew (i.e. scoop). rules_haskell then symlinks (copies on Windows) the
stack binary. Unfortunately, this breaks with scoop as the shim file is
then not found.
2019-11-22 17:27:42 +00:00
Andreas Herrmann
f4d0eb636e Replace Hazel by stack_snapshot (#2743)
* Unmangled libz.so and libbz2.so

* Use stack_snapshot instead of Hazel

* Remove Hazel

* Define stack_snapshot

* Update rules_haskell

* Document stack_snapshot

* Clean stack's lock file from aborted builds
2019-11-22 14:24:08 +00:00
Oliver Seeliger
15a7f72eae
Upgrade to canton 0.4.1 and reenable multi-ledger test (#3565) 2019-11-21 11:17:44 +01:00
Andreas Herrmann
6c2f4ebe8f Fix pip and pipenv (#3366)
* Fix pip and pipenv

Since the latest nixpkgs update pip is no longer contained in the python
derivation but instead in its own `python37Packages.pip` derivation.

Additionally the pipenv version in nixpkgs is incompatible with the pip
version, as pipenv expects a module to be present which is only
available in later versions. This change overrides pip with a version
update to fix this issue.

Upstream nixpkgs has already updated pip accordingly

* Update nixpkgs

The versions of `pipenv` and `pip` contained in `nixpkgs` where
previously incompatible. They are compatible in the new `nixpkgs`
revision.

Just overriding the pip version caused too many uncached rebuilds and
timed out on CI.

* Update nixpkgs further

To incorporate
efce3c1367

* Try to build fewer GHCs

* nix: drop sass override

nixpkgs-unstable is on the same page.

* nix: use upstream chromedriver

* nix: remove pex override

* nix: drop pypi2nix override

* Fix bazel-watcher

* try upgrading rules go

* Remove overrides for ibazel

* Upgrade rules_go

* Remove references to sass overrides
2019-11-20 15:27:32 +01:00
Moritz Kiefer
1bc4bb76a4 Make AnyChoice and AnyContractKey take template type into account (#3541)
* Make AnyChoice and AnyContractKey take template type into account

fixes #3540

* Update template desugaring

* Switch to proper ghc-lib release
2019-11-20 10:40:14 +00:00
Samir Talwar
fbd6d77e62 ledger-api-test-tool-on-canton: Download Canton from canton.io. (#3533)
So we don't interfere with Canton download metrics.
2019-11-19 17:40:08 +00:00
Martin Huschenbett
24d22012a8 Upgrade to nodejs-10.16.3 on Windows (#3529)
This matches the version we use on Linux.
2019-11-19 15:08:39 +00:00
Moritz Kiefer
1370ef3083 Upgrade ghcide and haskell-lsp (#3499) 2019-11-18 10:56:19 +00:00
Andreas Herrmann
c994703c0c Update rules_haskell (#3473)
* Update rules_haskell & rules_nixpkgs

* Define dadew POSIX toolchain on Windows

* Build hpp with stack and Cabal

* Replace Hazel hpp by @stackage hpp

* replace backslash by forward slash

* Cabal wrapper exclude bindist includes

* ghci-grpc patch fix missing argument

* Switch to rules_haskell master
2019-11-15 10:50:12 +00:00
Samir Talwar
d897b6b55a ledger-api-test-tool-on-canton: Upgrade Canton to v0.4.0. (#3468)
* ledger-api-test-tool-on-canton: Upgrade Canton to v0.4.0.

* workspace: Expose a Java library from the Canton external dependency.`

* workspace: Use double quotes for multiline strings too.
2019-11-14 14:55:38 +00:00
Samir Talwar
c5d9d1014e
ledger-api-test-tool: Run tests on Canton, and fix a few issues. (#3446)
* ledger-api-test-tool-on-canton: Run Canton.

* ledger-api-test-tool-on-canton: Run the SemanticTests against Canton.

This was _so_ much work.

These will probably be flaky, so can't merge them in until we fix the
underlying issues with the tests around multi-participant allocation.

* ledger-api-test-tool: Wait for parties to arrive on all participants.

Thanks to Canton for exacerbating this bug.

Can't turn on all the tests on Canton yet as there are other spurious
failures. Will investigate next.

* ledger-api-test-tool: Move all contract key tests to ContractKeys.

If a ledger doesn't support ContractKeys, they need to be able to turn
these tests off.

* ledger-api-test-tool: If a test name is misspelled, fail immediately.

I keep getting names slightly wrong in the `--exclude` arguments and
wondering why it didn't work.

* client_server: Revert client_server_test.bzl.

Turns out I don't know what I'm doing. ¯\_(ツ)_/¯

* sandbox: Don't call `.toString` on an array. Doesn't do much.

* dev-env: Don't untar netcat automatically.

It needs to be installed with Pacman.
2019-11-13 17:27:28 +01:00
Moritz Kiefer
4ed90187bb Fetch grpc and protobuf Haskell libraries from Hackage (#3444)
* Fetch grpc and protobuf Haskell libraries from Hackage

All the changes that resulted in us fetching them from git, have been
included in the latest Hackage releases.

* Switch back to a custom build file for grpc-haskell-core

* Remove grpc-haskell-core from hazel packages
2019-11-13 11:19:01 +00:00
Andreas Herrmann
964ca64951 Pass -optl-s (#3423) 2019-11-12 08:55:39 +00:00
Moritz Kiefer
7db11ca427 Add toAnyContractKey and fromAnyContractKey (#3408)
* Add toAnyContractKey and fromAnyContractKey

This is necessary to add exerciseByKey to DAML triggers.

* Switch to proper ghc-lib release

* Remove unnecessary filter

* Bump timeout because macos is terrible

* bazel fmt
2019-11-11 17:35:53 +00:00
Andreas Herrmann
33e47828e3
Bazel 1.1 (#3249)
* bazel: 0.28.1 --> 1.1.0

* bazel-watcher sha256

* Fix missing line in patch

* proto_source_root --> strip_import_prefix

See https://github.com/bazelbuild/bazel/issues/7153 for details.

* Update rules_nixpkgs

Required to avoid errors of the form
```
ERROR: An error occurred during the fetch of repository 'node_nix':
   parameter 'sep' may not be specified by name, for call to method split(sep, maxsplit = None) of 'string'
```

and
```
ERROR: An error occurred during the fetch of repository 'node_nix':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
		_execute_or_fail(repository_ctx, <3 more arguments>)
	File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
		fail(<1 more arguments>)

Cannot build Nix attribute 'nodejs'.
Command: [/Users/runner/.nix-profile/bin/nix-build, /private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/node_nix/nix/bazel.nix, "-A", "nodejs", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
src/main/tools/process-tools.cc:173: "setitimer": Invalid argument
```

* Update rules_scala

* .proto has been removed, use [ProtoInfo] instead

See
https://docs.bazel.build/versions/1.1.0/be/protocol-buffer.html#proto_library

* python3_nix add nix_file attribute

To avoid the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/BUILD:66:1: //:nix_python3_runtime depends on @python3_nix//:bin/python in repository @python3_nix which failed to fetch. no such package '@python3_nix//': Traceback (most recent call last):
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
                _execute_or_fail(repository_ctx, <3 more arguments>)
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
                fail(<1 more arguments>)

Cannot build Nix attribute 'python3'.
Command: [/home/aj/.nix-profile/bin/nix-build, "-E", "import <nixpkgs> { config = {}; overlays = []; }", "-A", "python3", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
error: anonymous function at /home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs.nix:3:1 called with unexpected argument 'config', at (string):1:1
```

* rules_haskell unnamed string.split(_, maxsplit = _)

The keyword argument may no longer be named.

* string.replace(_, _, maxsplit = _) may not be named

* Move proto sources from deps to data

Fixes

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/daml-lf/archive/BUILD.bazel:150:1: in deps attribute of scala_test rule //daml-lf/archive:daml_lf_archive_reader_tests_test_suite_src_test_scala_com_digitalasset_daml_lf_archive_DecodeV1Spec.scala: '//daml-lf/archive:daml_lf_1.6_archive_proto_srcs' does not have mandatory providers: 'JavaInfo'. Since this rule was created by the macro 'da_scala_test_suite', the error might have been caused by the macro implementation
```

* Define sha256 for haskell_ghc__paths

Bazel 1.1.0 fails on missing hashes.

* Disable --incompatible_windows_native_test_wrapper

* //compiler/daml-extension don't modify sources

Modifying sources in-place can cause issues on Windows, where build
actions are not sandboxed and changes on sources can affect other build
steps.

* bazel-genfiles --> bazel-bin

The bazel-genfiles symlink has been removed since Bazel 1.0.
See https://github.com/bazelbuild/bazel/issues/8651

* Mark dev_env_tool repository rule as configure

See
https://docs.bazel.build/versions/1.1.0/skylark/lib/globals.html#repository_rule

* Move data deps into data attribute

* Mark dev_env_tool as local = True

* Manually fetch @makensis_dev_env
2019-11-11 10:06:03 +01:00
Moritz Kiefer
028dfcf6d4 Expose projection from contract key to maintainers (#3396) 2019-11-09 18:00:00 +00:00
Andreas Herrmann
6877d5279c
recursively symlink all files in dev_env_tool (#3377)
* dev_env_tool - recursively symlink all files

* dev_env_tool: add prefix attribute

* Replace nodejs by dev_env_tool

* postgresql.exe --> postgres.exe

* Remove unused dev_env_package

And rename dev_env_package --> dev_env_tool

* dev_env_tool macOS compatbility

BSD find does not support `-printf`.

* ReserServiceIT: 4 attempts on Windows
2019-11-08 09:40:26 +01:00
Moritz Kiefer
d7f5b3b436 Add support for on-disk incremental builds in damlc build (#3378)
* Add support for on-disk incremental builds in damlc build

* Normalise file paths of internal modules because Windows

* stop stealing my $s hlint

* Apparently jars are also called exe

* Address review comments

* Bump to proper ghcide revision
2019-11-07 16:56:42 +00:00
Shayne Fletcher
c8a9e0b295 Update hlint; add some dlint smoke tests (#3319) 2019-11-01 18:29:51 +00:00
Shayne Fletcher
11c1bf0fba
Upgrade hlint (#3306) 2019-10-31 15:08:21 -04:00
Rohan Jacob-Rao
00fc6bf100 Log when user views scenario results (#3271)
* Extend Priority type with Telemetry value

* Include Telemetry logging level in logger

* Log viewing of scenario results

* Update ghcide with telemetry logging level
2019-10-28 23:42:20 +00:00