Commit Graph

60 Commits

Author SHA1 Message Date
Robin Krom
4ede3fc711
docs: publish cheat sheet under docs.daml.com/cheat-sheet (#5841)
* docs: publish daml-cheat-sheet on docs.daml.com

* separate cheat sheet rule

* Fixed `@daml-cheat-sheet`

* Uses `_config.yml` to determine the source directory root.
* Uses `tar h` to resolve symbolic links. Otherwise the tarball just
  contains symbolic links to the execroot.
* Uses flags to make `tar` and `gzip` reproducible, i.e. avoid
  timestamps and the like.

* cleanup

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-05-05 17:35:30 +00:00
Sofia Faro
6047cca194
daml-docs: Generate Hoogle database with anchor table from generating docs. (#5704)
* Fix all DAML hoogle links.

This PR changes how the hoogle database is rendered in damldocs, so it has access to the mapping of anchors to URLs that is produced during rendering of rst docs. This way these will not get out of sync.

I also added an --output-anchor option in preparation for fixing external/cross-package references.

changelog_begin
changelog_end

* Update settings.json

undo accidental vscode settings change.

* Update README.md
2020-04-23 16:12:48 +01:00
Sofia Faro
ab93435442
Split standard library docs up one page per module. (#5636)
* One module per page in stdlib docs.

changelog_begin

- [DAML Docs] The standard library docs have been split up per module.

changelog_end

* Adjust stdlib annotations

* Update compiler/damlc/daml-stdlib-src/DA/Generics.daml

Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update compiler/damlc/base-rst-index-template.rst

Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Move to daml/stdlib

* Update daml-doc tests

* Remove the Module prefix in the index/toctree

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-04-22 13:23:46 +01:00
Rohan Jacob-Rao
37ca4af529
create-daml-app: Add end-to-end tests to integration tests (#5540)
This PR gets yarn test running the Puppeteer end-to-end tests in the GSG integration test.

The first step of the test is adding the extra dependencies (Jest, Puppeteer and more). The GSG recommends a yarn add command, but this does not work against HEAD. This is because yarn add does not use resolutions in the parent package.json, and then complains about unknown versions 0.0.0 of the daml TS libaries. The solution here is to hack in the extra dependencies into the ui/package.json and then yarn install. This works, but it hard codes version numbers which we would need to maintain. I would like to be able to say version "latest", which is what yarn add would install.

The next step of the test is to copy the index.test.ts file and run yarn test in CI mode.

I've moved the GSG test to a new create-daml-app-tests target. It's marked "exclusive" in Bazel until we figure out how to avoid hardcoding port numbers. This is a bit tricky since the HTTP port is hardcoded in a couple of places in ui/package.json.

Finally, this PR gets the GSG testing docs to reference the code in the new templates/create-daml-app-test-resources folder.

changelog_begin
changelog_end
2020-04-17 12:42:24 -04:00
Moritz Kiefer
7c031f25f7
Upgrade rules_nodejs to version 1.6.0 (#5539)
* Upgrade rules_nodejs to version 1.6.0

closes #5367

This includes the fixes for the issues in jest that we’ve been seeing.

changelog_begin
changelog_end

* Fix eslint rules

* A bit of progress

* Try to add LinkablePackageInfo (doesn’t seem to work yet)

* Add rootDirs

* revert da_ts_library

* da_ts_library: add LinkablePackageInfo info

* Remove react hook workaround

Since rules_nodejs 1.6.0 this fails with the following error:
```
  ● Test suite failed to run

    Configuration error:

    Could not locate module react mapped as:
    /.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.

    Please check your configuration for these entries:
    {
      "moduleNameMapper": {
        "/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
      },
      "resolver": null
    }

      49 | // like a promis without being one.
      50 | /* eslint-disable @typescript-eslint/no-floating-promises */
    > 51 | var react_1 = __importStar(require("react"));
         |                            ^
      52 | var react_hooks_1 = require("@testing-library/react-hooks");
      53 | var index_1 = __importStar(require("./index"));
      54 | var events_1 = require("events");

      at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
      at Object.<anonymous> (index.test.js:51:28)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```

* rootDirs is not needed for tsc

This is only required for ts_project

* Update yarn Bazel packages

* docs/theme add missing dependencies

* Remove unused attribute module_root

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-17 12:03:33 +00:00
Rohan Jacob-Rao
7fafe6a08a
GSG: test messaging feature and remove copied code snippets (#5507)
* Diff with messaging feature and some noise manually removed

* Bazel target to use patch file in other build targets

* Patch file as data dep for integration tests

* Attempt to patch and test messaging feature in create-daml-app test

changelog_begin
changelog_end

* Use exports_files instead of filegroup

* Remove file existence checks that don't make sense

* Add patch to dev_env and reference it from integration tests

* Include patch on windows for later

* Set up yarn env again after codegen

* Restore file check

* Fix typo in comment on util function

* Add Tasty steps to make process explicit

* Use messaging patch for code snippets in GSG

* Use messaging code from template instead of copy

* Remove copied message code

* Refactor script to copy template code with messaging patch

* No need to retry second yarn install (only local deps should be updated)
2020-04-11 19:53:44 +00:00
Moritz Kiefer
ccacddbbc6
Add documentation for upgrading using DAML script and triggers (#5465)
* Add documentation for upgrading using DAML script and triggers

This PR adds documentation for how you can automate the upgrade
process from the Coin example using DAML Script and DAML Triggers.

Having written this, I think we might want to add some integration
tests for this but it’s rather complex to setup since it ties together
so many pieces so I’d like to leave it out for now. We do at least
test that the code compiles and we all knows that code that compiles
never has bugs.

changelog_begin
changelog_end

* Apply suggestions from code review

Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>

Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
2020-04-07 14:36:40 +02:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Rohan Jacob-Rao
3895c2d01f
Replace quickstart with new getting started guide (#5392)
* Move quickstart to java bindings section

* Change title of quickstart to Getting Started with ...

* Move GSG to Getting Started (and rename in index)

* Rewrite a bunch of references to quickstart

* Update reference and give more specific name

changelog_begin
- [Docs] Replace IOU quickstart with full stack Getting Started Guide
changelog_end

* Replace daml-ledger link

* Use getting started guide in redirects
2020-04-03 15:44:19 +00:00
Rohan Jacob-Rao
6d9db3fea3
GSG: use code samples from the template instead of manual copies (#5396)
* Copy templates into docs build directory to reference in code snippets

changelog_begin
changelog_end

* Remove ui-before code copies

* Rename daml folder to daml-after and remove tags for before code

* Fix a link and wording
2020-04-03 13:18:18 +02:00
Andreas Herrmann
89a9f5c7d2
tarball reproducibility (#5258)
* integration-tests reproducibility

* package-app reproducibility

* Make remaining tar czf reproducible

* package-app

CHANGELOG_BEGIN
CHANGELOG_END

* Reproducibility of remaing tar invocation

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-31 10:09:52 +02:00
Moritz Kiefer
19496482f6
Replace damlpatterns tarballs by a proper template (#5273)
There is no reason why patterns should be a tarball and the current
tarballs are broken anyway. Why do we even have 2 …

changelog_begin
changelog_end
2020-03-30 12:35:22 +00:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
nickchapman-da
d81caa9568
rework daml_compile bazel rule (#5070)
Avoiding `damlc compile/package` commands (which we would like to deprecate), and replace with plain `damlc build` together with a post dar->dalf extraction step in the couple of places where we actually want the .dalf for testing.

changelog_begin
changelog_end
2020-03-18 23:06:55 +00:00
Nemanja
ff926d0a0f
TypeScript lexer for TS syntax highlighting (#4900)
* Resolves #4304

Added highlighting to the application architecture section

Highlighter that works

The latest highlighter

* Added the lexer to pdf and html and restructured how conf.py is called within the theme

* Added copyright headers

* add typescript.py to srcs

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-03-13 16:46:23 +01:00
Robin Krom
8fe497ea5d
lanugage:docs: include generated docs on docs.daml.com (#4924)
This includes the generated docs for the typescipt libraries daml-react,
daml-ledger and daml-types in the documentation presented on
docs.daml.com. Next step is to create better readmes in this libraries.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-10 18:46:54 +01:00
Moritz Kiefer
4510bae9b6
Split upgrade models into a separate package (#4921)
* Split upgrade models into a separate package

This PR splits the upgrade example into 3 packages instead of 2 which
avoids a dependency from the model on the old model. This is explained
in the documentation.

changelog_begin
changelog_end

* Fix typo
2020-03-10 17:24:17 +01:00
nickchapman-da
944a23e302
New bazel rule -- daml_build_test -- to build a DAML project from its daml.yaml config, and setup a bazel test rule to validate the resulting .dar using daml validate-dar. (#4891)
Dependencies on other DAML projects are declared with the `dar_dict` attribute of the build rule. This attribute also declares the names by which the `.dar` files are known in the client project, corresponding to the references in the `daml.yaml` config.

The new rule is used build & test the upgrade documentation example code.

changelog_begin
changelog_end
2020-03-10 14:23:23 +00:00
Gerolf Seitz
2b76d1962c
Deprecate ledger initialization with scenarios (#4864)
* Deprecate ledger initialization with scenarios

CHANGELOG_BEGIN
[Sandbox] Initializing the sandbox with scenarios is now deprecated in
favor of using DAML Script. The scenario parameter will be removed in
the near future. A warning is logged on startup.

The DAML SDK templates and quickstart guide are using DAML Script.

See the DAML Script migration guide for more information:
https://docs.daml.com/daml-script/index.html#using-daml-script-for-ledger-initialization
CHANGELOG_END
2020-03-09 09:30:18 +01:00
nickchapman-da
a64610925c
Extend the documentation for DAML upgrade to shows project setup, build and deploy steps. (#4794)
changelog_begin
changelog_end
2020-03-04 15:23:14 +00:00
Moritz Kiefer
1b5140ed50
Add a first version of new upgrade docs (#4750)
This PR adds a first version of update documentation and removes the
existing docs which were talking about `damlc migrate`.
So far the docs focus on the high-level approach to upgrades taken in
DAML and give an example of how to structure upgrade contracts.

What is not covered so far (and I’d like to leave that for a separate
PR) is:

1. Technical details: How are things split up into packages, which
restrictions apply to data-dependencies, …
2. Deployment and Running the upgrade via triggers/daml script/…
3. Common patterns for handling this in UIs (e.g. “locking” old contracts)

changelog_begin
changelog_end
2020-03-02 09:54:39 +01:00
Moritz Kiefer
8c14d16718
Disable pdf docs builds on macos (#4724)
This disables the PDF docs builds on MacOS on CI (they are still built
locally by default) and removes them from the Nix closure by
introducing a separate ci-cached attribute that filters out texlive.

Since we built `nix-build nix -A tools -A cached` on CI, I’ve also
removed all the Tex stuff from tools which only means that it ends up
in PATH which nobody seems to care about.

changelog_begin
changelog_end
2020-02-26 14:52:08 +00:00
Gary Verhaegen
5a117dc358
introduce new release process (#4513)
Context
=======

After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.

This PR is a proposal for a new release process designed under the
following constraints:

- Reuse as much as possible of the existing infrastructure, to minimize
  effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
  not meant for general public consumption, but can still be used by savvy
  users without jumping through too many extra hoops (ideally just
  swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
  release status, with as few changes as possible in-between, so we can be
  confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
  releases, to avoid discovering non-transient problems while trying to
  promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
  keep the same approval process for SOC2 auditability.

The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:

```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```

This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.

Version numbers
===============

Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:

- Semantic versioning (which we chose as the version format for the SDK
  version number) allows for "prerelease" version numbers as well as
  "metadata"; an example of a complete version string would be
  `1.2.3-nightly.201+server12.43`. The "main" part of the version string
  always has to have 3 numbers separated by dots; the "prerelease"
  (after the `-` but before the `+`) and the "metadata" (after the `+`)
  parts are optional and, if present, must consist of one or more segments
  separated by dots, where a segment can be either a number or an
  alphanumeric string. In terms of ordering, metadata is irrelevant and
  any version with a prerelease string is before the corresponding "main"
  version string alone. Amongst prereleases, segments are compared in
  order with purely numeric ones compared as numbers and mixed ones
  compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
  which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
  `-`, or a transition between a number and a letter. Version strings
  are compared element-wise, with numeric segments being compared as
  numbers. Alphabetic segments are treated specially if they happen to be
  one of a handful of magic words (such as "alpha", "beta" or "snapshot"
  for example) which count as "qualifiers"; a version string with a
  qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
  which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
  special ordering amongst qualifiers. Other alphabetic segments are
  compared alphabetically and count as being "after" their prefix
  (`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
  separated by `.`, plus an optional (though deprecated) alphanumeric
  "version tag" separated by a `-`. I could not find any official
  documentation on ordering for the version tag; numeric segments are
  compared as numbers.
- npm uses semantic versioning so that is covered already.

After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:

```
0.13.53-alpha.20200214.3025.ef5d32b7
```

where the components are, respectively:

- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
  master.
- `3025`: the number of commits in master up to the release commit
  (included). Because we have a linear, append-only master branch, this
  uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
  not strictly speaking necessary, but makes it a lot more convenient to
  identify the commit.

The main downsides of this format are:

1. It is not a valid format for GHC packages. We do not publish GHC
  packages from the SDK (so far we have instead opted to release our
  Haskell code as separate packages entirely), so this should not be an
  issue. However, our SDK version currently leaks to `ghc-pkg` as the
  version string for the stdlib (and prim) packages. This PR addresses
  that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
  would see the above version number as `0.13.53.20200214.3025`, which
  should be enough to uniquely identify it. Note that, as far as I could
  find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
  it makes it more scary. However, I have been told that this may be
  long enough to cause issues on Windows by pushing us past the max path
  size limitation of that "OS". I suggest we try it and see what
  happens.

The upsides are:

- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
  they have the same date, allowing us to release same-day patches if
  needed. (Note: that is, commits that were made on the same day; the
  release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
  immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
  GHC).

Alternatives I considered:

- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
  be accepted by all schemes, but will not sort as expected under semantic
  versioning (though Maven will be fine). I have no idea how it will sort
  under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
  This is not valid semantic versioning and is therefore rejected by
  npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
  what is generally done in the Java world, but we then lose track of what
  version is actually in use and I'm concerned about bug reports. This
  would also not let us publish to the main Maven repo (at least not more
  than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
  version formats. However, it would not clearly indicate to humans that
  it is not meant as a stable version, and would sort differently under
  semantic versioning (which counts it as a prerelease, i.e. before
  `0.13.53`) than under maven (which counts it as a patch, so after
  `0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
  number of prereleases in-between `0.13.52` and the next. This is
  currently the fallback plan if Windows path length causes issues. It
  would be less convenient to map releases to commits, but it could still
  be done via querying the history of the `LATEST` file.

Release notes
=============

> Note: We have decided not to have release notes for snapshot releases.

Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.

But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.

The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.

> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.

New release process
===================

Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.

Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:

1. Before running the release step, CI will checkout the commit
  specified in the LATEST file. This requires separating the release
  step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
  that gets the version to build from an environment variable, with a
  default of `0.0.0` to remain consistent with the current `daml-head`
  behaviour.

Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.

The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.

I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 17:01:23 +01:00
Gary Verhaegen
8811006617
docs cron: more reliable checksums (#4102)
The docs build is currently not reproducible as it include to-the-minute
time-of-build information. It also includes some Sphinx binary caches
which I suppose will also not be reproducible (though I have not checked
the details there).

This commit attempts to remove all sources of non-reproducibility from
the docs build, though this is hard to test without having a stable,
older release to compare with.

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-20 16:21:34 +01:00
Robin Krom
0a26591849
upgrading to newest nodejs_rules (#4057)
* upgrading to newest nodejs_rules

CHANGELOG_BEGIN
CHANGELOG_END

* addressing andreas comments
2020-01-16 15:55:32 +01:00
Gary Verhaegen
878429e3bf
update copyright notices to 2020 (#3939)
copyright update 2020

* update template
* run script: `dade-copyright-headers update .`
* update script
* manual adjustments
* exclude frozen proto files from further header checks (by adding NO_AUTO_COPYRIGHT files)
2020-01-02 21:21:13 +01:00
Moritz Kiefer
ec5d832c48
Generate API docs for DAML script and include them in the SDK docs (#3568)
* Generate API docs for DAML script and include them in the SDK docs

* Update daml-script/daml/Daml/Script.daml

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
2019-11-21 12:59:09 +01:00
Moritz Kiefer
7deca903bb Add documentation for DAML script and bundle it in the SDK (#3527) 2019-11-19 17:19:57 +00:00
Martin Huschenbett
ab07b05fcb Clean up rules_daml (the bazel rules for DAML) (#3495)
There are rules that are used nowhere and seem like old cruft.
2019-11-18 08:12:47 +00:00
Moritz Kiefer
3cd5cfc1d7 Include daml-docs for daml-trigger in the documentation (#3263)
* Include daml-docs for daml-trigger in the documentation

* Fix link to trigger docs
2019-10-28 19:30:23 +00:00
Andreas Herrmann
2bd1db490a
Replace bazel-deps by rules_jvm_external (#3253)
* Update bazel-common to fix javadoc issues

Specifically, to fix the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
        File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
                javadoc_library(name = 'rs-grpc-bridge_javadoc')
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
                dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```

* Define Maven deps using rules_jvm_external

* Pin artifacts

* Remove bazel-deps generated targets

* Remove bazel-deps

* Switch to rules_jvm_external targets

* update bazel documentation

* pom_file: There are no more bazel-deps targets

* BAZEL-JVM.md `maven_install` typo
2019-10-28 13:53:14 +01:00
Moritz Kiefer
e31526c35c Use the same CopyTrigger in the docs and the tests (#3213)
* Use the same CopyTrigger in the docs and the tests

That way, we actually test what we use in our docs which seem like a
much better idea than duplicating the code.

* Add debugging output on failures
2019-10-17 12:52:15 +00:00
Moritz Kiefer
dbddead461
Add a first draft of documentation for DAML triggers (#3181)
* Add a first draft of documentation for DAML triggers

The API will still change in a bunch of ways but I’d rather get some
docs in place now and update them as we change things than not have
any docs at all.

* Fix path to daml.yaml

* s/bot/trigger/

* fix source code markers

* Fix tests

* Update docs/source/triggers/index.rst

Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2019-10-15 16:50:43 +02:00
Remy
e45b585307 daml-lf: repackage daml-lf archive proto (daml_lf -> daml_lf_dev) (#3169) 2019-10-15 09:36:11 +00:00
Remy
d64df4b5f7 daml-lf: clean achive bazel build (#3175)
* clean the mess in daml-lf achive bazel build

* add release note

* Address Moritz's comments

* more release notes
2019-10-15 08:36:46 +00:00
Moritz Kiefer
3554596f9a Wait for all gRPC requests to finish before shutting down gRPC (#2551)
This fixes all flakiness in `damlc test` that I was able to
reproduce. Previously, I got it to fail in about 10% of the cases
whereas now I have successfully run tests 200 times under load without
issues.

There were two issues at play here:

1. We run scenarios in separate threads to be able to kill the running
Shake session quickly even if a scenario has an infinite loop or
something like that (there is a timeout but it’s quite long). This
could result in one of those left-over threads trying to issue a
request while we are already trying to shut down.

To fix that, we wait for the concurrency semaphore to be empty before
shutting down.

2. Just waiting for scenario executions is not quite sufficient as
`runAction` does not wait for all rules to finish (we could just use
runActionSync in `damlc test` but I’d rather make this work
properly). While we do wait for all scenario executions to finish
there is one gRPC request in offInterest that we do not wait for:
gcCtxs.

To fix this, I’ve now routed all gRPC requests through the semaphore
which means that we will also wait for these requests to finish (or
prevent them from spawning).

This makes more sense anyway as scenario executions are mostly fairly
cheap requests while things like setting up the context are expensive
so we want to limit their concurrency.

We should make the concurrency limit configurable but I’ll leave that
for a separate PR.
2019-08-15 09:39:30 +00:00
Moritz Kiefer
ef635ea301
Mark tests that use Haskell gRPC bindings as flaky (#2521)
I’m hoping to investigate this soon but for now this should hopefully
result in less time spend restarting tests.
2019-08-13 20:38:51 +02:00
Gary Verhaegen
99ea93168d
update copyright notices (#2499) 2019-08-13 17:23:03 +01:00
Leonid Shlyapnikov
b940951a76
HTTP JSON API first version (#1994)
* Cleanup

* WIP

* first integration test + fixture

* minor cleanup

* Implementing ContractService.lookup

* Reverting back to endpoints.all (all2 did not work)

* Cleanup

* replace ApiValue ADT with aliases to daml-lf/transaction Value ADT

* porting rest of navigator to LF Value ADT

* Command Service WIP

* CommandService WIP

* porting more of navigator to LF Value ADT

* last error, not first

* rename ApiValueImplicits file

* special conversion features for ImmArray and FrontStack

- just .to[ImmArray] or .to[FrontStack] any random collection

* finish porting most of navigator main code

* use numeric indices for record field name fallback when pretty-printing

* tuples are not serializable

* use numeric indices for label fallback in JSON verbose encoding

* make traverseEitherStrictly more likely to preserve the seq's class

* to shortcut for ImmArraySeq .to[ImmArraySeq]

* compiling, passing navigator backend tests

* test traverseEitherStrictly more, er, strictly

* pass scalacopts through to scaladoc

* deal with unused warning

* remove unneeded function

* simpler error reporting, more private functions in ApiCodecCompressed

* move slowApply to FrontStack, test it so it actually works

* remove unneeded toStrings; better error from impossible ValueTuple case

* scalafmt FrontStackSpec

* support alternative, label-free record JSON encoding

* Adding domain.CreateCommand + corresponding json formats and dummy json format for lav1.value.Record

* CommandService.create should be done... need to test it

* TODO added

* Cleanup

* move ApiCodecCompressed, ApiValueImplicits, and some aliases to new lf-value-json package

* Using tagged TemplateId type instead of Identifier + exercise command WIP

* adapt navigator to moved pieces

* start defining scalacheck extension to ApiCodecCompressedSpec

* CommandService.exercise + introducing CommandMeta

* Adding command endpoints, can't test them yet, need lf value json formats

* fuse some list operations

- suggested by @stefanobaghino-da; thanks

* blue error message

* Minor fixes after merging librify-navigator-json-compressed, #2136

* experiment with an inductive case in TypedValueGenerators

* finish a List case for TypedValueGenerators; it's revealing

* Introducing API value to LF value converter,

CommandsValidator takes IdentifierResolverLike instead of IdentifierResolver

* cleanup

* remove accidentally readded duplicate aliases

* start tying knots in TypedValueGenerators

* verbatim copy ApiCodecCompressedSpec to lf-value-json

* shift some tests from navigator to lf-value-json

* test Optional and Map for ApiCodecCompressed

* heavier random testing of ApiCodecCompressed

* remove unused dependencies from lf-value-json

* adding value json writer

* cleanup

* Revert "cleanup"

This reverts commit 2e4d153f

* fixing the build

* cleanup

* cleaning up imports

* JsValue to API value is done, needs a test

* cleanup

* use scalac -Ypartial-unification in http-json

* simplify some Traverse instances

* factor CreateCommand and ExerciseCommand traverse instances

* Command create integration test WIP

* Command create integration test WIP, got rid of the JsonReader and JsonWriter for the values, converting values explicitly

* Extracting DomainJsonDecoder and DomainJsonEncoder

* LfV refactoring

* Create command serialize/deserialize test works

* cleanup

* resolving conflicts

* More json encode/decode tests

* logging

* command/create passes integration test now

* Adding readme

* grammar

* TODO added

* GetActiveContractsResponse encoding

* ideintifier conversion renaming

* PackageService resolveTemplateId returns domain.TemplateId now

* Resolving LF Identifier instead of Template ID, this should also work for Exercise command decoding

* cleaning up a bit

* daml-lf: show type in TypedValueGenerators-driven errors

* exercise command json encoding/decoding works

* command/exercise IOU_Transfer integration test passes now

* avoid filter for Gens; makes many contract ID gens not fail

* test ApiCodecCompressed against 100 random types, 20 random values each

* Updating README instructions

* improving error handling, failed futures, get logged and reported to the user now as 500

* [ROUTING DSL] Removing routing DSL, it did not work

* getting rid of HttpEntity.Strict match + cleanup

* fixing the merge conflict

* updating README

* use Show.shows instead of new Show

* List(_) isn't checked, but Seq(_) is slightly safer

* improving test assertions

* Adding /contracts/lookup implementation

* http-json: use ImmArraySeq instead of List; use toRightDisjuction

* http-json: .toList.toSet is shorter than fold

* http-json: replace .leftMap.map with .bimap

* http-json: use subst instead of reimplementing JsonFormat

* http-json: remove unused ExceptionHandler

* http-json: safer == comparison

* Adding two test cases for expected errors

* Adding BazelRunfiles.rlocation magic that supposed to handle windows path for bazel dependencies

* http-json: import, not extend
2019-07-29 16:49:57 -04:00
Moritz Kiefer
edff8a416d
Move files in daml-foundations/daml-ghc to compiler/damlc (#2037)
* Move files in daml-foundations/daml-ghc to compiler/damlc

There is also a bit of refactoring going on to actually split things
apart into sensible targets. What is still missing is a cleanup of the
module hierarchy and a cleanup of the test targets but I’ll leave
those for separate PRs.

As a nice bonus, this also reduces dependencies between targets so it
will speed up compiles.

* Update .hie-bios
2019-07-08 17:55:51 +02:00
Andreas Herrmann
df7bff6288 Update to bazel-0.27 (#1957)
* Bazel: 0.24.0 -> 0.27.0

* Update rules_haskell for Bazel 0.27 compatibility

* Update bazel-deps and bazel-watcher

* Windows escape JVM flags

* load commands at top of .bzl file

Bazel 0.27 no longer allows load commands that are not at the beginning
of the file.

* Update Bazel rules

* subpackage boundary

* native is not defined in BUILD files

* yarn: @bazel/hide-bazel-files

Seems to be required since latest rules_nodejs version. Otherwise, yarn
fails with errors about existing BUILD or BUILD.bazel files.

* grpc-java plugin visibility

* Update fat_cc_library

* Nix Python3 toolchain

* Iteration over depset

* dev_env_package: Create symlinks one level deeper

To prevent symlinking the BUILD file as well. The nested BUILD file
confuses Bazel as of 0.27 and rules_nodejs cannot find the node
executable anymore.

* Update rules_nodejs

* Add managed_directories for node_modules

* hie-bios: Extract bazel-genfiles from bazel info

Bazel 0.27 changed the genfiles location which breaks the hie-core test
on macOS.

* update cc_wrapper to Bazel 0.27

* bazel info -> bazel info bazel-genfiles

* Fix typo in BUILD

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2019-07-05 14:04:47 +00:00
Michał Majcherski
2afefb7883
windows: root build (#1894)
* windows: root build

* windows: fixed haskell bindings tests

* windows: disable client_server_test test

* windows: marking daml_test flaky due to #1907

* windows: removing da-hs-damlc-app run from build.ps1

* windows: disable hie-core alias of currently disabled target
2019-06-28 12:55:31 +02:00
Bernhard Elsner
d88b8b471e
Daml intro (#1717)
* DAML Intro Sections 1 - 7

* Apply suggestions from code review

Co-Authored-By: Beth Aitman <bethaitman@users.noreply.github.com>

* Address Beth's comments

* Rebase and fix release notes

* Add suggestions for Haskell reading
2019-06-18 11:56:06 +02:00
Brian Healey
45c1dda78f
correct broken ledger-api-introduction links (#1565)
* correct broken ledger-api-introduction links

* remove extra line break

* add required signatory to bindings-java code snippet
2019-06-07 15:18:21 -04:00
Martin Huschenbett
34d436d77d Drop support for DAML-LF 1.1 from damlc (#1231)
* Drop support for DAML-LF 1.1 from damlc

Part of #853.

* Fix docs test

* Add show party test back

* Update release notes

* Remove target version from Java codegen test
2019-05-27 16:07:35 +00:00
Nick Smith
2636ad0b57
Fixes #1280 docs: Fix the inclusion of the Javadocs into the documentation (#1281) 2019-05-21 14:57:09 +02:00
Leonid Shlyapnikov
b57246642c
Build quickstart-scala as part of the bazel build cycle, #675 (#1138)
Set up exports instead of loading `variables.bzl` with transitive deps
2019-05-16 10:21:52 -04:00
Nick Smith
6f6f3337c7
Fixes 895: Improve DA Bazel rules for building javadocs. (#896)
* Fixes 895: Improve DA Bazel rules for building javadocs.

Extend the da_java_library Bazel macro to also build the Javadoc for the
target. Add the Javadoc artefacts to the release procedure.
2019-05-14 09:40:30 +02:00
Beth Aitman
1b6df13861 Improve docs flow (#1101)
* Improve docs flow

* Move integration kit as Simon requested

* Update daml-foundations/daml-ghc/base-rst-prefix.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
2019-05-13 18:24:18 +00:00
Leonid Shlyapnikov
0ca414f09c
quickstart-scala daml assistant template cleanup (#614) (#1007) 2019-05-08 11:18:22 -04:00