The Daml smart contract language
Go to file
nickchapman-da ee4f89378d
Speedy Tail Call Optimization (#6003)
* Speedy Tail call optimization

The goal of this PR is to achieve Tail call optimization #5767

Tail call optimization means that tail-calls execute without consuming resources. In particular, they must not consume stack space.

Speedy has two stacks: The `env`-stack and the `kontStack`. For an optimized tail call in Speedy, we must not extend either. In Speedy, all function calls are executed via the code `enterFullyAppliedFunction`. The behaviour of this code (prior to this PR) is as follows:

(1) Push the values of all args and free-variables on the env-stack (because that's where the code expects to find them), and
(2) Push a KPop continuation on the kontStack, which will restore the env-stack to its original size before returning to the code which made the function call.

We must stop doing both these things. We achieve this as follows:

(1) We address function args and free-vars via a new machine component: the current `frame`.
(2) We make continuations responsible for restoring their own environment.

As well as achieving proper tail calls, we also gain a performance improvement by (a) removing the many pushes to the env-stack, and (b) never having to push (and then later re-enter) a KPop continuation. The args array and the free-vars array already existed, so there is no additional cost associated with constructing these array. The only extra costs (which are smaller than the gains) are that we must manage the new `frame` component of the machine, and we must record frame/env-size information in continuations so they can restore their environment.

To make use of the frame, we need to identify (at compile time) the run-time location for every variable in a speedy expression. This is done during the `closureConvert` phase. At run-time, an environment is now composed of both the existing env-stack and the frame. The only values which now live on the env-stack are those introduced by let-bindings and pattern-match-destructuring. All other are found in the frame.

Changes to SEExpr:
- Introduce a new expression form `SELoc`, with 3 sub classes: SELocS/SELocA/SELocF to represent the run-time location of a variable.
- SELocS/A/F execute by calling corresponding lookup function in Speedy: getEnv(Stack,Arg,Free).
- SEMakeClo takes a list of SELoc instead of list of int.
- During closure conversion all SEVar are replaced by an SELocS/A/F.
- SEVar are not allowed to exist at run-time (just as SEAbs may not exist).
- We adapt the synthesised code for SEBuiltinRecursiveDefinition: FoldL, FoldR, EqualList

It is worth noting the prior code also had the notion of before/after closureConvert, but SEVar was used for both meanings: Prior to closureConvert it meant the relative-index of the enclosing binder (lambda,let,..). After closureConvert it meant the relative-offset from the top of the env-stack where the value would be found at run-time. These are not quite the same! Now we have different sub-types (SEVar vs SELoc), this change of mode is made more explicit.

Run-time changes:
- Use the existing `KFun` continuation as the new `Frame` component.
- `KFun` allows access to both the args of the current application, and the free-vars of the current closure.
- A variable is looked up by it's run-time location (SELocS/A/F)
- A function application is executed (`enterFullyAppliedFunction`), by setting the machine's `frame` component to the new current `KFun`.
- When a continuation (KArg, KMatch, KPushTo, KCatch) is pushed, we record the current Frame and current stack depth within the continuation, so when it is entered, it can call `restoreEnv` to restore the environment to the state when the continuation was pushed.

Changes to Compiler:

- The required changes are to the `closureConvert` and `validate`.
- `closureConvert` `remaps` is now a `Map` from the `SEVar`s relative-index to `SELoc`
- `validate` now tracks 3-ints (maxS,masA,maxF)

changelog_begin
changelog_end

* changes for Remy

* Changes for Martin

* test designed explicitly to blow if the free variables are captured incorrectly

* address more comments

* improve comment about shift in Compiler
2020-05-20 12:37:52 +01:00
.github check for changelog (#3963) 2020-01-07 17:19:50 +01:00
.vscode open-sourcing daml 2019-04-04 09:33:38 +01:00
3rdparty update copyright notices to 2020 (#3939) 2020-01-02 21:21:13 +01:00
bazel_tools Enforce a dependency on node_nix (#6035) 2020-05-19 16:58:00 +00:00
build-scripts replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
ci trigger all releases from master (#6016) 2020-05-19 19:18:10 +02:00
compatibility Enforce a dependency on node_nix (#6035) 2020-05-19 16:58:00 +00:00
compiler WIP: Draft version constraint generation (#5472) 2020-05-20 08:08:18 +02:00
daml-assistant CI: Run PostgreSQL once for all Scala tests. (#5919) 2020-05-14 09:06:34 +02:00
daml-lf Speedy Tail Call Optimization (#6003) 2020-05-20 12:37:52 +01:00
daml-script Speedy Tail Call Optimization (#6003) 2020-05-20 12:37:52 +01:00
dev-env Include Bazel patch to cache exclusive tests on Windows (#6009) 2020-05-18 16:01:29 +00:00
docs Forward port release notes for 1.0.1 and 1.1.0 (#5962) 2020-05-13 14:34:00 +02:00
extractor CI: Run PostgreSQL once for all Scala tests. (#5919) 2020-05-14 09:06:34 +02:00
ghc-lib update copyright notices to 2020 (#3939) 2020-01-02 21:21:13 +01:00
infra Fix launchd killing VMWare process at end of script execution (#6006) 2020-05-18 10:54:15 -04:00
language-support language: send live event when reconnecting (#5982) 2020-05-19 16:57:16 +02:00
ledger fix ACS race condition in sandbox-classic; enable random Websocket testing (#5969) 2020-05-19 13:26:37 -04:00
ledger-api Document required protobuf definitions for bindings (#5664) 2020-04-22 10:27:23 +00:00
ledger-service fix ACS race condition in sandbox-classic; enable random Websocket testing (#5969) 2020-05-19 13:26:37 -04:00
libs-haskell Make daml start work with --sandbox-port=0 (#5630) 2020-04-20 17:07:30 +02:00
libs-scala Add put and getIfPresent methods to Cache (#6007) 2020-05-18 11:53:08 +00:00
navigator Show hidden navigator server CLI option (#5667) 2020-04-22 12:02:12 +02:00
nix WIP: Draft version constraint generation (#5472) 2020-05-20 08:08:18 +02:00
oss-compliance replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
release trigger all releases from master (#6016) 2020-05-19 19:18:10 +02:00
replacements replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
rules_daml Fix find: ./local-spawn-runner not found issue (#5793) 2020-04-30 15:09:04 +02:00
scala-protoc-plugins Use com.daml as root package (#5343) 2020-04-05 19:49:57 +02:00
templates Bump timeout in create-daml-app tests (#5928) 2020-05-11 11:40:47 +00:00
triggers Implement a simple profiler for DAML scenarios (#5957) 2020-05-18 17:20:44 +00:00
.bazelignore Ignore compatibility directory (#5687) 2020-04-23 10:33:38 +00:00
.bazelrc Configure Bazel repository cache for build (#5733) 2020-04-27 10:10:21 +00:00
.dadew patching Bazel on Windows (infra bits, no patch yet) (#5918) 2020-05-12 23:16:04 +02:00
.dockerignore open-sourcing daml 2019-04-04 09:33:38 +01:00
.envrc open-sourcing daml 2019-04-04 09:33:38 +01:00
.gitattributes Remove unreleased.rst (#3547) 2019-11-20 15:16:57 +00:00
.gitignore Include the first feature in compat tests for create-daml-app (#6002) 2020-05-18 14:37:16 +02:00
.hie-bios Update rules_haskell (#4751) 2020-03-13 16:49:34 +01:00
.hlint.yaml replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
.mergify.yml replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
.scalafmt.conf Revert "Adopt trailingCommas option in scalafmt (#4214)" (#4257) 2020-01-29 11:43:35 +00:00
.sha256 Improve performance of stakeholder retrieval from SQL (#3207) 2019-10-28 14:16:21 +01:00
.watchmanconfig open-sourcing daml 2019-04-04 09:33:38 +01:00
azure-cron.yml add default machine capability (#5912) 2020-05-09 18:21:42 +02:00
azure-pipelines.yml fix release diff (#6042) 2020-05-20 11:58:33 +02:00
BAZEL-bash.md add Bazel bash help (#2223) 2019-07-19 15:31:45 +00:00
bazel-haskell-deps.bzl WIP: Draft version constraint generation (#5472) 2020-05-20 08:08:18 +02:00
BAZEL-haskell.md Remove bzip2 library (#5105) 2020-03-20 15:05:33 +01:00
bazel-java-deps.bzl Upgrade scala compiler silencer to 1.6.0 (#5822) 2020-05-04 21:30:46 +00:00
BAZEL-JVM.md Use com.daml as root package (#5343) 2020-04-05 19:49:57 +02:00
BAZEL.md Document Nix cleanup (#5907) 2020-05-08 11:12:43 +02:00
BUILD Configure python toolchain via nixpkgs_python_configure (#5702) 2020-04-23 17:30:32 +02:00
build.ps1 Publish Bazel logs (#5821) 2020-05-04 11:14:40 +02:00
build.sh CI: Run PostgreSQL once for all Scala tests. (#5919) 2020-05-14 09:06:34 +02:00
CHANGELOG Move unreleased user-facing features to its own file (#1762) 2019-06-19 16:32:03 +02:00
CODE_OF_CONDUCT.md open-sourcing daml 2019-04-04 09:33:38 +01:00
CODEOWNERS Make Remy and Martin codeowner of everything interpreter (#5952) 2020-05-13 10:51:28 +00:00
CONTRIBUTING.md Add a reason text field to RejectReason.Inconsistent (#5180) (#5815) 2020-05-04 12:54:17 +01:00
COPY replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
daml-logo.png #2785 add daml_logo.png for README (#2787) 2019-09-06 09:35:32 +02:00
deps.bzl Enforce a dependency on node_nix (#6035) 2020-05-19 16:58:00 +00:00
dotfiles open-sourcing daml 2019-04-04 09:33:38 +01:00
fmt.sh Fix ./fmt.sh check (#5718) 2020-04-24 12:04:14 +00:00
LATEST trigger all releases from master (#6016) 2020-05-19 19:18:10 +02:00
LICENSE update copyright notices to 2020 (#3939) 2020-01-02 21:21:13 +01:00
maven_install.json Upgrade scala compiler silencer to 1.6.0 (#5822) 2020-05-04 21:30:46 +00:00
NOTICES remove MissingH (#3948) 2020-01-06 14:36:14 +01:00
package.json Upgrade rules_nodejs to version 1.6.0 (#5539) 2020-04-17 12:03:33 +00:00
README.md update copyright notices to 2020 (#3939) 2020-01-02 21:21:13 +01:00
release.sh trigger all releases from master (#6016) 2020-05-19 19:18:10 +02:00
report-std-change.sh replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
SECURITY.md Create SECURITY.md (#1357) 2019-05-31 08:58:21 +01:00
stack-snapshot.yaml WIP: Draft version constraint generation (#5472) 2020-05-20 08:08:18 +02:00
tsconfig.json open-sourcing daml 2019-04-04 09:33:38 +01:00
unreleased.rst replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
unreleased.sh replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
Upgrading.md Replace Hazel by stack_snapshot (#2743) 2019-11-22 14:24:08 +00:00
WORKSPACE WIP: Draft version constraint generation (#5472) 2020-05-20 08:08:18 +02:00
workspace_status.sh fix sitemap generation (#5775) 2020-04-30 15:02:08 +02:00
yarn.lock Upgrade rules_nodejs to version 1.6.0 (#5539) 2020-04-17 12:03:33 +00:00

DAML logo

Download License Build

Copyright 2020 Digital Asset (Switzerland) GmbH and/or its affiliates. All Rights Reserved. SPDX-License-Identifier: Apache-2.0

Welcome to the DAML repository!

This repository hosts all code for the DAML smart contract language and SDK, originally created by Digital Asset. DAML is an open-source smart contract language for building future-proof distributed applications on a safe, privacy-aware runtime. The DAML SDK is a set of tools to help you develop applications based on DAML.

Using DAML

To download DAML, follow the installation instructions. Once installed, to try it out, follow the quickstart guide.

If you have questions about how to use DAML or how to build DAML-based solutions, please ask them on StackOverflow using the daml tag.

Contributing to DAML

We warmly welcome contributions. If you are looking for ideas on how to contribute, please browse our issues. To build and test DAML:

1. Clone this repository

git clone git@github.com:digital-asset/daml.git
cd daml

2. Set up the development dependencies

Our builds require various development dependencies (e.g. Java, Bazel, Python), provided by a tool called dev-env.

Linux and Mac

On Linux and Mac dev-env can be installed with:

  1. Install Nix by running: bash <(curl https://nixos.org/nix/install)
  2. Enter dev-env by running: eval "$(dev-env/bin/dade assist)"

If you don't want to enter dev-env manually each time using eval "$(dev-env/bin/dade assist)", you can also install direnv. This repo already provides a .envrc file, with an option to add more in a .envrc.private file.

Windows

On Windows you need to enable long file paths by running the following command in an admin powershell:

Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem' -Name LongPathsEnabled -Type DWord -Value 1

Then start dev-env from PowerShell with:

.\dev-env\windows\bin\dadew.ps1 install
.\dev-env\windows\bin\dadew.ps1 sync
.\dev-env\windows\bin\dadew.ps1 enable

In all new PowerShell processes started, you need to repeat the enable step.

3. First build and test

We have a single script to build most targets and run the tests. On Linux and Mac run ./build.sh. On Windows run .\build.ps1. Note that these scripts may take over an hour the first time.

To just build do bazel build //..., and to just test do bazel test //.... To read more about Bazel and how to use it, see the Bazel site.

On Mac if building is causing trouble complaining about missing nix packages, you can try first running nix-build -A tools -A cached nix repeatedly until it completes without error.

4. Installing a local copy

On Linux and Mac run daml-sdk-head which installs a version of the SDK with version number 0.0.0. Set the version: field in any DAML project to 0.0.0 and it will use the locally installed one.

On Windows:

bazel build //release:sdk-release-tarball
tar -vxf .\bazel-bin\release\sdk-release-tarball.tar.gz
cd sdk-*
daml\daml.exe install . --activate

That should tell you what to put in the path, something along the lines of C:\Users\admin\AppData\Roaming\daml\bin. Note that the Windows build is not yet fully functional.

Caching: build speed and disk space considerations

Bazel has a lot of nice properties, but they come at the cost of frequently rebuilding "the world". To make that bearable, we make extensive use of caching. Most artifacts should be cached in our CDN, which is configured in .bazelrc in this project.

However, even then, you may end up spending a lot of time (and bandwidth!) downloading artifacts from the CDN. To alleviate that, by default, our build will create a subfolder .bazel-cache in this project and keep an on-disk cache. This can take about 10GB at the time of writing.

To disable the disk cache, remove the following lines:

build:linux --disk_cache=.bazel-cache
build:darwin --disk_cache=.bazel-cache

from the .bazelrc file.

If you work with multiple copies of this repository, you can point all of them to the same disk cache by overwriting these configs in either a .bazelrc.local file in each copy, or a ~/.bazelrc file in your home directory.

Haskell profiling builds

To build Haskell executables with profiling enabled, pass -c dbg to Bazel, e.g. bazel build -c dbg damlc. If you want to build the whole SDK with profiling enabled use daml-sdk-head --profiling.