The Daml smart contract language
Go to file
Stephen Compall 2893e26872
fix MAXIMUM_NUMBER_OF_STREAMS errors when doing many websocket queries (#15733)
* log more termination

From the timeout loop:

+ fmm-outer
+ fmm-inner
x ACS-before-tx
x tx-after-ACS

* spam eagerCancel=true and see what happens

From the timeout loop:

+ after-split
+ IDSS-outer
+ fmm-outer
+ contractsAndBoundary
+ tx-after-ACS
+ fmm-inner
+ GTSFP-outer
x ACS-before-tx

* passing acs-and-tx tests

* trying combinations of reverting eagerCancel settings

- setting eagerCancel = false in acsAndBoundary causes the ACS
  cancellation to fail (first test), but the tx cancellation still
  succeeds

- setting eagerCancel = false in project2 causes both the ACS and tx
  stream cancellation tests (first and third tests) to fail

- the offset broadcast in acsFollowingAndBoundary appears to be
  redundant with respect to cancellation, so we revert it in the
  interest of conservatism

* make test size small

* current measurement

Still fine after the refactoring of logTermination and removal of fmm-*.

+ GTSFP-outer
+ contractsAndBoundary
x IDSS-outer-2
+ after-split
+ tx-after-ACS
+ IDSS-outer-1
x ACS-before-tx

* set level of the logTermination messages to trace
2022-12-16 16:41:08 -05:00
.github Retire changelog entries mechanism (#15423) 2022-11-02 14:58:46 +00:00
.vscode Remove .vscode/settings.json (#13008) 2022-02-22 11:42:45 +01:00
3rdparty Further copyright updates (#12249) 2022-01-04 16:32:17 +01:00
bazel_tools Custom error / fix error for wrong expected return type of a choice. (#15855) 2022-12-13 18:08:32 +00:00
build-scripts hunt down DAML (#13191) 2022-03-08 14:25:25 +01:00
ci faster cache fix (#15824) 2022-12-07 17:20:34 +01:00
compatibility update compat versions (2022-12-08) (#15830) 2022-12-08 11:28:53 +01:00
compiler Remove "Module" from daml-doc H1 headers (#15862) 2022-12-16 10:19:08 +00:00
daml-assistant remove dead code (#15739) 2022-11-30 12:06:46 +00:00
daml-lf [Speedy] create Scenario Machine (#15588) 2022-12-16 08:20:42 +00:00
daml-script [Speedy] create Scenario Machine (#15588) 2022-12-16 08:20:42 +00:00
dev-env Add pre-commit config and install hook in dev-env (#15637) 2022-12-07 13:09:50 +00:00
docs Add metrics to execution services [PLEN-64] (#15861) 2022-12-15 15:34:25 +00:00
ghc-lib Fix erroneous code action/lens suggestions in Daml Studio (#15731) 2022-12-13 12:06:40 +00:00
infra remove Victor's access to data bucket (#15629) 2022-11-24 14:47:37 +01:00
language-support Conversion between javaapi.data.Value and lf.value.Value (#15885) 2022-12-15 19:27:55 +00:00
ledger Flaky test fixes for Canton deduplication tests (#15914) 2022-12-16 20:33:18 +00:00
ledger-api Metering clarifications for Daml Hub [DPP-1314] (#15898) 2022-12-16 10:46:32 +00:00
ledger-service fix MAXIMUM_NUMBER_OF_STREAMS errors when doing many websocket queries (#15733) 2022-12-16 16:41:08 -05:00
libs-haskell Clean up uses of locateRunfiles (#15446) 2022-11-10 13:20:43 +01:00
libs-scala Metering clarifications for Daml Hub [DPP-1314] (#15898) 2022-12-16 10:46:32 +00:00
navigator update qs, express for security (#15874) 2022-12-12 15:52:19 -05:00
nix Add pre-commit config and install hook in dev-env (#15637) 2022-12-07 13:09:50 +00:00
observability Generate doc instances for metrics groupable by name [DPP-1355]. (#15908) 2022-12-16 13:26:11 +00:00
release Metering clarifications for Daml Hub [DPP-1314] (#15898) 2022-12-16 10:46:32 +00:00
replacements update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
rules_daml review daml-intro-10 (#15268) 2022-10-18 17:10:50 +02:00
runtime-components Remove overloaded methods in command client (#15519) 2022-11-23 16:10:06 +01:00
scala-protoc-plugins/scalapb Drop gRPC server side custom codegen (#15763) 2022-12-12 08:35:56 +01:00
security Generate CSV file from all test evidences for ledger client components (#15372) 2022-11-29 12:35:22 +01:00
templates bump canton to 20221213 (#15886) 2022-12-15 09:25:54 +00:00
test-evidence Refine "component" of asset field (#15833) 2022-12-13 10:15:00 +01:00
triggers [Speedy] create Scenario Machine (#15588) 2022-12-16 08:20:42 +00:00
.bazelignore Remove redundant clean --expunge (#6890) 2020-07-28 09:49:36 +02:00
.bazelrc remove workspace_status.sh (#13866) 2022-05-12 12:41:27 +02:00
.dadew fix gsg on node 18 (#15373) 2022-11-28 11:52:47 +01:00
.envrc Use POSIX compliant check for DADE_NO_PRE_COMMIT (#15827) 2022-12-07 22:10:18 +01:00
.gitattributes Remove unreleased.rst (#3547) 2019-11-20 15:16:57 +00:00
.gitignore hunt down DAML (#13191) 2022-03-08 14:25:25 +01:00
.hie-bios Add da-hls helper (#13853) 2022-05-12 10:53:10 +02:00
.hlint.yaml Clean up uses of locateRunfiles (#15446) 2022-11-10 13:20:43 +01:00
.pre-commit-config.yaml Add pre-commit config and install hook in dev-env (#15637) 2022-12-07 13:09:50 +00:00
.prettierignore Add da-hls helper (#13853) 2022-05-12 10:53:10 +02:00
.prettierrc.yaml Add autoformatting to ts bindings & navigator frontend (#12693) 2022-02-11 11:44:52 +00:00
.scalafmt.conf Update scalafmt to 3.5.1. (#13584) 2022-04-28 09:57:04 +00:00
azure-cron.yml run fix_bazel_cache on cron (#15575) 2022-11-15 13:07:41 +00:00
azure-pipelines.yml restore compat update (#13663) 2022-04-22 17:28:16 +02:00
BAZEL-bash.md add Bazel bash help (#2223) 2019-07-19 15:31:45 +00:00
BAZEL-go.md Update protobuf docs plugin (#11880) 2021-11-25 20:51:24 +00:00
bazel-haskell-deps.bzl Fix erroneous code action/lens suggestions in Daml Studio (#15731) 2022-12-13 12:06:40 +00:00
BAZEL-haskell.md Add note on using da-hls on other bazel targets (#13868) 2022-05-12 13:39:46 +02:00
bazel-java-deps.bzl Bump Protobuf (#15866) 2022-12-12 18:02:21 +00:00
BAZEL-JVM.md Support for golden signals metrics on akka-http APIs [PLEN-12] (#15310) 2022-11-09 16:55:12 +01:00
BAZEL-oracle.md Write a few lines on how to run tests against an Oracle database (#10230) 2021-07-09 14:14:03 +02:00
BAZEL.md Added a note re: nix re-install, cache issues. (#15469) 2022-11-10 14:39:01 +01:00
buf-ledger-api.yaml Metering clarifications for Daml Hub [DPP-1314] (#15898) 2022-12-16 10:46:32 +00:00
buf-ledger-configuration.yaml Remove kvutils buf configuration and other leftovers [KVL-1427] (#13732) 2022-04-28 16:00:23 +02:00
buf-lf-transaction.yaml Remove kvutils buf configuration and other leftovers [KVL-1427] (#13732) 2022-04-28 16:00:23 +02:00
buf-participant-integration-api.yaml update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
BUILD Add pre-commit config and install hook in dev-env (#15637) 2022-12-07 13:09:50 +00:00
build.ps1 Drop dadew uninstall (#12566) 2022-01-25 12:11:16 +00:00
build.sh build.sh: fix $1 (#14764) 2022-08-19 10:46:33 +02:00
canton_dep.bzl bump canton to 20221215 (#15910) 2022-12-16 08:46:09 +01:00
CHANGELOG Retire changelog entries mechanism (#15423) 2022-11-02 14:58:46 +00:00
CODE_OF_CONDUCT.md open-sourcing daml 2019-04-04 09:33:38 +01:00
CODEOWNERS Add platform enablement team as code owner to observability (#15893) 2022-12-14 18:03:48 +00:00
CONTRIBUTING.md Retire changelog entries mechanism (#15423) 2022-11-02 14:58:46 +00:00
COPY update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
daml-logo.png Replace Daml logo on README (#8756) 2021-02-04 16:28:52 +01:00
deps.bzl Bump Protobuf (#15866) 2022-12-12 18:02:21 +00:00
fmt.sh fmt: add --diff to help output (#14734) 2022-08-17 13:10:51 +02:00
go_deps.bzl update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
go.mod Update protobuf docs plugin (#11880) 2021-11-25 20:51:24 +00:00
go.sum Update protobuf docs plugin (#11880) 2021-11-25 20:51:24 +00:00
hls_snapshot.json Remove HLS plugins ormolu and fourmolu (#15110) 2022-09-28 09:57:16 +02:00
hls-snapshot.yaml Remove HLS plugins ormolu and fourmolu (#15110) 2022-09-28 09:57:16 +02:00
LATEST start 2.3.8 process (#15892) 2022-12-15 16:49:18 +01:00
LICENSE Further copyright updates (#12249) 2022-01-04 16:32:17 +01:00
maven_install_2.13.json Bump Protobuf (#15866) 2022-12-12 18:02:21 +00:00
NIGHTLY_PREFIX bump nightly (#15741) 2022-11-30 16:00:12 +01:00
NOTICES update NOTICES file (#15888) 2022-12-14 17:29:59 +01:00
NOTICES.md Fix typo in NOTICES.md (#12884) 2022-02-11 08:19:26 +00:00
package.json update qs, express for security (#15874) 2022-12-12 15:52:19 -05:00
Pipfile add blackduck scan to run on master (#6130) (#8161) 2020-12-07 19:59:39 +00:00
README.md Display latest version by semantic version (#14320) 2022-07-01 13:32:13 +00:00
release.sh update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
security-evidence.md [Engine] compute agreement as part of contract caching (#15901) 2022-12-15 11:29:06 +00:00
SECURITY.md Daml case and logo (#8433) 2021-01-08 12:50:15 +00:00
shell.nix Use POSIX compliant check for DADE_NO_PRE_COMMIT (#15827) 2022-12-07 22:10:18 +01:00
stack-snapshot.yaml Drop patch for mod package in lsp-types (#12921) 2022-02-15 10:24:05 +01:00
stackage_snapshot_windows.json Drop patch for mod package in lsp-types (#12921) 2022-02-15 10:24:05 +01:00
stackage_snapshot.json Drop patch for mod package in lsp-types (#12921) 2022-02-15 10:24:05 +01:00
tsconfig.json open-sourcing daml 2019-04-04 09:33:38 +01:00
unreleased.sh unreleased: skip blank lines (#13828) 2022-05-10 11:00:19 +02:00
Upgrading.md Upgrade Nixpkgs [KVL-1045] (#10624) 2021-08-20 12:34:19 +00:00
WORKSPACE fix gsg on node 18 (#15373) 2022-11-28 11:52:47 +01:00
yarn.lock update qs, express for security (#15874) 2022-12-12 15:52:19 -05:00

Daml logo

Download License Build

Copyright (c) 2022 Digital Asset (Switzerland) GmbH and/or its affiliates. All Rights Reserved. SPDX-License-Identifier: Apache-2.0

Welcome to the Daml repository!

This repository hosts all code for the Daml smart contract language and SDK, originally created by Digital Asset. Daml is an open-source smart contract language for building future-proof distributed applications on a safe, privacy-aware runtime. The SDK is a set of tools to help you develop applications based on Daml.

Using Daml

To download Daml, follow the installation instructions. Once installed, to try it out, follow the quickstart guide.

If you have questions about how to use Daml or how to build Daml-based solutions, please join us on the Daml forum. Alternatively, if you prefer asking on StackOverflow, please use the daml tag.

Contributing to Daml

We warmly welcome contributions. If you are looking for ideas on how to contribute, please browse our issues. To build and test Daml:

1. Clone this repository

git clone git@github.com:digital-asset/daml.git
cd daml

2. Set up the development dependencies

Our builds require various development dependencies (e.g. Java, Bazel, Python), provided by a tool called dev-env.

Linux

On Linux dev-env can be installed with:

  1. Install Nix by running: bash <(curl -sSfL https://nixos.org/nix/install)
  2. Enter dev-env by running: eval "$(dev-env/bin/dade assist)"

If you don't want to enter dev-env manually each time using eval "$(dev-env/bin/dade assist)", you can also install direnv. This repo already provides a .envrc file, with an option to add more in a .envrc.private file.

Mac

On Mac dev-env can be installed with:

  1. Install Nix by running: bash <(curl -sSfL https://nixos.org/nix/install)

    • This is a multi-user installation (there is no single-user installation option for macOS). Because of this, you need to configure /etc/nix/nix.conf to use Nix caches:
    1. Add yourself as a nix trusted user by running echo "extra-trusted-users = $USER" | sudo tee -a /etc/nix/nix.conf

    2. Restart the nix-daemon by running sudo launchctl stop org.nixos.nix-daemon && sudo launchctl start org.nixos.nix-daemon

  2. Enter dev-env by running: eval "$(dev-env/bin/dade assist)"

If you don't want to enter dev-env manually each time using eval "$(dev-env/bin/dade assist)", you can also install direnv. This repo already provides a .envrc file, with an option to add more in a .envrc.private file.

Note that after a macOS update it can appear as if Nix is not installed. This is because macOS updates can modify shell config files in /etc, which the multi-user installation of Nix modifies as well. A workaround for this problem is to add the following to your shell config file in your $HOME directory:

# Nix
if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
  . '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
fi
# End Nix

See https://github.com/NixOS/nix/issues/3616 for more information about this issue.

MacOS M1

The above procedure will use and build native arm64 M1 binaries for this project. However, note that at the time of writing the CI system of the Daml project does not yet include MacOS M1 nodes. Therefore, the M1 configuration is untested on CI, and the remote cache is not populated with native M1 artifacts.

If you encounter issues with a native M1 build, then you can configure project to build x86-64 binaries instead and run them through Rosetta. To do that replace the contents of the file nix/system.nix with the following content:

"x86_64-darwin"

Windows

On Windows you need to enable long file paths by running the following command in an admin powershell:

Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem' -Name LongPathsEnabled -Type DWord -Value 1

You also need to configure Bazel for Windows:

echo "build --config windows" > .bazelrc.local

Note, if you are on a Windows ad-hoc or CI machine you can use ci/configure-bazel.sh instead of performing these steps manually. In that case, you should checkout the daml repository into the path D:\a\1\s in order to be able to use remote cache artifacts.

Then start dev-env from PowerShell with:

.\dev-env\windows\bin\dadew.ps1 install
.\dev-env\windows\bin\dadew.ps1 sync
.\dev-env\windows\bin\dadew.ps1 enable

In all new PowerShell processes started, you need to repeat the enable step.

3. First build and test

We have a single script to build most targets and run the tests. On Linux and Mac run ./build.sh. On Windows run .\build.ps1. Note that these scripts may take over an hour the first time.

To just build do bazel build //..., and to just test do bazel test //.... To read more about Bazel and how to use it, see the Bazel site.

On Mac if building is causing trouble complaining about missing nix packages, you can try first running nix-build -A tools -A cached nix repeatedly until it completes without error.

4. Installing a local copy

On Linux and Mac run daml-sdk-head which installs a version of the SDK with version number 0.0.0. Set the version: field in any Daml project to 0.0.0 and it will use the locally installed one.

On Windows:

bazel build //release:sdk-release-tarball
tar -vxf .\bazel-bin\release\sdk-release-tarball-ce.tar.gz
cd sdk-*
daml\daml.exe install . --install-assistant=yes

That should tell you what to put in the path, something along the lines of C:\Users\admin\AppData\Roaming\daml\bin. Note that the Windows build is not yet fully functional.

Caching: build speed and disk space considerations

Bazel has a lot of nice properties, but they come at the cost of frequently rebuilding "the world". To make that bearable, we make extensive use of caching. Most artifacts should be cached in our CDN, which is configured in .bazelrc in this project.

However, even then, you may end up spending a lot of time (and bandwidth!) downloading artifacts from the CDN. To alleviate that, by default, our build will create a subfolder .bazel-cache in this project and keep an on-disk cache. This can take about 10GB at the time of writing.

To disable the disk cache, remove the following lines:

build:linux --disk_cache=.bazel-cache
build:darwin --disk_cache=.bazel-cache

from the .bazelrc file.

If you work with multiple copies of this repository, you can point all of them to the same disk cache by overwriting these configs in either a .bazelrc.local file in each copy, or a ~/.bazelrc file in your home directory.

Shared memory segment issues

On macOS at least, it looks like our setup does not always properly close the resources PostgreSQL uses. After a number of test runs, you may encounter an error message along the lines of:

FATAL:  could not create shared memory segment: No space left on device
DETAIL:  Failed system call was shmget(key=5432001, size=56, 03600).
HINT:  This error does *not* mean that you have run out of disk space. It occurs either if all available shared memory IDs have been taken, in which case you need to raise the SHMMNI parameter in your kernel, or because the system's overall limit for shared memory has been reached.
        The PostgreSQL documentation contains more information about shared memory configuration.
child process exited with exit code 1

In this case, this is a memory leak, so increasing SHMNI (or SHMALL etc.) as suggested will only delay the issue. You can look at the existing shared memory segments on your system by running ipcs -mcopt; this will print a line per segment, indicating the process ID of the last process to connect to the segment as well as the last access time and the number of currently connected processes.

If you identify segments with no connected processes, and you are confident you can remove them, you can do so with ipcrm $sid, where $sid is the process ID displayed (as the second column) by ipcs. Not many macOS applications use shared memory segments; if you have verified that all the existing memory segments on your machine need to be deleted, e.g. because they have all been created by PostgreSQL instances that are no longer running, here is a Bash invocation you can use to remove all shared memory segments from your system.

This is a dangerous command. Make sure you understand what it does before running it.

for shmid in $(ipcs -m | sed 1,3d | awk '{print $2}' | sed '$d'); do ipcrm -m $shmid; done

Haskell profiling builds

To build Haskell executables with profiling enabled, pass -c dbg to Bazel, e.g. bazel build -c dbg damlc. If you want to build the whole SDK with profiling enabled use daml-sdk-head --profiling.