The Daml smart contract language
Go to file
Brian Healey ca66061b28
Ledger Api Oracle support (#9277)
* WIP : first cut at changed schema files for oracle
Define Oracle as DbType and handle necessary case match switches for it
recomputed shas for oracle migration scripts
Oracle fixtures
get things compiling
Able to connect to Oracle
Working through getting schema definitions functional with Oracle
runnable schema definitions only for active tables on oracle
delete commented lines in schema scripts
use oracle enterprise
correct inadvertently changed postgres schemas
WIP - latest oracle-ificiation
passing upload packages spec
add additional test for package upload entry read
correct typo in oracle database spec name
use BLOB for parties ledger_offset
package_entries use hex version of offset for range queries
reformat and update shas for sql scripts
binary numeric implicit conversion for oracle
correct duplicate exception text for oracle
parties test passing on oracle
add additional column to hold hex offset for party_entries
party_entries working for all dbs
scalafmt
Configuration ledger_offset should be BLOB
update sha of oracle sql files
enable passing tests in order
remove misleading null comments
define additional custom VARRAY types
add participant-integration-api-oracle tests to linux-oracle job
Add TODO for places where we need to deal with separate implicit imports for Oracle vs Postgres/H2
oracle implicit conversions for custom arrays and other problematic types

Do not override default debug level for all tests in participant-integration-api

CHANGELOG_BEGIN
Ledger API and Indexer Oracle Support
CHANGELOG_END

passing TransactionWriterSpec
passing JdbcLedgerDaoCompletionsSpec JdbcLedgerDaoDivulgenceSpec
passing JdbcLedgerDaoContractsSpec
All Oracle tests passing apart from one post-commit validation test

* Remove JdbcLedgerDaoValidatedOracleSpec as this is only relevant for classic postgres-backed sandbox

* rebase to master -- offsets are now varchar2 rather than blob

* remove use of DBMS_LOB operations

* remove all greater than/less than variants for DBMS_LOB

* revert postgres files that need not be touched

* code review feedback : avoid code duplication

* avoid indirection in type names for oracle arrays

* code review: HexString implicit conversions are not needed

* code review: Oracle case is not yet implemented for appendonlydao

* code review: Oracle case is not yet implemented for appendonlydao (cleanup import)

* code review: revert files that should not be touched

* address code review feedback: db specific imports for command completion become part of queries

* code review: perform db-specific reserved word escape to avoid case match

* code review: remove all dbms_lob comparison operations

* use simpler insert into with ignore dupes hint for oracle

* code review: avoid db specific match case in events range, use db specific limitClause

* code review: restore group by on Binary and Array fields for H2 and Postgres, disable for Oracle

* code review: restore group by on Binary and Array fields for H2 and Postgres, disable for Oracle

* code review: restore group by on binary and array fields for non-oracle dbs, honour the calculation of limit size from QueryParty.ByArith

* code review: honour the calculation of limit size from QueryParty.ByArith

* code review: drop user after oracle test

* code review: remove drop user as it throws errors due to dangling sessions

* code review: revert incorrectly changed postgres schema files

* code review: clean up TODOs

* Remove // before hostname for consistency with other oracle connection strings

* code review: unambiguously scope table column referenced in select and where queries

* code review: correct duplicate table alias
2021-04-26 18:34:17 -04:00
.github Daml case and logo (#8433) 2021-01-08 12:50:15 +00:00
.vscode Update .vscode/settings.json (#8425) 2021-01-07 14:32:33 +00:00
3rdparty Disable deriving-defaults per file (#8684) 2021-01-29 11:01:55 +00:00
bazel_tools Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
build-scripts Upgrade the LF library build setup (resolver, dependencies, version) (#8865) 2021-02-17 07:07:17 +01:00
ci Ledger Api Oracle support (#9277) 2021-04-26 18:34:17 -04:00
compatibility compatibility: Use the same Scala version as the root. (#9502) 2021-04-26 17:49:43 +00:00
compiler Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
daml-assistant Set supported jdbc driver names at compile time (#9489) 2021-04-26 09:19:31 +02:00
daml-lf Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
daml-script Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
dev-env Add a flag to build daml-sdk-head EE (#9482) 2021-04-23 11:44:02 +02:00
docs Optionally emit setTime actions in daml export (#9469) 2021-04-22 12:10:05 +00:00
extractor Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
ghc-lib copyright updates (#8610) 2021-01-22 15:03:36 +00:00
infra more robust macOS cleanup (#9456) 2021-04-21 12:10:47 +02:00
language-support Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
ledger Ledger Api Oracle support (#9277) 2021-04-26 18:34:17 -04:00
ledger-api Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
ledger-service Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
libs-haskell daml build: add a --access-token-file for remote dependencies (#9358) 2021-04-08 23:20:52 +02:00
libs-scala Ledger Api Oracle support (#9277) 2021-04-26 18:34:17 -04:00
navigator Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
nix Run Canton tests with JDK11 (#9355) 2021-04-08 10:42:40 +02:00
release Rotate release rotation (#9465) 2021-04-21 12:10:11 +02:00
replacements update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
rules_daml Fix doctests (#9225) 2021-03-24 12:56:37 +01:00
runtime-components Set supported jdbc driver names at compile time (#9489) 2021-04-26 09:19:31 +02:00
scala-protoc-plugins Upgrade scalafmt and enable trailing commas (#8437) 2021-01-09 11:37:37 +01:00
templates Fix scala version in quickstart-scala (#9401) 2021-04-13 15:54:40 +02:00
triggers Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
.bazelignore Remove redundant clean --expunge (#6890) 2020-07-28 09:49:36 +02:00
.bazelrc Upgrade to Bazel 4.0 (#8798) 2021-02-11 10:08:47 +01:00
.dadew Update VC redist to 14.28.29325.2 (#8431) 2021-01-07 16:31:25 +00:00
.envrc open-sourcing daml 2019-04-04 09:33:38 +01:00
.gitattributes Remove unreleased.rst (#3547) 2019-11-20 15:16:57 +00:00
.gitignore Commit compatibility/.bazelrc (#8955) 2021-02-25 19:53:30 +01:00
.hie-bios Update rules_haskell (#4751) 2020-03-13 16:49:34 +01:00
.hlint.yaml update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
.mergify.yml update mergify config for ci split (#8390) 2021-01-05 11:31:48 +01:00
.scalafmt.conf Pin scalafmt version in configuration file (#8447) 2021-01-11 10:05:58 +00:00
azure-cron.yml Clean broken entries from the Bazel cache (#8668) 2021-01-28 17:57:09 +00:00
azure-pipelines.yml fix release job (#9166) 2021-03-17 09:46:02 +00:00
BAZEL-bash.md add Bazel bash help (#2223) 2019-07-19 15:31:45 +00:00
bazel-haskell-deps.bzl pkgid data deps (#9153) 2021-03-18 12:17:38 +01:00
BAZEL-haskell.md Switch to GHC 8.10.3 (#8394) 2021-01-25 11:53:53 +00:00
bazel-java-deps.bzl Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
BAZEL-JVM.md Daml case and logo (#8433) 2021-01-08 12:50:15 +00:00
BAZEL.md Enforce Java formatting style with google-java-format (#8686) 2021-01-29 16:50:18 +00:00
BUILD Add Oracle support in the trigger service (#9286) 2021-03-31 18:39:08 +02:00
build.ps1 Move artifact publishing out of yaml files (#9071) 2021-03-11 11:44:02 +01:00
build.sh Allow to clean up more than one process listening on a port (#9293) 2021-03-30 19:21:42 +00:00
CHANGELOG Move unreleased user-facing features to its own file (#1762) 2019-06-19 16:32:03 +02:00
CODE_OF_CONDUCT.md open-sourcing daml 2019-04-04 09:33:38 +01:00
CODEOWNERS offboarding @hurryabit (#9398) 2021-04-13 11:59:24 +00:00
CONTRIBUTING.md Daml case and logo (#8433) 2021-01-08 12:50:15 +00:00
COPY update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
daml-logo.png Replace Daml logo on README (#8756) 2021-02-04 16:28:52 +01:00
deps.bzl upgrade canton to 0.23.0 (#9422) 2021-04-15 17:29:41 +00:00
dotfiles open-sourcing daml 2019-04-04 09:33:38 +01:00
fmt.sh Commit compatibility/.bazelrc (#8955) 2021-02-25 19:53:30 +01:00
ghcide_snapshot_windows.json Switch to GHC 8.10.3 (#8394) 2021-01-25 11:53:53 +00:00
ghcide_snapshot.json Switch to GHC 8.10.3 (#8394) 2021-01-25 11:53:53 +00:00
ghcide-snapshot.yaml Switch to GHC 8.10.3 (#8394) 2021-01-25 11:53:53 +00:00
LATEST Release v1.13.0-snapshot.20210419.6730.0.8c3a8c04 (#9439) 2021-04-19 13:23:12 +02:00
LICENSE copyright updates (#8610) 2021-01-22 15:03:36 +00:00
maven_install_2.12.json KVL-874 Change the TelemetryContext to use SpanAttributes instead of raw AttributeKeys and remove an unused object (#9474) 2021-04-26 15:31:46 +02:00
maven_install_2.13.json KVL-874 Change the TelemetryContext to use SpanAttributes instead of raw AttributeKeys and remove an unused object (#9474) 2021-04-26 15:31:46 +02:00
NOTICES update NOTICES file (#9494) 2021-04-25 08:11:20 +02:00
NOTICES.md cleanup oss-compliance & NOTICES.md (#8242) 2020-12-10 16:45:56 +01:00
package.json Upgrade elliptic dependency to address vulnerabilities (#9055) 2021-03-09 14:44:10 +01:00
Pipfile add blackduck scan to run on master (#6130) (#8161) 2020-12-07 19:59:39 +00:00
README.md Release EE SDK tarballs and installer (#9086) 2021-03-11 14:49:48 +01:00
release.sh Point out where the documentation for release.sh is (#8996) 2021-03-03 11:55:30 +00:00
SECURITY.md Daml case and logo (#8433) 2021-01-08 12:50:15 +00:00
stack-snapshot.yaml Add try-catch syntax. (#9203) 2021-03-25 13:36:09 +00:00
stackage_snapshot_windows.json Add try-catch syntax. (#9203) 2021-03-25 13:36:09 +00:00
stackage_snapshot.json Add try-catch syntax. (#9203) 2021-03-25 13:36:09 +00:00
tsconfig.json open-sourcing daml 2019-04-04 09:33:38 +01:00
unreleased.sh update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
Upgrading.md Upgrade rules_haskell and pin stack_snapshot (#6548) 2020-07-02 18:55:09 +02:00
WORKSPACE compatibility: Use the same Scala version as the root. (#9502) 2021-04-26 17:49:43 +00:00
workspace_status.sh update copyright notices for 2021 (#8257) 2021-01-01 19:49:51 +01:00
yarn.lock Upgrade elliptic dependency to address vulnerabilities (#9055) 2021-03-09 14:44:10 +01:00

Daml logo

Download License Build

Copyright (c) 2021 Digital Asset (Switzerland) GmbH and/or its affiliates. All Rights Reserved. SPDX-License-Identifier: Apache-2.0

Welcome to the Daml repository!

This repository hosts all code for the Daml smart contract language and SDK, originally created by Digital Asset. Daml is an open-source smart contract language for building future-proof distributed applications on a safe, privacy-aware runtime. The SDK is a set of tools to help you develop applications based on Daml.

Using Daml

To download Daml, follow the installation instructions. Once installed, to try it out, follow the quickstart guide.

If you have questions about how to use Daml or how to build Daml-based solutions, please join us on the Daml forum. Alternatively, if you prefer asking on StackOverflow, please use the daml tag.

Contributing to Daml

We warmly welcome contributions. If you are looking for ideas on how to contribute, please browse our issues. To build and test Daml:

1. Clone this repository

git clone git@github.com:digital-asset/daml.git
cd daml

2. Set up the development dependencies

Our builds require various development dependencies (e.g. Java, Bazel, Python), provided by a tool called dev-env.

Linux and Mac

On Linux and Mac dev-env can be installed with:

  1. Install Nix by running: bash <(curl -sSfL https://nixos.org/nix/install)
  2. Enter dev-env by running: eval "$(dev-env/bin/dade assist)"

If you don't want to enter dev-env manually each time using eval "$(dev-env/bin/dade assist)", you can also install direnv. This repo already provides a .envrc file, with an option to add more in a .envrc.private file.

Windows

On Windows you need to enable long file paths by running the following command in an admin powershell:

Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem' -Name LongPathsEnabled -Type DWord -Value 1

Then start dev-env from PowerShell with:

.\dev-env\windows\bin\dadew.ps1 install
.\dev-env\windows\bin\dadew.ps1 sync
.\dev-env\windows\bin\dadew.ps1 enable

In all new PowerShell processes started, you need to repeat the enable step.

3. First build and test

We have a single script to build most targets and run the tests. On Linux and Mac run ./build.sh. On Windows run .\build.ps1. Note that these scripts may take over an hour the first time.

To just build do bazel build //..., and to just test do bazel test //.... To read more about Bazel and how to use it, see the Bazel site.

On Mac if building is causing trouble complaining about missing nix packages, you can try first running nix-build -A tools -A cached nix repeatedly until it completes without error.

4. Installing a local copy

On Linux and Mac run daml-sdk-head which installs a version of the SDK with version number 0.0.0. Set the version: field in any Daml project to 0.0.0 and it will use the locally installed one.

On Windows:

bazel build //release:sdk-release-tarball
tar -vxf .\bazel-bin\release\sdk-release-tarball-ce.tar.gz
cd sdk-*
daml\daml.exe install . --activate

That should tell you what to put in the path, something along the lines of C:\Users\admin\AppData\Roaming\daml\bin. Note that the Windows build is not yet fully functional.

Caching: build speed and disk space considerations

Bazel has a lot of nice properties, but they come at the cost of frequently rebuilding "the world". To make that bearable, we make extensive use of caching. Most artifacts should be cached in our CDN, which is configured in .bazelrc in this project.

However, even then, you may end up spending a lot of time (and bandwidth!) downloading artifacts from the CDN. To alleviate that, by default, our build will create a subfolder .bazel-cache in this project and keep an on-disk cache. This can take about 10GB at the time of writing.

To disable the disk cache, remove the following lines:

build:linux --disk_cache=.bazel-cache
build:darwin --disk_cache=.bazel-cache

from the .bazelrc file.

If you work with multiple copies of this repository, you can point all of them to the same disk cache by overwriting these configs in either a .bazelrc.local file in each copy, or a ~/.bazelrc file in your home directory.

Shared memory segment issues

On macOS at least, it looks like our setup does not always properly close the resources PostgreSQL uses. After a number of test runs, you may encounter an error message along the lines of:

FATAL:  could not create shared memory segment: No space left on device
DETAIL:  Failed system call was shmget(key=5432001, size=56, 03600).
HINT:  This error does *not* mean that you have run out of disk space. It occurs either if all available shared memory IDs have been taken, in which case you need to raise the SHMMNI parameter in your kernel, or because the system's overall limit for shared memory has been reached.
        The PostgreSQL documentation contains more information about shared memory configuration.
child process exited with exit code 1

In this case, this is a memory leak, so increasing SHMNI (or SHMALL etc.) as suggested will only delay the issue. You can look at the existing shared memory segments on your system by running ipcs -mcopt; this will print a line per segment, indicating the process ID of the last process to connect to the segment as well as the last access time and the number of currently connected processes.

If you identify segments with no connected processes, and you are confident you can remove them, you can do so with ipcrm $sid, where $sid is the process ID displayed (as the second column) by ipcs. Not many macOS applications use shared memory segments; if you have verified that all the existing memory segments on your machine need to be deleted, e.g. because they have all been created by PostgreSQL instances that are no longer running, here is a Bash invocation you can use to remove all shared memory segments from your system.

This is a dangerous command. Make sure you understand what it does before running it.

for shmid in $(ipcs -m | sed 1,3d | awk '{print $2}' | sed '$d'); do ipcrm -m $shmid; done

Haskell profiling builds

To build Haskell executables with profiling enabled, pass -c dbg to Bazel, e.g. bazel build -c dbg damlc. If you want to build the whole SDK with profiling enabled use daml-sdk-head --profiling.