* daml-lf: make Decimal type safe * daml-lf: create Utf8String type * daml-lf: cleanup in data package * Address Stephen Comments * daml-lf: remove UTF8String |
||
---|---|---|
.. | ||
archive | ||
data | ||
engine | ||
interface | ||
interpreter | ||
lfpackage | ||
notes | ||
parser | ||
repl | ||
scenario-interpreter | ||
spec | ||
testing-tools | ||
tests | ||
transaction | ||
transaction-scalacheck | ||
validation | ||
governance.rst | ||
README.md |
The unified DAML-LF interpreter and engine
This package contains the canonical in-memory LF ASTs of both the public interface and the whole contents, decoders from on-wire LF to those, and an interpreter for LF.
Additionally a separate package is provided for a standalone REPL allowing loading of .dalf files and interpretation of pure functions, updates and scenarios.
We provide both Bazel-based and Sbt-based builds for this project. The Sbt builds are provided solely for development purposes, to facilitate incremental compilation and IDE integration. You can simply import the sbt project for development, but if making changes, know that the Bazel build is the sole source of truth for CI and releases.
Components
-
archive
contains the Protobuf definition of the LF format, and Protobuf utilities for reading it into a raw memory form. This should reflect the official LF specification at any given time. As with the LF specification, changes to the Protobuf definition are governed by the DAML-LF Governance process. -
interface
is an ADT of the "public interface" of a given LF package, meaning its templates, their choices, and serializable data types in the package. The ADT does not includedef
s or expressions. A reader from the raw protobuf is included. The ADT is usable from Java. -
lfpackage
is the canonical LF ADT, containing all information about an LF package. Its main consumer is theinterpreter
, which compiles from this faithful representation of the protobuf LF archive into a lower-level AST that is then interpreted.The current plan with
lfpackage
is to be able to load both old and new LF versions into, so that the interpreter and other consumers can work with a common format.For most use cases
lfpackage
is too complex, andinterface
is more convenient; if you need the extra information, this is available, though, but without guarantees of stability. -
transaction
holds ADTs related to the interpretation of LF, aslfpackage
represents the definitions in LF. The base of these is the Value ADT, representing serializable values (i.e. values of serializable LF type). Building on that is the Transaction ADT, representing ledger updates.Both have associated Protobuf definitions, also contained in this package, and are used in
interpreter
andengine
respectively. -
data
contains utility datatypes used in the engine, and functions designed around specified LF semantics. For example, if you want LF-compatible decimal handling, theDecimal
API is a good source of useful functions. -
interpreter
is the "unified interpreter" used for both the sandbox and the production ledger. It is an efficient CEK machine, interpreting thelfpackage
terms using a (non-serializable) internal value model, ultimately producingtransaction
s. Most downstream will want to useengine
in addition to this, because only the pure interpreter lives here. -
engine
holds the ledger state oninterpreter
's behalf and implements all of its public-facing aspects, such as theCommand
interface, events, and loaded packages. -
scenario-interpreter
practically demonstrates whyinterpreter
is separate fromengine
: it is a small set of library functions usinginterpreter
to evaluate scenarios from an LF. -
repl
is the below-described REPL, manipulating an internal engine state and running scenarios at your command. -
testing-tools
helps you run scenarios from Scalatest.
Building and testing
DAML-LF uses Bazel to build and test the components. Please refer to top-level
BAZEL.md
and BAZEL-JVM.md
documents for high-level instructions on how to
use Bazel and how it works within IntelliJ.
To get a list of build targets:
bazel query //daml-foundations/daml-lf/...
To build and test everything:
bazel build //daml-foundations/daml-lf/...
bazel test //daml-foundations/daml-lf/...
To watch a target and re-run tests when files change:
ibazel test //daml-foundations/daml-lf/...
All the above can of course take more fine-grained targets as arguments. "..." means all targets under this directory, recursively. ":all" would specify all targets in the specified directory.
To load a package in the scala repl you will need to add a "@repl" target to BUILD.bazel:
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_repl")
scala_repl(
name = "interpreter@repl",
deps = [
":interpreter"
]
)
This target can then be invoked with "bazel run":
da$ bazel run //daml-lf/interpreter:interpreter@repl
or:
interpreter$ bazel run interpreter@repl
Since "rules_scala" does not currently support incremental compilation you will need to help Bazel along a bit by keep the dependency graph lean. Try to divide your code into separate scala_library targets as build results are cached at this level. Preferably unrelated modules should be separate scala_library targets, unvisible to the outside. A visible scala_library target should then collect the unrelated modules into a single target that can be depended on from outside.
DAML-LF-REPL Usage
The REPL can be compiled with bazel build //:daml-lf-repl
and run with
bazel run //:daml-lf-repl -- repl
. The //:
prefix is not needed when
at repository root.
Note that the REPL currently only supports loading from .dalf files, not from .dar files.
Example use:
$ bazel run //:daml-lf-repl -- repl daml> :load project.dalf daml> Project.double 4 8 daml> :scenario Project.tests ...
See :help
for more instructions.
The REPL application also provides commands test
and testAll
for
running scenarios in packages:
$ bazel run //:daml-lf-repl -- testAll $PWD/project.dalf $ bazel run //:daml-lf-repl -- test Project.tests $PWD/project.dalf
NOTE: When running via bazel run
one needs to specify full path (or relative path from repo root), since Bazel runs all commands from repository root.
Scala house rules
-
Do not use
Seq
in the interpreter's code paths, with the possible exceptions of accepting inputs in external APIs. UseImmArray
,FrontStack
, andBackStack
as appropriate.The reason for this rule is that
Seq
hides completely the performance of operations -- for example it defines cons and snoc and append for all structures even if it requires a full copy for an array.ImmArray
should be used in cases where you do not need to append or prepend content often. It is however very cheap to slice theImmArray
(removing elements from either end).FrontStack
should be used when needing to build up a list of elements by prepending elements. Both single elements or chunks in the form ofImmArray
can be prepended. A typical use case if traversing a tree in topological order by keeping a stack of children to still be visited.BackStack
is likeFrontStack
but you can append rather than prepend. For example if you find yourself building and then reversing a list, useBackStack
instead. -
Avoid mutable data structures in external APIs. This is not set in stone but generally a code smell.
-
Try to always define functions on user-provided data structures (of which we have a lot of in this codebase) to be tail recursive. The typical way to do this is by defining little "interpreters" to perform your function. XXX put good example here once we have an established pattern. In doubt, ask Francesco Mazzoli or Gyorgy Farkas about this.
-
Disable "Optimize imports on the fly" and the "Optimize Imports" shortcut in IntelliJ IDEA, since they mess up diffs and can subtly, insidiously change the semantics of your code (Scala imports are order sensitive). You can disable "on the fly" at Menu -> Preferences -> Editor -> General -> Auto Import -> Scala, and the shortcut key in Preferences -> Keymap -> search Optimize Imports -> double-click result -> Remove.