daml/ledger/sandbox-classic
Brian Healey ca66061b28
Ledger Api Oracle support (#9277)
* WIP : first cut at changed schema files for oracle
Define Oracle as DbType and handle necessary case match switches for it
recomputed shas for oracle migration scripts
Oracle fixtures
get things compiling
Able to connect to Oracle
Working through getting schema definitions functional with Oracle
runnable schema definitions only for active tables on oracle
delete commented lines in schema scripts
use oracle enterprise
correct inadvertently changed postgres schemas
WIP - latest oracle-ificiation
passing upload packages spec
add additional test for package upload entry read
correct typo in oracle database spec name
use BLOB for parties ledger_offset
package_entries use hex version of offset for range queries
reformat and update shas for sql scripts
binary numeric implicit conversion for oracle
correct duplicate exception text for oracle
parties test passing on oracle
add additional column to hold hex offset for party_entries
party_entries working for all dbs
scalafmt
Configuration ledger_offset should be BLOB
update sha of oracle sql files
enable passing tests in order
remove misleading null comments
define additional custom VARRAY types
add participant-integration-api-oracle tests to linux-oracle job
Add TODO for places where we need to deal with separate implicit imports for Oracle vs Postgres/H2
oracle implicit conversions for custom arrays and other problematic types

Do not override default debug level for all tests in participant-integration-api

CHANGELOG_BEGIN
Ledger API and Indexer Oracle Support
CHANGELOG_END

passing TransactionWriterSpec
passing JdbcLedgerDaoCompletionsSpec JdbcLedgerDaoDivulgenceSpec
passing JdbcLedgerDaoContractsSpec
All Oracle tests passing apart from one post-commit validation test

* Remove JdbcLedgerDaoValidatedOracleSpec as this is only relevant for classic postgres-backed sandbox

* rebase to master -- offsets are now varchar2 rather than blob

* remove use of DBMS_LOB operations

* remove all greater than/less than variants for DBMS_LOB

* revert postgres files that need not be touched

* code review feedback : avoid code duplication

* avoid indirection in type names for oracle arrays

* code review: HexString implicit conversions are not needed

* code review: Oracle case is not yet implemented for appendonlydao

* code review: Oracle case is not yet implemented for appendonlydao (cleanup import)

* code review: revert files that should not be touched

* address code review feedback: db specific imports for command completion become part of queries

* code review: perform db-specific reserved word escape to avoid case match

* code review: remove all dbms_lob comparison operations

* use simpler insert into with ignore dupes hint for oracle

* code review: avoid db specific match case in events range, use db specific limitClause

* code review: restore group by on Binary and Array fields for H2 and Postgres, disable for Oracle

* code review: restore group by on Binary and Array fields for H2 and Postgres, disable for Oracle

* code review: restore group by on binary and array fields for non-oracle dbs, honour the calculation of limit size from QueryParty.ByArith

* code review: honour the calculation of limit size from QueryParty.ByArith

* code review: drop user after oracle test

* code review: remove drop user as it throws errors due to dangling sessions

* code review: revert incorrectly changed postgres schema files

* code review: clean up TODOs

* Remove // before hostname for consistency with other oracle connection strings

* code review: unambiguously scope table column referenced in select and where queries

* code review: correct duplicate table alias
2021-04-26 18:34:17 -04:00
..
src Ledger Api Oracle support (#9277) 2021-04-26 18:34:17 -04:00
.gitattributes Split sandbox code into separate packages (#6695) 2020-07-17 17:06:06 +02:00
BUILD.bazel Switch from @silent to @nowarn. (#9498) 2021-04-26 19:46:14 +00:00
README.md Daml case and logo (#8433) 2021-01-08 12:50:15 +00:00

Overview

This document is to help internal engineers work with the Sandbox and the new ledger API. Note: If you encounter bugs. Please report issues you find in the #team-ledger-api channel.

Daml Sandbox

To build a fat JAR with the sandbox built from HEAD run

bazel build //ledger/sandbox-classic:sandbox-classic-binary_deploy.jar

Sandbox application can be run from command line with the following command:

java -jar bazel-bin/ledger/sandbox-classic/sandbox-classic-binary_deploy.jar [options] <archive>

as run from the main project root directory (adjust the location of the JAR according to your working directory).

Command line arguments

  -p, --port <value>       Sandbox service port. Defaults to 6865.
  -a, --address <value>    Sandbox service host. Defaults to binding on all addresses.
  --dalf                   Parse provided archives as Daml-LF Archives instead of DARs.
  --static-time            Use static time, configured with TimeService through gRPC.
  -w, --wall-clock-time    Use wall clock time (UTC). When not provided, static time is used.
  -o, --sim-time-offset <value>
                           Use simulated time with the given duration (ISO-8601 with optional `-` prefix) as offset relative to UTC. For example, supplying `-PT6M` will result in the server time lagging behind UTC by 6 minutes. When not provided, static time is used.
  --no-parity              Disables Ledger Server parity mode. Features which are not supported by the Platform become available.
  --scenario <value>       If set, the sandbox will execute the given scenario on startup and store all the contracts created by it.
  --daml-lf-archive-recursion-limit <value>
                           Set the recursion limit when decoding Daml-LF archives (.dalf files). Default is 1000
  <archive>...             Daml archives to load. Only Daml-LF v1 Archives are currently supported.
  --pem <value>            TLS: The pem file to be used as the private key
  --crt <value>            TLS: The crt file to be used as the cert chain. Required if any other TLS parameters are set.
  --cacrt <value>          TLS: The crt file to be used as the trusted root CA.
  --help                   Print the usage text

Compatibility

Sandbox uses models compiled in to the DAR format.

Note that the new Ledger API only supports Daml 1.0 or above codebases compiled to Daml-LF v1. Again, using the Daml packaging as suggested above will ensure that you are generating dar files that the Sandbox can consume.

Ledger API

The new Ledger API uses gRPC. If you just want to create / exercise contracts, I suggest you start by looking at command_service.proto, which exposes a synchronous API to the Daml ledger.

Logging

You can enable debug logging in Sandbox with sandbox-log-level system property:

$ java -jar ./bazel-bin/ledger/sandbox-classic/sandbox-classic-binary_deploy.jar --log-level=DEBUG $PWD/bazel-bin/ledger/sandbox/Test.dar

Or when started from Bazel with:

$ bazel run //ledger/sandbox-classic:sandbox-classic-binary -- --log-level=DEBUG $PWD/bazel-bin/ledger/sandbox/Test.dar

Profiling

You can enable profiling in Sandbox by passing a directory where the profiling information should be written via the --profile-dir flag, e.g.

$ bazel run //ledger/sandbox-classic:sandbox-classic-binary -- --profile-dir=/write/profiles/here/ ...

DISCLAIMER: Profiling is not intended to be used in production setups since it slows down Daml execution significantly and writes a lot of profiling information to disk.

For every command submitted to the Sandbox, a JSON file named like <submission time>-<command description>-<seed>.json is written to the directory specified via --profile-dir. In this file name, <command description> is a string of the form create:TemplateName or exercise:TemplateName:ChoiceName.

These JSON files can be viewed using the speedscope flamegraph visualizer. The easiest way to install speedscope is to run

$ npm install -g speedscope

See the Offline usage section of its documentation for alternatives.

Once speedscope is installed, a specific profile can be viewed via

$ speedscope /path/to/profile.json