Custom bindings doc + iou-no-codegen scala example (#1569)

* Custom bindings doc + iou-no-codegen scala example

* Fixing formatting

* Review suggestion

Co-Authored-By: Stephen Compall <scompall@nocandysw.com>

* Adding bazel build for `language-support/scala/examples/iou-no-codegen`

* Review suggestions

* Final touches + addressing review comments

* Adding notes about codegen alternatives

* Removing detectedOs lazy val
This commit is contained in:
Leonid Shlyapnikov 2019-06-12 09:26:54 -04:00 committed by GitHub
parent be974260ce
commit 3eab89e6c6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
20 changed files with 565 additions and 1 deletions

View File

@ -40,6 +40,7 @@ Building applications
app-dev/bindings-scala/index
app-dev/bindings-js
app-dev/grpc/index
app-dev/bindings-x-lang/index
app-dev/app-arch
SDK tools

View File

@ -0,0 +1 @@
../../../../../language-support/scala/examples/iou-no-codegen/

View File

@ -0,0 +1 @@
../../../getting-started/quickstart

View File

@ -0,0 +1 @@
../../../../../language-support/scala/examples/quickstart-scala

View File

@ -0,0 +1,110 @@
.. Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
.. SPDX-License-Identifier: Apache-2.0
Creating your own bindings
##########################
This page gets you started with creating custom bindings for the Digital Asset distributed ledger.
Introduction
============
Digital Asset currently provides bindings for the following programming languages:
- :doc:`Java </app-dev/bindings-java/index>`
- :doc:`Scala </app-dev/bindings-scala/index>`
- :doc:`JavaScript (Node.js) </app-dev/bindings-js>`
You can create bindings for any programming language supported by `gRPC <https://grpc.io/docs/>`_.
What do we mean by "bindings"? Bindings for a language consist of two main components:
- Ledger API
Client "stubs" for the programming language, -- the remote API that allows sending ledger commands and receiving ledger transactions. You have to generate **Ledger API** from `the gRPC protobuf definitions in the daml repository on GitHub <https://github.com/digital-asset/daml/tree/master/ledger-api/grpc-definitions>`_. **Ledger API** is documented on this page: :doc:`/app-dev/grpc/index`. The `gRPC <https://grpc.io/docs/>`_ tutorial explains how to generate client "stubs".
- Codegen
A code generator is a program that generates classes representing DAML contract templates in the language. These classes incorporate all boilerplate code for constructing: :ref:`com.digitalasset.ledger.api.v1.CreateCommand` and :ref:`com.digitalasset.ledger.api.v1.ExerciseCommand` corresponding for each DAML contract template.
Technically codegen is optional. You can construct the commands manually from the auto-generated **Ledger API** classes. However, it is very tedious and error-prone. If you are creating *ad hoc* bindings for a project with a few contract templates, writing a proper codegen may be overkill. On the other hand, if you have hundreds of contract templates in your project or are planning to build language bindings that you will share across multiple projects, we recommend including a codegen in your bindings. It will save you and your users time in the long run.
Note that for different reasons we chose codegen, but that is not the only option. There is really a broad category of metaprogramming features that can solve this problem just as well or even better than codegen; they are language-specific, but often much easier to maintain (i.e. no need to add a build step). Some examples are:
- `F# Type Providers <https://docs.microsoft.com/en-us/dotnet/fsharp/tutorials/type-providers/creating-a-type-provider#a-type-provider-that-is-backed-by-local-data>`_
- `Template Haskell <https://wiki.haskell.org/Template_Haskell>`_
- Scala macro annotations (not future-proof enough to use when implementing the last Scala codegen)
Building Ledger Commands
========================
No matter what approach you take, either manually building commands or writing a codegen to do this, you need to understand how ledger commands are structured. This section demonstrates how to build create and exercise commands manually and how it can be done using contract classes generated by Scala codegen.
Create Command
--------------
Let's recall an **IOU** example from the :doc:`Quickstart guide </getting-started/quickstart>`, where `Iou` template is defined like this:
.. literalinclude:: ./code-snippets/quickstart/template-root/daml/Iou.daml
:language: daml
:lines: 9-15
Here is how to manually build a :ref:`com.digitalasset.ledger.api.v1.CreateCommand` for the above contract template in Scala:
.. literalinclude:: ./code-snippets/iou-no-codegen/application/src/main/scala/com/digitalasset/quickstart/iou/IouCommands.scala
:start-after: // <doc-ref:iou-no-codegen-create-command>
:end-before: // </doc-ref:iou-no-codegen-create-command>
If you do not specify any of the above fields or type their names or values incorrectly, or do not order them exactly as they are in the DAML template, the above code will compile but fail at run-time because you did not structure your create command correctly.
Codegen should simplify the command construction by providing auto-generated utilities to help you construct commands. For example, when you use :doc:`Scala codegen </app-dev/bindings-scala/index>` to generate contract classes, a similar contract instantiation would look like this:
.. literalinclude:: ./code-snippets/quickstart-scala/application/src/main/scala/com/digitalasset/quickstart/iou/IouMain.scala
:start-after: // <doc-ref:iou-contract-instance>
:end-before: // </doc-ref:iou-contract-instance>
Exercise Command
----------------
To build :ref:`com.digitalasset.ledger.api.v1.ExerciseCommand` for `Iou_Transfer`:
.. literalinclude:: ./code-snippets/quickstart/template-root/daml/Iou.daml
:language: daml
:lines: 23, 52-55
manually in Scala:
.. literalinclude:: ./code-snippets/iou-no-codegen/application/src/main/scala/com/digitalasset/quickstart/iou/IouCommands.scala
:start-after: // <doc-ref:iou-no-codegen-exercise-command>
:end-before: // </doc-ref:iou-no-codegen-exercise-command>
versus creating the same command using a value class generated by :doc:`Scala codegen </app-dev/bindings-scala/index>`:
.. literalinclude:: ./code-snippets/quickstart-scala/application/src/main/scala/com/digitalasset/quickstart/iou/IouMain.scala
:start-after: // <doc-ref:iou-exercise-transfer-cmd>
:end-before: // </doc-ref:iou-exercise-transfer-cmd>
Summary
=======
When creating custom bindings for the Digital Asset distributed ledger, you will need to:
- generate **Ledger API** from the gRPC definitions
- decide whether to write a codegen to generate ledger commands or manually build them for all contracts defined in your DAML model.
The above examples should help you get started. If you are creating custom binding or have any questions, see the :doc:`/support/support` page for how to get in touch with us.
Links
=====
- A Scala example that demonstrates how to manually construct ledger commands: https://github.com/digital-asset/daml/tree/master/language-support/scala/examples/iou-no-codegen
- A Scala codegen example: https://github.com/digital-asset/daml/tree/master/language-support/scala/examples/quickstart-scala
- gRPC documentation: https://grpc.io/docs/
- Digital Asset Ledger API gRPC protobuf definitions: https://github.com/digital-asset/daml/tree/master/ledger-api/grpc-definitions

View File

@ -44,8 +44,9 @@ DAML SDK documentation
app-dev/bindings-scala/index
app-dev/bindings-js
app-dev/grpc/index
app-dev/bindings-x-lang/index
app-dev/app-arch
.. toctree::
:titlesonly:
:maxdepth: 2

View File

@ -120,3 +120,16 @@ da_scala_binary(
"//ledger/sandbox",
],
)
da_scala_binary(
name = "iou-no-codegen-bin",
srcs = glob(["iou-no-codegen/application/src/main/scala/**/*.scala"]),
main_class = "com.digitalasset.quickstart.iou.IouMain",
resources = glob(["iou-no-codegen/application/src/main/resources/**/*"]),
scalacopts = ["-Xsource:2.13"],
deps = [
"//language-support/scala/bindings",
"//language-support/scala/bindings-akka",
"//ledger-api/rs-grpc-bridge",
],
)

View File

@ -0,0 +1,43 @@
# iou-no-codegen example
This example demonstrates how to:
- create a contract manually constructing a `Create` command and submitting it to the ledger
- subscribe to receive ledger events
- exercise a choice manually constructing an `Exercise` command and submitting to the ledger
This examples requires a running sandbox.
## To start a sandbox running IOU example
- create a new DAML Assistant `quickstart-scala` project
```
$ daml new quickstart-scala quickstart-scala
```
- change directory to this project
```
$ cd quickstart-scala
```
- compile DAR, start sandbox and navigator processes
```
$ daml start
```
## To run the iou-no-codegen example:
- Run sbt command from `iou-no-codegen` folder:
```
$ sbt "application/runMain com.digitalasset.quickstart.iou.IouMain <sandbox-host-name> <sandbox-port-number> <iou-package-id>"
```
Default sandbox port is 6865.
IOU template package ID (`iou-package-id`) can be found in the Navigator. Templates screen displays all available contract templates, e.g:
```
Iou:Iou@fc3e49291d12ef5f46a3b51398558257a469884c46211942c1559cf0be46872c
Iou:IouTransfer@fc3e49291d12ef5f46a3b51398558257a469884c46211942c1559cf0be46872c
IouTrade:IouTrade@fc3e49291d12ef5f46a3b51398558257a469884c46211942c1559cf0be46872c
```
Package ID is the part of the template ID after `@` character. In the above example, all templates are from the package with package ID: `fc3e49291d12ef5f46a3b51398558257a469884c46211942c1559cf0be46872c`.
To connect to the sandbox running on localhost, listening to the default port and with the above package ID:
```
$ sbt "application/runMain com.digitalasset.quickstart.iou.IouMain localhost 6865 fc3e49291d12ef5f46a3b51398558257a469884c46211942c1559cf0be46872c"
```

View File

@ -0,0 +1 @@
../../../../VERSION

View File

@ -0,0 +1,14 @@
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %level - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>
</configuration>

View File

@ -0,0 +1,86 @@
// Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package com.digitalasset.quickstart.iou
import java.util.UUID
import akka.stream.Materializer
import akka.stream.scaladsl.{Sink, Source}
import akka.{Done, NotUsed}
import com.digitalasset.api.util.TimeProvider
import com.digitalasset.api.util.TimestampConversion.fromInstant
import com.digitalasset.ledger.api.domain.LedgerId
import com.digitalasset.ledger.api.refinements.ApiTypes.{ApplicationId, WorkflowId}
import com.digitalasset.ledger.api.v1.command_submission_service.SubmitRequest
import com.digitalasset.ledger.api.v1.commands.{Command, Commands}
import com.digitalasset.ledger.api.v1.ledger_offset.LedgerOffset
import com.digitalasset.ledger.api.v1.transaction.Transaction
import com.digitalasset.ledger.api.v1.transaction_filter.{Filters, TransactionFilter}
import com.digitalasset.ledger.client.LedgerClient
import com.digitalasset.quickstart.iou.FutureUtil.toFuture
import com.google.protobuf.empty.Empty
import scala.concurrent.duration._
import scala.concurrent.{ExecutionContext, Future}
class ClientUtil(
client: LedgerClient,
applicationId: ApplicationId,
ttl: Duration,
timeProvider: TimeProvider) {
import ClientUtil._
private val ledgerId = client.ledgerId
private val packageClient = client.packageClient
private val commandClient = client.commandClient
private val transactionClient = client.transactionClient
def listPackages(implicit ec: ExecutionContext): Future[Set[String]] =
packageClient.listPackages().map(_.packageIds.toSet)
def ledgerEnd(implicit ec: ExecutionContext): Future[LedgerOffset] =
transactionClient.getLedgerEnd.flatMap(response => toFuture(response.offset))
def submitCommand(party: String, workflowId: WorkflowId, cmd: Command.Command): Future[Empty] = {
val now = timeProvider.getCurrentTime
val commands = Commands(
ledgerId = LedgerId.unwrap(ledgerId),
workflowId = WorkflowId.unwrap(workflowId),
applicationId = ApplicationId.unwrap(applicationId),
commandId = uniqueId,
party = party,
ledgerEffectiveTime = Some(fromInstant(now)),
maximumRecordTime = Some(fromInstant(now.plusNanos(ttl.toNanos))),
commands = Seq(Command(cmd))
)
commandClient.submitSingleCommand(SubmitRequest(Some(commands), None))
}
def nextTransaction(party: String, offset: LedgerOffset)(
implicit mat: Materializer): Future[Transaction] =
transactionClient
.getTransactions(offset, None, transactionFilter(party))
.take(1L)
.runWith(Sink.head)
def subscribe(party: String, offset: LedgerOffset, max: Option[Long])(f: Transaction => Unit)(
implicit mat: Materializer): Future[Done] = {
val source: Source[Transaction, NotUsed] =
transactionClient.getTransactions(offset, None, transactionFilter(party))
max.fold(source)(n => source.take(n)) runForeach f
}
override lazy val toString: String = s"ClientUtil{ledgerId=$ledgerId}"
}
object ClientUtil {
def transactionFilter(parties: String*): TransactionFilter =
TransactionFilter(parties.map((_, Filters.defaultInstance)).toMap)
def uniqueId: String = UUID.randomUUID.toString
def workflowIdFromParty(p: String): WorkflowId =
WorkflowId(s"$p Workflow")
}

View File

@ -0,0 +1,22 @@
// Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package com.digitalasset.quickstart.iou
import com.digitalasset.ledger.api.v1.event.{ArchivedEvent, CreatedEvent, Event}
import com.digitalasset.ledger.api.v1.transaction.Transaction
object DecodeUtil {
def decodeCreatedEvent(transaction: Transaction): Option[CreatedEvent] =
for {
event <- transaction.events.headOption: Option[Event]
created <- event.event.created: Option[CreatedEvent]
} yield created
def decodeArchivedEvent(transaction: Transaction): Option[ArchivedEvent] = {
for {
event <- transaction.events.headOption: Option[Event]
archived <- event.event.archived: Option[ArchivedEvent]
} yield archived
}
}

View File

@ -0,0 +1,11 @@
// Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package com.digitalasset.quickstart.iou
import scala.concurrent.Future
object FutureUtil {
def toFuture[A](o: Option[A]): Future[A] =
o.fold(Future.failed[A](new IllegalStateException(s"Empty option: $o")))(a =>
Future.successful(a))
}

View File

@ -0,0 +1,54 @@
// Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package com.digitalasset.quickstart.iou
import com.digitalasset.ledger.api.v1.commands.Command.Command
import com.digitalasset.ledger.api.v1.commands.{CreateCommand, ExerciseCommand}
import com.digitalasset.ledger.api.v1.value._
/**
* Examples of how to construct ledger commands "manually".
*/
object IouCommands {
// <doc-ref:iou-no-codegen-create-command>
def iouCreateCommand(
templateId: Identifier,
issuer: String,
owner: String,
currency: String,
amount: BigDecimal): Command.Create = {
val fields = Seq(
RecordField("issuer", Some(Value(Value.Sum.Party(issuer)))),
RecordField("owner", Some(Value(Value.Sum.Party(owner)))),
RecordField("currency", Some(Value(Value.Sum.Text(currency)))),
RecordField("amount", Some(Value(Value.Sum.Decimal(amount.toString)))),
RecordField("observers", Some(Value(Value.Sum.List(List())))),
)
Command.Create(
CreateCommand(
templateId = Some(templateId),
createArguments = Some(Record(Some(templateId), fields))))
}
// </doc-ref:iou-no-codegen-create-command>
// <doc-ref:iou-no-codegen-exercise-command>
def iouTransferExerciseCommand(
templateId: Identifier,
contractId: String,
newOwner: String): Command.Exercise = {
val transferTemplateId = Identifier(
packageId = templateId.packageId,
moduleName = templateId.moduleName,
entityName = "Iou_Transfer")
val fields = Seq(RecordField("newOwner", Some(Value(Value.Sum.Party(newOwner)))))
Command.Exercise(
ExerciseCommand(
templateId = Some(templateId),
contractId = contractId,
choice = "Iou_Transfer",
choiceArgument = Some(Value(Value.Sum.Record(Record(Some(transferTemplateId), fields))))
))
}
// </doc-ref:iou-no-codegen-exercise-command>
}

View File

@ -0,0 +1,134 @@
// Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
package com.digitalasset.quickstart.iou
import java.time.Instant
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import com.digitalasset.api.util.TimeProvider
import com.digitalasset.grpc.adapter.AkkaExecutionSequencerPool
import com.digitalasset.ledger.api.refinements.ApiTypes.{ApplicationId, WorkflowId}
import com.digitalasset.ledger.api.v1.ledger_offset.LedgerOffset
import com.digitalasset.ledger.api.v1.value.Identifier
import com.digitalasset.ledger.client.LedgerClient
import com.digitalasset.ledger.client.configuration.{
CommandClientConfiguration,
LedgerClientConfiguration,
LedgerIdRequirement
}
import com.digitalasset.quickstart.iou.ClientUtil.workflowIdFromParty
import com.digitalasset.quickstart.iou.DecodeUtil.decodeCreatedEvent
import com.digitalasset.quickstart.iou.FutureUtil.toFuture
import com.typesafe.scalalogging.StrictLogging
import scala.concurrent.duration._
import scala.concurrent.{Await, ExecutionContext, Future}
import scala.util.{Failure, Success}
object IouMain extends App with StrictLogging {
if (args.length != 3) {
logger.error("Usage: LEDGER_HOST LEDGER_PORT IOU_PACKAGE_ID")
System.exit(-1)
}
private val ledgerHost = args(0)
private val ledgerPort = args(1).toInt
private val packageId = args(2)
private val iouTemplateId =
Identifier(packageId = packageId, moduleName = "Iou", entityName = "Iou")
private val issuer = "Alice"
private val newOwner = "Bob"
private val asys = ActorSystem()
private val amat = ActorMaterializer()(asys)
private val aesf = new AkkaExecutionSequencerPool("clientPool")(asys)
private def shutdown(): Unit = {
logger.info("Shutting down...")
Await.result(asys.terminate(), 10.seconds)
()
}
private implicit val ec: ExecutionContext = asys.dispatcher
private val applicationId = ApplicationId("IOU Example")
private val timeProvider = TimeProvider.Constant(Instant.EPOCH)
private val clientConfig = LedgerClientConfiguration(
applicationId = ApplicationId.unwrap(applicationId),
ledgerIdRequirement = LedgerIdRequirement("", enabled = false),
commandClient = CommandClientConfiguration.default,
sslContext = None
)
private val clientF: Future[LedgerClient] =
LedgerClient.singleHost(ledgerHost, ledgerPort, clientConfig)(ec, aesf)
private val clientUtilF: Future[ClientUtil] =
clientF.map(client => new ClientUtil(client, applicationId, 30.seconds, timeProvider))
private val offset0F: Future[LedgerOffset] = clientUtilF.flatMap(_.ledgerEnd)
private val issuerWorkflowId: WorkflowId = workflowIdFromParty(issuer)
private val newOwnerWorkflowId: WorkflowId = workflowIdFromParty(newOwner)
def validatePackageId(allPackageIds: Set[String], packageId: String): Future[Unit] =
if (allPackageIds(packageId)) Future.successful(())
else
Future.failed(
new IllegalArgumentException(
s"Uknown package ID passed: $packageId, all package IDs: $allPackageIds"))
val issuerFlow: Future[Unit] = for {
clientUtil <- clientUtilF
offset0 <- offset0F
_ = logger.info(s"Client API initialization completed, Ledger ID: ${clientUtil.toString}")
allPackageIds <- clientUtil.listPackages
_ = logger.info(s"All package IDs: $allPackageIds")
_ <- validatePackageId(allPackageIds, packageId)
createCmd = IouCommands.iouCreateCommand(
iouTemplateId,
"Alice",
"Alice",
"USD",
BigDecimal("99999.00"))
_ <- clientUtil.submitCommand(issuer, issuerWorkflowId, createCmd)
_ = logger.info(s"$issuer sent create command: $createCmd")
tx0 <- clientUtil.nextTransaction(issuer, offset0)(amat)
_ = logger.info(s"$issuer received transaction: $tx0")
createdEvent <- toFuture(decodeCreatedEvent(tx0))
_ = logger.info(s"$issuer received created event: $createdEvent")
exerciseCmd = IouCommands.iouTransferExerciseCommand(
iouTemplateId,
createdEvent.contractId,
newOwner)
_ <- clientUtil.submitCommand(issuer, issuerWorkflowId, exerciseCmd)
_ = logger.info(s"$issuer sent exercise command: $exerciseCmd")
} yield ()
val returnCodeF: Future[Int] = issuerFlow.transform {
case Success(_) =>
logger.info("IOU flow completed.")
Success(0)
case Failure(e) =>
logger.error("IOU flow completed with an error", e)
Success(1)
}
val returnCode: Int = Await.result(returnCodeF, 10.seconds)
shutdown()
System.exit(returnCode)
}

View File

@ -0,0 +1,44 @@
import sbt._
import Versions._
import Artifactory._
version in ThisBuild := "0.0.1"
scalaVersion in ThisBuild := "2.12.8"
isSnapshot := true
lazy val parent = project
.in(file("."))
.settings(
name := "iou-no-codegen",
publishArtifact in (Compile, packageDoc) := false,
publishArtifact in (Compile, packageSrc) := false
)
.aggregate(application)
lazy val application = project
.in(file("application"))
.settings(
name := "application",
commonSettings,
libraryDependencies ++= applicationDependencies,
)
lazy val commonSettings = Seq(
scalacOptions ++= Seq(
"-feature",
"-target:jvm-1.8",
"-deprecation",
"-Xfatal-warnings",
"-unchecked",
"-Xfuture",
"-Xlint:_,-unused"
),
resolvers ++= daResolvers,
classpathTypes += "maven-plugin",
)
lazy val applicationDependencies = Seq(
"com.daml.scala" %% "bindings" % daSdkVersion,
"com.daml.scala" %% "bindings-akka" % daSdkVersion,
)

View File

@ -0,0 +1,11 @@
// Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
import sbt._
object Artifactory {
val daResolvers: Seq[MavenRepository] = Seq(
Resolver.bintrayRepo("digitalassetsdk", "DigitalAssetSDK"),
Resolver.mavenLocal
)
}

View File

@ -0,0 +1,13 @@
// Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
object Versions {
private val daSdkVersionKey = "da.sdk.version"
val daSdkVersion: String = sys.props.get(daSdkVersionKey).getOrElse(sdkVersionFromFile())
println(s"$daSdkVersionKey = ${daSdkVersion: String}")
private def sdkVersionFromFile(): String =
"10" + sbt.IO.read(new sbt.File("./SDK_VERSION").getAbsoluteFile).trim
}

View File

@ -0,0 +1 @@
sbt.version=1.2.7

View File

@ -134,7 +134,9 @@ object IouMain extends App with StrictLogging {
offset1 <- clientUtil.ledgerEnd
// <doc-ref:iou-exercise-transfer-cmd>
exerciseCmd = iouContract.contractId.exerciseIou_Transfer(actor = issuer, newOwner = newOwner)
// </doc-ref:iou-exercise-transfer-cmd>
_ <- clientUtil.submitCommand(issuer, issuerWorkflowId, exerciseCmd)
_ = logger.info(s"$issuer sent exercise command: $exerciseCmd")
_ = logger.info(s"$issuer transferred IOU: $iouContract to: $newOwner")