* Refactor gRPC request cancellation support in gRPC-haskell
Previously, we had a separate constructor that took an additional
callback that had access to the client call so that we could use that
for cancellation. This PR removes the separate constructor and instead
changes the callback accepted by ClientReaderRequest (and for
consistency also the one accepted by ClientBiDiRequest) to pass the
client call.
This seems like a simpler solution so I’m more hopeful that we will be
able to upstream this change (this is the only API breaking change we
made afaik). There are two caveats here:
1. This will break existing consumers that use ClientReaderRequest. At
this stage in gRPC-haskell this is very reasonable and upstream
doesn’t seem to try very hard to avoid those.
2. The callback is called slightly later than the custom callback we
added before so you have to wait a bit longer until you can cancel
things. At least on our test suite that doesn’t seem to make a
difference.
* Go back to building grpc-haskell-core using c2hs
This should hopefully avoid issues like the CSize vs CULong issue we
had a while back and might fix some of the issues we have been seeing
on CI.
I’m marking the Haskell ledger bindings as non-flaky for now so we can
see if the issues reappear.
* Fix path
* Fix c2hs runfiles
* s/basedir/dirname/
* Fix varname
* Remove fixme \o/
* Mark hs ledger bindings flaky again
The only tests that relied on this, were the ones for package
management and party management but we can fairly easily remove that
dependency by only checking the diff in packages and thereby folding
the listKnown* and uploadParty/uploadDar file into a combined test.
I’ve also bumped one timeout that I’ve seen fail on CI and managed to
get to fail locally under load.
I am unable to get the tests to fail now locally so I’ve marked them
as non-flaky on everything but Windows (we’ve seen weird segfaults
there).
* Use per-test parties in HS ledger bindings tests
This should hopefully solve the flakiness issues caused by the reset
service. For now, there is a very small number of tests that we run on
an isolated sandbox namely the tests for party management and package
management.
* shut up hlint
* Disable reset tests
* Supporting producing sdist tarballs for the HS ledger bindings
The README.md has an explanation for how you can use this.
This should hopefully allow others to experiment with the bindings.
* Read stakeholders through the active contract service
Fixes#2070
Also standardizes the use of American English spelling for `serialize`
and `deserialize`. Standardize. Serialize. Americanize.
* Remove Haskell bindings test workaround
* Address https://github.com/digital-asset/daml/pull/2316#discussion_r308193661
This is a breaking change on the ledger api. So far we could rely on the
assumption that contractId == eventId. This assumption doesn't hold
anymore in a daml-on-x setting, where the eventId is created by the
api server, but the absolute contractId is created by the ledger
implementation.
Instead of going through weird contortions to store the relevant data in
the existing database schema, it is more viable in the long term to
remove that field and instead provide facilities to lookup transactions
by contractId.
Contributes to #2068.
* DA.Bazel.Runfiles based on bazel-runfiles
* locateRunfilesMb -> locateRunfiles
* .exe extension on Windows
* Add docstring to locateRunfiles
* bazel-runfiles: Normalize on Windows
* loading Dars from ZipInputStreams
formatting and cleanup
fixing DarReaderTest not compiling
fmt
using IOUtils to read the ZipInputStream
fixing CodeGen
fixing broken legacy DAR loading
fixed v2 ReferenceServer
cleanup
* fixing rebase conflicts
* introduced a fromFile method to avoid having to do boiler-plating where the input is a File
* addressing leaking resources
* cleanup
* inlining closures
* closing the darStream
* support rpc: GetFlatTransactionByEventId
* support rpc: GetFlatTransactionById
* test: GetTransactions
* support rpc: GetTransactionTrees, and types: TransactionTree,TreeEvent
* reorder TransactionService.hs to match .proto
* rpcs: GetTransactionByEventId, GetTransactionById
* windows: root build
* windows: fixed haskell bindings tests
* windows: disable client_server_test test
* windows: marking daml_test flaky due to #1907
* windows: removing da-hs-damlc-app run from build.ps1
* windows: disable hie-core alias of currently disabled target
* support: CommandCompletionService.completionEnd
* support checkpoints from: CommandCompletionService.completionStream
* address review comments
* add type for LedgerOffset; prefer AbsOffset when known
* extend submit/complete testcase
* Define LedgerService using ReaderT to avoid boilerplate
* re-export TimeoutSeconds from GRPC.HighLevel.Client, instead of defining as a newtype
* rename: Valuable -> IsLedgerValue
We add a dependency in bindings-java to grpc-netty, so it is
automatically available in the classpath for users. Otherwise they get
an error like [0] if they don't add it explicitly.
We also publish daml-lf-archive-java to maven central.
[0] io.grpc.ManagedChannelProvider$ProviderNotFoundException: No
functional channel service provider found. Try adding a dependency
on the grpc-okhttp, grpc-netty, or grpc-netty-shaded artifact
Previously, we got the package id by listing all packages and taking
the first. That is not a valid assumption and broke with a PR. This PR
changes the tests to use the DAR reader that we already used for
visualization (now factored into a separate library) and reads the
main dalf using that.
* store archive size in `DarReader`
* rename `SandboxTemplateStore` to `SandboxPackageStore`
* store package info in `SandboxPackageStore`
* introduce package upload / read to write / index services
not tested yet, just a ton of plumbing
* WIP test the package service
* Fix build errors after rebase
* Move packages service to v2
* Ledger API client uses ledger API types
* Fix ReflectionIT
* Correctly handle uploading invalid dar files
* Fix reading DAR entry file sizes
* Improve package management IT
* Improve handling of duplicate packages
* Fix language-support build
* Use unique party and command names
* Rename lfpackage to language
* Rename SandboxPackageStore to InMemoryPackageStore
* Remove getCurrentTime ledger method
* Improve package management IT
* Move InMemoryActiveContracts and InMemoryPackageStore
* Use case object for UploadDarResult.Ok
* Address review comments
* Update release notes
Fixes#1311
* Use BazelRunfiles in test
* Fixes#1204: Release bindings and codegens to Maven Central.
Upload the Java and Scala Bindings with the respective code
generator binaries to Sonatype Open Source Repository
Host for synchronization with Maven Central.
Nim:
* fix index numbering in demo.md
* add in game "help" command to show available commands
* small code simplification for code review
GroupChat:
* DAML chat-room model, with support for multiple chat-groups with entry by invitation
* simplify: pass Invitation
* fix bug spotted by Bernhard -- Gain access to a Group without a valid invite
* Fixes#1600. Improve Bazel Scala source JAR generation.
Include sources from JAR files defined as 'sources' to a Bazel Scala
target. The improvement will allow inclusion of generated source targets
in sources JARs of dependencies.
* Use temporary directory in the dar_to_scala() rule.
Place the output of the Scala code generator in a temporary
directory instead of a directory at the 'primary' output path as Bazel
rules should never have directory dependencies.
* language-support/scala: add support for specifying roots explicitly
By default, code-gen will use all templates in the given package as roots, and
generate code for all types used. This patch adds the ability to specify roots
explicitly via a set of regex.es to match the (fully qualified) template names.
This is an experiment, tests and UI need to be adjusted later.
* scala codegen rootfilter: move filtering to envIFace handling
* doc for --root option
* scalafmt
* test filterTemplatesBy
* release note for --root Scala codegen option
* Primitive.LegacyTemplateId replaces 2-tuple TemplateId.unapply; 3-tuple variant introduced
* ensure the suggested replacements are well-typed and behaved
* suggest LegacyTemplateId for 2-valued form
* Drop support for DAML-LF 1.3 from damlc
This means `damlc` now only supports versions allowing arbitrary expressions
as contract keys. This is very helpful for the upcoming new template
de/re-sugaring underpinning generic templates.
* Unpin a few DAML-LF target versions in tests
* Fix golden tests
* Adapt changelog to rebase
* Rename DAML-LF scenario tests to 1.dev
* Update tests to --target 1.dev
* Fix release notes
* use with syntax in Daml
* attend to code review comments
* move ledger services implementation to own sub-module
* use ledger reset service when running tests for haskell-ledger-bindings
* expose LL.ClientCall in high level interface
* cancel streaming gRPC calls when attached stream is closed
* fix modification to gRPC-haskell so existing interface in preserved
* ClientCall and clientCallcancel were already available on HighLevel interface
I'm refurbishing the DAML-LF 1.3 specific tests to run against the
latest DAML-LF version (by removing the explicit target version in the
daml_compile target).
Going forward we only "reify" specific DAML LF versions when breaking
changes or new features are introduced, and we need to test the old
behavior specifically (just like we did for DAML LF 1.0 and 1.1).
* Drop support for DAML-LF 1.1 from damlc
Part of #853.
* Fix docs test
* Add show party test back
* Update release notes
* Remove target version from Java codegen test
* ValuePrimitiveEncoding.Mapped, mixin for the "axiom" parts of mapped and xmapped
* remove ValuePrimitiveEncoding.mapped
- it was okay with 1 inductive case, but now there will be 4, so just
mixin Mapped instead
* mark members of ValuePrimitiveEncoding.Mapped final
- mixins are easier to get right if you mark everything final
* add Maps to random tests
* replace <: Template[A] with : TC in ValuePrimitiveEncoding#valueContractId
- no current subclass uses this context bound, but it may be useful for
encodings that wish to make ContractId encodings dependent on the type
parameter
* replace : TC with no bound in ValuePrimitiveEncoding#valueContractId
- fixes#1325
- can reintroduce : TC with #822
* stub test for generating with #1315's liberated contract IDs
- commented out because we still want to test codegen with stable LF
only
* undo useless changes
* ContractId#apply is unbounded
* Rename Value's ContractId to VContractId
* daml-lf: a bit more about PartyId
* daml-lf: Concatenable MatchingStringModule
* daml-lf make clear type used for Scenarios only
* daml-lf create ContractId, LedgerId, TransactionId
* sandbox-sql conversion util
* LedgerName -> LedgerString
* futher type cleanup in the sandbox
* daml-lf add test for LedgerString
* fixing tests
* a bit more safety in the DB
* Address Stephen's comments
* fix rebase
* More fixes for StringModule
* change length of LedgerString (256 -> 255)
* Use closable Stream in Haskell ledger bindings
* move PastAndFuture abstraction into ledger interface code
* distinguish past/future transactions in return of Ledger.getTransactionsPF
* first cut: nim-console running against a ledger
* remove flaky failing test
* Removing unused/broken daml.bzl rules, adding Scala codegen rule
Scala codegen rule: dar_to_scala follows the same approach
as dar_to_java with a few differences:
- dar_to_scala supports multiple dars as an input
- dar_to_scala does not try to compile the generated scala (separation
of responsibilities)
* Using dar_to_scala to compile quickstart-scala example
* Fixing formatting
* Add dependency to examples-quickstart-scala-bin
So if example does not compile, it can't be published.
* Fixing the path to the jar executable
* Changing scala codegen rule to rely on zipper instead of jar
JDK's jar creates srcjar with timestamped files, zipper doesn't
timestamp files. This means we can create reproducible/deterministic/
cacheable srcjars
* Addressing code review comments, wrong var name
* add createAnd syntax to prepare CreateAndExercise commands from Scala codegen contracts
* open-style ExerciseOn typeclass
* GADT-style ExerciseOn works perfectly well, so let's go with that instead
* testing implicit conversion + ExerciseOn TC lookup
* use scalatest-style typesafe === to avoid Anys
* many more ExerciseOn tests
* support ExerciseOn polymorphism in Primitive.exercise
* generate ExerciseOn references in Scala codegen, permitting `createAnd` to combine with exercise* methods
* refactor testCreateContractAndReceiveEvent to support other commands
* compare create and createAnd's command generation behavior in a simple test
* bazel style
* test CreateAndExercise from Scala codegen against sandbox
* release note for Scala codegen create-and-exercise support
* Added agreement_text field to the CreatedEvent in Ledger API.
* Changed java bindings + java codegen
* Changed utilities for scala codegen
* Made necessary changes in Sandbox to propagate the agreement text from ContractInst to the CreatedEvent
* Made changes to the navigator to show the agreement text in the contract details page when it is set and not empty
Fixes#1110
* no var no problem
further refactor
introduced InfraState
a bit less vars
encapsulating closes
SandboxServer starts automatically
rebase fixup
collecting state into a single object
some cleanup
removing exposed materializer
LedgerBackend is closed in SandboxServer
changed ownership of Ledger
fixing perf tests
fixing some compile errors
formatting
removing unused method
fixing integration test to use correct dar file
fixing issue with PostgresFixture and SandboxResource
Fix integration tests on Windows
* fixing rebase artifacts
* ledger-api-intgration-tests: Cleanup ledger ID handling.
This changes nothing in the logic of tests. It is a preparation for moving of
most of the tests to use Dynamic ledger ID mode for tests, which is a
preparation for running these tests against remote Ledger API endpoints.
* sandbox: Rework LedgerIdMode and move it up into common.
* Fixes 895: Improve DA Bazel rules for building javadocs.
Extend the da_java_library Bazel macro to also build the Javadoc for the
target. Add the Javadoc artefacts to the release procedure.
* Add a helper function to qualify things with respect to a module
* Supress some warnings I saw when building
* Move Tuple* and Either to daml-prim:DA.Types, so they have a nice name for Java users (instead of GHC.Tuple or DA.Internal.Prelude)
* Add copyright header
* Update daml-foundations/daml-ghc/daml-prim-src/DA/Types.daml
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
* Update daml-foundations/daml-ghc/daml-prim-src/DA/Types.daml
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
* Update the daml-lf translation of tuple types
* Fix up the scala bindings for how to use types
* Clarify what action people need to take following the release notes
* Fix an expected test
* Clarify the changelogs even more
* Run ScalaFmt
* Clean up the release notes a bit
* Use the changelog from Beth
* Update the release notes again
* Update docs/source/support/release-notes.rst
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Fix up the scenario error messages
* Remove ExercisedEvent in Event oneof.
The Event message is only used in the Transaction message. Flat
transactions do not contain exercised events, but only created and
archived events. Therefore we can remove the ExercisedEvent from the
Event oneof, without breaking transport compatibility.
HOWEVER: The Java Bindings used to use the data.Event class for both flat
transactions and transaction trees. To properly represent the actual
event types in the two transaction structures,
1) Event is now and interface and is only used in the Transaction class.
2) there is a new interface TreeEvent, which is used in the TransactionTree class.
* CreatedEvent implements Event and TreeEvent
* ExercisedEvent implements TreeEvent
* ArchivedEvent implements Event
Some "pathological" cases where an occurrence of an exercised event
would have resulted only in an exception, are now removed (see change in
LedgerApiV1.scala).
Fixes#960.
* haskell-ledger-bindings, test connect with sandbox
* use System.Time.Extra.timeout instead of rolling own
* fix module name to match hierachical path
* increase timeout to 10s, fix macOS CI?
* delete-trailing-whitespace
* dont specify daml version for daml_compile rule
* unalign module imports
* replace xxx with undefined
This test only worked by chance since in older JDK versions
java.time.Instant.now() didn’t have nanoseconds precision. As
evidenced by the test after this, nanoseconds are lost during a
roundtrip so this test breaks on newer JDK versions that increased the
precision. See https://bugs.openjdk.java.net/browse/JDK-8068730 for
more information.
* Add quickstart-scala template to DAML Assistant , #614
Removing the rule to create a quickstart-scala.tar.gz, it is not needed
* Return non-zero error code if failures detected, #614
* Reading project name from `daml.yaml`, #614
Removing Versions.scala.template
* Add quickstart-scala IOU example, DAML Assistant template, #614
* Move all scala examples under language-support/scala/examples
* Removing target dirs which don't get excluded by the glob, #614
If a variant itself is not serializable, but the synthesized record for
one of its constructors is, then said record is returned by the
interface reader in the set of type declarations, when the variant type
itself is not.
When constructing the InterfaceTree in preparation of the codegen, we
previously rejected such a situation.
We now generate Java code for such a synthesized record, as it is a more
generally correct way of interpreting DAML LF (i.e. the DAML compiler
could decide tomorrow to create such multi-component record names for
regular records).
In any case, we consider this to be an edge case, as the synthesized
record for the variant constructor cannot be used directly either from
DAML or the Ledger API.
Submitting a command via the CommandService now returns either the
transaction id (SubmitAndWaitForTransactionId), the flat transaction
(SubmitAndWaitForTransactionResponse), or the transaction tree
(SubmitAndWaitForTransactionTreeResponse).
This means that users don't have to wade through the transaction stream
to retrieve the resulting transaction. This is particularly useful in
combination with #479.
Fixes#406
* HL types for Ledger API. Command Submission & Completion services.
* whitespace
* remove renamed file that came back after bad merge
* delint
* adapt to change in transaction_service.proto
This change is needed in preparation of #406, where we want to return a
transaction tree and flat transaction after a SubmitAndWaitForTransaction(Tree).
* Drop DAML-LF 1.0 support from compiler
This will enable us to add `Functor`, `Applicative` and `Monad` instances
for `(->) r` in the `daml-stdlib`. We'll do this in a separate PR.
* Remove codegen test for DAML-LF 1.0
Also move Interface and InterfaceType out of the reader subpackage; they
belong with the rest of the data model at the iface root.
The specific mechanics of reading a Dar all the way to producing an
EnvironmentInterface are left to Scala codegen's Codegen and Java
codegen's CodeGenRunner; there's no consensus or great stability on the
best way to tie these pieces together, but all the pieces might as well
be available in the interface library at least.
Inspired by a query on Slack by @leonelag regarding reading the
codegen-relevant parts of dalfs and dars; thanks!
* daml-lf: move EnvironmentInterface to interface library from Scala codegen
* daml-lf: move Interface out of reader subpackage
* language-support/java: deal with moving Interface out of reader subpackage
* document Interface and EnvironmentInterface
* missed copyright header in reader package.scala
* extractor: deal with moving Interface out of reader subpackage
* navigator: deal with moving Interface out of reader subpackage
* Extract codegen-common module, #166
* Scala Codegen Main using the same option parser as Java Codegen, #166
There is one important difference, Scala Codegen does not allow mapping
dars to different package names, all dars have to be mapped to the same
package name.
Replace Scala Codegen println's with scala logging, respecting the
configured codegen verbosity
* Fix bazel formatting
* Update the release dry run script
* Releasing codegen-common
* Improving Scala Codegen error reporting (code review)
* Addressing codereview comments
* Make it explicit that we skip not supported option
* Add release notes entry
* Add CreateAndExercise command to Java Bindings data layer
* Add CreateAndExercise command to DAMLe
The CreateAndExerciseCommand allows users to create a contract and
exercise a choice on it within the same transaction. Users can use this
method to implement "callable update functions" by creating a template
that calls the update function in a choice body.
Fixes#382.
* Add CreateAndExercise command handling to the sandbox
* Add CreateAndExercise command to the Ledger API
* Update scala codegen sbt example to use the latest codegen binaries
Use DAR to generate Scala code, no reason to unzip DALFs.
* Add support for `DA.sdkVersion` system property.
To override the default sdkVersion:
```
> sbt -DDA.sdkVersion=100.12.6 compile
```
* Simplify the way examples run
* Remove compiler option that turned off fatal-warnings
The deprecated calls have been already addressed by someone
else.
* Remove `-deprecation` it is enforced throught common_scalacopts
The call to blockingGet can lead to no progress being made in certain
scenarios. Therefore I am removing the blocking call and replacing it
with a regular "doOnSuccess".
Add Dar Traverse and Equal, replace manually written test cases with
scalaz law checks;
Add test case for multiple DARs support;
Enforcing non empty list of input files
* Add buildifier targets.
The tool allows to check and format BUILD files in the repo.
To check if files are well formatted, run:
bazel run //:buildifier
To fix badly-formatted files run:
bazel run //:buildifier-fix
* Cleanup dade-copyright-headers formatting.
* Fix dade-copyright-headers on files with just the copyright.
* Run buildifier automatically on CI via 'fmt.sh'.
* Reformat all BUILD files with buildifier.
Excludes autogenerated Bazel files.
Finishes the work started in #324:
- drops deprecated `name` in identifiers, adopting `moduleName` and `entityName`
- use string to represent `timestamp`s to avoid a loss of precision
- use string for `date`s too for consistency and future-proofness
The work unconvered a quite serious bug caused by the lack of coverage
on the validation of timestamps: the model was expecting timestamps to
be of type timestamp, whereas in Ledger API values they are represented
by numbers.
* Fetch status.proto from remote, simplify JS gRPC codegen
Fetch the `status.proto` file (part of the standard gRPC distribution)
from a distribution channel. _Moreover_, use the recently introduced
`proto_gen` rule to simplify how the gRPC code for the Node.js bindings
are generated (and remove the need to have `google/rpc/status.proto`
locally in the repository.
* Add plugin_runfiles option to proto_gen
This allows use to add additional files to the bazel sandbox so that
plugins can refer to them. This will subsequently be used by the
protoc-gen-doc plugin.
Also, pass the plugin options via --name_opt parameter.
* Add missing status.proto dependency /language-support/java and /ledger
* Build proto docs using the proto_gen rule
To make this work, I had to turn on the bazel build flag
`--protocopt=--include_source_info` because we cannot turn enable this
flag only for specific build rules.
* Make /ledger-api/grpc-definitions:docs public again
* Revert to the old style of passing plugin arguments to --name_out=options:path
* Suppress output of unzipping
* Fix link for google.rpc.Status in proto-docs
Allows users to supply the argument -V or --verbosity with a number from 0 to 4 for additional logging.
Also, the first and last log message is logged as a warning, when it really
should just be on INFO level.
changing ports to use 6865 everywhere
no need for default ports
addressing missed outdated ports
changed more ports from 7600 to 6865
dealt with more 8080s
