* working generated representation type
* from/to method generation compiles
* moved GHC.Generics to DA.Generics
* removed strictness metainfo
* added test
* some cleanup
* deal with empty record constructors
* more fine grained testing
* Drop support for DAML-LF 1.3 from damlc
This means `damlc` now only supports versions allowing arbitrary expressions
as contract keys. This is very helpful for the upcoming new template
de/re-sugaring underpinning generic templates.
* Unpin a few DAML-LF target versions in tests
* Fix golden tests
* Adapt changelog to rebase
* Rename DAML-LF scenario tests to 1.dev
* Update tests to --target 1.dev
* Fix release notes
* Back parseInt/parseDecimal by DAML-LF primitives when available
DAML-LF 1.5 introduces two new primitives `FROM_TEXT_INT64` and
`FROM_TEXT_DECIMAL`. We translate `parseInt` and `parseDecimal` to
these primitives when compiling to DAML-LF 1.5 or later.
This fixes#1398.
* Fix DAML-LF decoder (Scala)
* Freeze DAML-LF 1.dev into DAML-LF 1.5
In other words, we release DAML-LF 1.5.
This is required for generic templates (#1387).
* description of FROM_TEXT_INT64 & FROM_TEXT_DECIMAL
* amend version history
add ``FROM_TEXT_INT64`` and ``FROM_TEXT_DECIMAL`` in the specification changelog
* typos
* Fix markup in DAML-LF spec
* Add release notes
* Drop support for DAML-LF 1.1 from damlc
Part of #853.
* Fix docs test
* Add show party test back
* Update release notes
* Remove target version from Java codegen test
* Isolate daml-on-x prototype
* Implement TransactionService of index api.
Towards the first milestone of the daml-on-x integration steps
* fix compiler error after last-minute refactoring
* formatting
* remove unnecessary dependency from api-server-damlonx
* Fix some inconsistencies.
* ghc compiles generics
* more general kind conversion
* rename varKind to tyVarKind for clarity
* added a comment regarding a check for origin of Meta constructor in conversion
* added reference to original Generics.hs and git commit hash
* added copyright header
* fixing doc generation
* update dar reader test
* no PolyKinds in generics
* functional dependencies and small fixes
* convert MetaData, MetaSel, MetaCons types
* Make the actors optional in DAML-LF's exercise instruction
If they are not present, the controllers will be filled in. Surface DAML
does this currenty anyway by fetching the contract and computing the
choice controllers before each `exercise`. This change will allow for
getting rid of the additional `fetch` preceding each `exercise`.
The compiler does not use the new form yet. I will do this in a separate
PR together with tests for the new behaviour.
This fixes#1347.
* Fix DAML-LF type checker test
* Check presence of actors for old DAML-LF versions in decoder
This is a preparation for making the actors in DAML-LF's exercise
instruction optional (see #1347).
This is technically a breaking change but since `exerciseExplicit` has never
been exposed via the documentation, we don't add a release note.
* Rename Value's ContractId to VContractId
* daml-lf: a bit more about PartyId
* daml-lf: Concatenable MatchingStringModule
* daml-lf make clear type used for Scenarios only
* daml-lf create ContractId, LedgerId, TransactionId
* sandbox-sql conversion util
* LedgerName -> LedgerString
* futher type cleanup in the sandbox
* daml-lf add test for LedgerString
* fixing tests
* a bit more safety in the DB
* Address Stephen's comments
* fix rebase
* More fixes for StringModule
* change length of LedgerString (256 -> 255)
* Lift restriction on serializable contract ids in DAML-LF 1.dev
In DAML-LF 1.dev, make `ContractId a` serializable whenever `a` is
serializable. This is part 2 of #1277.
* Reformat Scala
* Add changelog entry to daml_lf_1.proto
* Fix package valdiation for DAML-LF 1.4
Currently, complex contract keys are are not accepted for DAML-LF 1.4
although they should because we forgot to roll the version when freezing
DAML-LF 1.4.
@bitonic we need a way forward to avoid this in the future.
* Fix test
* Format Scala
* Add more tests for (un)serializable contract ids
I want to lift this restriction in a subsequent PR for the next version
of DAML-LF. Let's first make sure we have a correct implementation for the
current version though.
* fix test
* in serializability tests, check the modules are properly typed
The foldWithPathState is a fold over the transaction that allows
keeping extra state along the path of the traversal.
We add computePerPartyProjectionRoots in preparation for the
work on privacy-aware kvutils, where transaction nodes are stored
separately rather than in one blob. This function allows us to compute
the projected transaction roots for each party mentioned in the
transaction. We require this information to establish the order
of the root nodes when the size and form of the transaction is
blinded by assigning random node ids.
* Warn if there are any missing methods in type classes in DAML
Helps with #1194
* Make missing methods a fatal warning
* Update release-notes.rst
* Fix a test that had no signatories
* Update docs/source/support/release-notes.rst
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Relax syntactic restriction on contract keys in DAML-LF 1.dev
We lift the syntactic restriction that contact keys must be built using
only record constructions and projections entirely when compiling to
DAML-LF 1.dev. To make this more useful, we also search all sub-expressions
of `maintainer` in `key` during our rewriting of `maintainer` for using
`this` to using `key`.
As one of our next steps we should bring `key` into scope in `maintainer`
and perhaps deprecate the use of `this` at some point in the future.
* Fix versioning
* Adapt package validation to complext contract keys
* Added agreement_text field to the CreatedEvent in Ledger API.
* Changed java bindings + java codegen
* Changed utilities for scala codegen
* Made necessary changes in Sandbox to propagate the agreement text from ContractInst to the CreatedEvent
* Made changes to the navigator to show the agreement text in the contract details page when it is set and not empty
Fixes#1110
* Store divulgence info for contracts
* Store divulgence info for contracts
* some documenting refactors
* Use fail instead of sys.error in test
* Add release note
* Rename migration files
* Fix typo
* Fix updating divulgence info in scenarios
The ACS was comparing differently formatted contract IDs.
Fixes#1166
* stub away the components of LedgerValue
* rewrite v1 api converters to produce transaction Value ADT
* a needless 'case'
* shortcut methods for ImmArraySeq and tests
* porting extractor's JSON encoders to transaction Value
- JSON object ordering now matches record field order, instead of being
its reverse
- Includes a new encoding for records missing labels, to a list of
2-tuples
* porting extractor main transaction Value
- JSON date/time is just <num>s since epoch, maybe revisit
* adapt to Decimal newtype
* use daml-lf/data stringification for decimals in JSON
* snap aliases for Cid-less LedgerValue subtypes
* snap aliases for Cid-containing LedgerValue subtypes
* remove needless indirection
* test new missing-label Record JSON encoding
* write ImmArray traverse with syntax extension
* remove RecordField; move Name parsing to former RecordField converter
* daml-lf: make Decimal type safe
* daml-lf: create Utf8String type
* daml-lf: cleanup in data package
* Address Stephen Comments
* daml-lf: remove UTF8String
* windows: fixed daml-lf tests for Windows by using Bazel's rlocation
* more consistent logging on CI; publishing Windows test logs on failure
* windows: fix daml-lf engine tests
* windows: add diff tool to msys
* enable Return wartremover wart
* remove return keyword from various places in daml-lf
* remove return keyword from various places in ledger
* simpler ImmArray equals
* move traverseEitherStrictly to point of use
* remove return from ledger-api-server-example
* Add a helper function to qualify things with respect to a module
* Supress some warnings I saw when building
* Move Tuple* and Either to daml-prim:DA.Types, so they have a nice name for Java users (instead of GHC.Tuple or DA.Internal.Prelude)
* Add copyright header
* Update daml-foundations/daml-ghc/daml-prim-src/DA/Types.daml
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
* Update daml-foundations/daml-ghc/daml-prim-src/DA/Types.daml
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
* Update the daml-lf translation of tuple types
* Fix up the scala bindings for how to use types
* Clarify what action people need to take following the release notes
* Fix an expected test
* Clarify the changelogs even more
* Run ScalaFmt
* Clean up the release notes a bit
* Use the changelog from Beth
* Update the release notes again
* Update docs/source/support/release-notes.rst
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Fix up the scenario error messages
Up to now, the engine blindly assumed that contract ids pointed to
contracts of the right type. However, this assumption is faulty:
contract ids coming from the Ledger API cannot be type checked
in command translation since we need access to the contract itself
to do so.
This caused some seriously surprising / broken behavior: one could
send an exercise command with the wrong template id and still go
through, or break internal invariants about the type of choices.
This commit fixes this by checking that the type of the contract
instances we fetch is correct at runtime.
cc @hurryabit @dajmaki @remyhaemmerle-da @S11001001 @meiersi-da
* Re-order fields in records if all labels are provided
Fixes#988
* Add line to release notes
* Fix line on release notes
* Document change on the Ledger API definition
* Simplify pattern matching
* Improve and fix tests
Integration tests now look for the "Missing field label" rather then the
"Mismatching record label" when inspecting errors. Furthermore, a test
has been added that ensures that repetitions in labels still raise an
error.
* Add comment
Comment about relationship between fields length check and fields reordering based on labels
Co-Authored-By: stefanobaghino-da <43749967+stefanobaghino-da@users.noreply.github.com>
* Improve Ledger API docstring
Explicitly state the semantics of labeled fields (i.e. repeated keys are not accepted)
Co-Authored-By: stefanobaghino-da <43749967+stefanobaghino-da@users.noreply.github.com>
* record keys locally to a single update, fixes#969
* do not delete keys in machine for nonconsuming choices, fixes#973
* amend release notes
* actualize new test results
* Drop LedgerTestingHelpers.asyncFromTimeService as unused.
* ledger-api-test-tool: Add clues to asserts in the main code paths.
This ensures that users get more informative errors for the currently used
semantic tests.
* semantic tester: add operation description to an underlying ledger.
This allows to provide richer error messages to users of LedgerApiTestTool.
* Map case of Equal[Value] was not properly recursive
* protect SortedLookupList from equals incoherence
* simplify ImmArray creation
* clean up boilerplate in Equal[Value] definition
* use match2 for isReplayedBy
* Allow to mangle names used in scenarios before their executions.
This allows to run a scenario against a long-running server repeatedly and avoid
clashes between runs, since each party is unique (up to the randomness used).
* ScenarioRunner: add test for partyNameMangler
* clean up v3 contract key case in TransactionVersions
* clean up duplicate cases in ValueVersions.assignVersion
* make LanguageMinorVersion a proper ADT, aliased LanguageVersion.Minor
* port DecodeV1 to LanguageMinorVersion ADT via implicit conversion
- @remyhaemmerle-da might prefer the explicit application of Stable/Dev
* make releasePrecedes private
* add dev versions to the timeline
* copyright header for LanguageMinorVersion
* build the package database for 1.dev, support 1.dev as --target
* test presence and new-ness of "dev" in timeline
* report dev versions in engine info; note 1.dev target in release notes
* spec and governance rules on 1.dev and minor dev in general
* make the governance rule about dev more direct
* 1.x
* missed plural; thanks @leo-da
* further definition of 1.dev in LFv1 spec; thanks @remyhaemmerle-da
* remove outdated flags from comments
* update tests in preparation for visibility rules
* check visibility of contract keys, fixes#751
* add release notes
* include visibility check in scenario runner
* stephen's suggestions
* fix type error in api server example
* scalafmt
* bring TransactionFiltration.scala up to speed with disclosure
see
<faf1cde1a1/ledger-api/grpc-definitions/com/digitalasset/ledger/api/v1/event.proto>
for an in-depth description. fixes#794.
* add tests for contract key visibility
* move a lot of ledger testing utils out of command checks
* move a lot of ledger testing utils out of command checks
* remove useless checks regarding divulgence
* add tests regarding event witnesses
* update release notes
* re-introduce safety when disclosing
Also move Interface and InterfaceType out of the reader subpackage; they
belong with the rest of the data model at the iface root.
The specific mechanics of reading a Dar all the way to producing an
EnvironmentInterface are left to Scala codegen's Codegen and Java
codegen's CodeGenRunner; there's no consensus or great stability on the
best way to tie these pieces together, but all the pieces might as well
be available in the interface library at least.
Inspired by a query on Slack by @leonelag regarding reading the
codegen-relevant parts of dalfs and dars; thanks!
* daml-lf: move EnvironmentInterface to interface library from Scala codegen
* daml-lf: move Interface out of reader subpackage
* language-support/java: deal with moving Interface out of reader subpackage
* document Interface and EnvironmentInterface
* missed copyright header in reader package.scala
* extractor: deal with moving Interface out of reader subpackage
* navigator: deal with moving Interface out of reader subpackage
Add field 'usedPackages' to Transaction to record the packages
that were used during interpretation to produce said transaction.
This serves as a hint to specify what packages are required to
validate the transaction. Note that this may not be sufficient
for future implementations that e.g. compile DAML-LF to JVM
bytecode and require packages containing only type definitions
as inputs.
This hint is mainly required for ledger implementations for which the
packages are tracked similarly to contract instances, and which
require that all inputs to their transaction must be declared
beforehand.
This work is required for the participant-state key-value utilities
described in issue #410 and implemented in PR #637.
* removing major LF dev version from Haskell proto codecs
* removing major LF dev version from scenario service client
* missed import
* remove Scala support for dev major version; remove --allow-dev option from sandbox cli
* Version.minorFromCliOption function
* don't build daml-stdlib artifacts for dev major
* remove damlc CLI --target dev
* release note about removed dev major LF version
* governance now discusses minor dev, no more major dev
* don't build from daml_lf_dev.proto anymore
* remove daml_lf_dev.proto
* raise deprecated release
* reserve 9999 in the ArchivePayload sum, as suggested by @bitonic
* use reserved proto keyword, as suggested by @bitonic
- `reserved` cannot occur within `oneof` block
* remove --allow-dev test
* dev removal release note followed the previous release; move it back to HEAD
* Move to using proto3-wire from upstream
* Move to upstream proto3-suite, with some custom patches in my fork
* Delete the BUILD.bazel for hte proto3 stuff, not used and the test was failing
* Delete the old proto3-wire and proto3-suite forks
* Delete proto3-wire
* Prettify BUILD.bazel files, sort the deps
* Remove some special cases from the license checker
* Delete unused Nix files from grpc-haskell
* Switch to upstream proto3-suite
* Make old-time work on Windows
* Formatting
* Patch rules_haskell to use a response file for -optP to avoid overflowing argument size limits on Windows
* Update 3rdparty/haskell/BUILD.old-time
Co-Authored-By: neil-da <35463327+neil-da@users.noreply.github.com>
* Update the comments in old-time
* Use the revised location of proto3-suite
* GenTransaction.WithTxValue alias
* undo parts of parent dealing explicitly with serialization
* use GenNode.WithTxValue more often
* use Node*.WithTxValue more often
* daml-on-x: Drop StateInit and use AtomicReference
- Drop the StateInit update message and introduce "getLedgerInitialConditions".
- Switch to AtomicReference from SyncVar as this is non-blocking and hence safer to use from Akka.
* ledger/participant-state: Rename recordTimeEpoch to initialRecordTime
and use the time from timeProvider at the ledger object initialization
time as the initial record time.
* fmt
* Fix use of repo as external workspace. Fix compilation issue in IndexState.scala
* Refix the use of this repo as external bazel workspace
* Fix compilation after rebase
* refmt
* seal engine.Error
* don't reinvent flatMap in CommandExecutorImpl
* add DivulgenceVisibility sandbox test case
- suggested by @oggy-; thanks
* add global implicit disclosure step to sandbox ACS transaction acceptance
* reorder ActiveContractsManager and include an implicitlyDisclose action
* add submitter argument to SubmissionHandle#lookupActiveContract
- the receiver must filter its response based on witnesses thereof
* propagate implicit disclosure in more places
* it's a bunch of functions
* proper error from the no-divulgence test
* rename ActiveContracts and ActiveContractsSteps to ...InMemory and ActiveContracts
* we need another table
* rename ActiveContract.witnesses to disclosedTo
* don't exclude other disclosures in ActiveContractsManager
* add local implicit disclosure when adding transaction to sandbox acs
* extra argument in ledger-api-server-example
* too late to blind?
* adapt to 10fcab8#339
* scalafmt
* non-total map get
* disable postgres test in SandboxSemanticTestsLfRunner until implicit disclosure added
* un-flatten witnesses and divulgences from "disclosedTo"; fixes DivulgenceIT
* comment on filtering of lookupActiveContract
- suggested by @francesco-da; thanks
* semantic errors from parent
* scenario to test divulgence success
* submit line 6 as owner; disable divulgence test under SQL
* comment what else to reenable
* Add release notes entry
* Add CreateAndExercise command to Java Bindings data layer
* Add CreateAndExercise command to DAMLe
The CreateAndExerciseCommand allows users to create a contract and
exercise a choice on it within the same transaction. Users can use this
method to implement "callable update functions" by creating a template
that calls the update function in a choice body.
Fixes#382.
* Add CreateAndExercise command handling to the sandbox
* Add CreateAndExercise command to the Ledger API
* extracting large command test from TransactionServiceIT
* make `preprocessCommands` more performant (see comment)
* fix duplicated command generation in `preprocessCommands`
* enable in-memory test-fixture as well
* Add semantic test for the reference server
Currently the semantic test is failing. Likely due to the
location annotation changes.
* Do not compare location annotations in isReplayedBy
The location annotations may not, and do not need to, match due to the
fact that the reconstructed update expression may not exactly match
the original one, and since the interpreter currently picks the closest
location annotation we cannot guarantee that they exactly match.
* scalafmt.
* buildifier.
* client_server_test: Increase timeout to 60s
Spawning a JVM-based server can easily take a long time on a very
loaded system (e.g. when running `bazel test //ledger/...` with enough
parallelism), so better have a high default timeout.
* ledger/api-server-damlonx: Address code review
* Fix client_server_test runner compilation. Bump timeout.
* Mark the reference server semantic test exclusive
* fmt reference/BUILD.bazel
Add Dar Traverse and Equal, replace manually written test cases with
scalaz law checks;
Add test case for multiple DARs support;
Enforcing non empty list of input files
The DAML-LF Archive Protobuf definitions, as packaged and distributed
via Bintray and the SDK, currently packages the files in a directory
structure that does not match with the actual one, causing the usage of
`protoc` on the packaged definitions to raise errors. This commit fixes
it by packaging the files in the same directory structure as the one
found in the repository.
* fix various conversion functions from string to Decimal
Fixes#399.
This fixes a critical bug -- since:
* The DAML-LF spec specifies that the scale of `Decimal` is 10 --
that is, there are at most 10 digits past the decimal point:
<79bbf5c794/daml-lf/spec/value.rst (field-decimal)>.
* However, the code converting from the string representation that
we get on the wire was _not_ performing this check. This is due
to two reasons:
- `Decimal.check` is a function that checks whether a given
`Decimal` is within the upper and lower bounds of what the
DAML-LF spec specifies, but crucially it does _not_ check that
the scale is not exceeded:
<79bbf5c794/daml-lf/data/src/main/scala/com/digitalasset/daml/lf/data/Decimal.scala (L31)>.
This behavior is correct in some cases (more on that later),
but not in others. Crucially, `Decimal.fromString`, which was
supposed to check if a decimal string literal is valid, used
this function, which means that it accepted string literals
containing numbers out of the required scale, rounding them to
make them fit within the scale. This function has been renamed
to `Decimal.checkWithinBoundsAndRound`, and a new function
`Decimal.checkWithinBoundsAndWithinScale` has been added, which
fails if the number provided has data not within the scale.
`Decimal.fromString` now uses
`Decimal.checkWithinBoundsAndWithinScale`.
- `ApiToLfEngine.parseDecimal` assumed that `Decimal.fromString` _did_
fail when passed numbers that were in any way invalid, and
moreover did _not_ use the result of `Decimal.fromString`, but rather
re-parsed the string into an unconstrained `BigDecimal`:
<79bbf5c794/ledger/ledger-api-common/src/main/scala/com/digitalasset/platform/participant/util/ApiToLfEngine.scala (L96)>.
The reason for the code to work this way is that in the past
it was responsible for converting decimal strings both for the
current engine but also for an older version of the engine which
handled decimals of a different type. Both issues have been fixed.
* Therefore, `Decimal`s with scale exceeding the specified scale
made it into the engine, and contracts could be created containing
this invalid value.
* Once on the ledger, these bad numbers can be used to produce extremely
surprising results, due to how `Decimal` operations are
implemented. Generally speaking, all operations on `Decimal`
first compute the result and then run the output through
`Decimal.checkWithinBoundsAndRound`. The reason for this behavior
is that we specify multiplication and division as rounding their
output. Consider the case where the bad value 0.00000000005 made
it to the engine, and is then added to 100. The full-precision
result will be 100.00000000005, which after rounding becomes 100.
Therefore, on a ledger where such invalid values exist, it is not
the case that `y > 0 ==> x + y != x`, and so on.
Thanks a bunch to @briandbecker for the excellent bug report.
* fix failing test using to much precision
* Add buildifier targets.
The tool allows to check and format BUILD files in the repo.
To check if files are well formatted, run:
bazel run //:buildifier
To fix badly-formatted files run:
bazel run //:buildifier-fix
* Cleanup dade-copyright-headers formatting.
* Fix dade-copyright-headers on files with just the copyright.
* Run buildifier automatically on CI via 'fmt.sh'.
* Reformat all BUILD files with buildifier.
Excludes autogenerated Bazel files.
* Add location information to DAML-LF produced by damlc
This is required to get error locations in the scenario view. Rigth now,
the location information for `create`/`exercise` still points to the
template/choice. I'll fix that in a separate PR.
* Fix test expectations
* Fix more tests
* Do not divulge contracts to observers in nonconsuming exercises
Disables support for non-default ledger feature flags, as they
are meaningless since ledger server logic does not respect the flags.
Instead of large refactoring to add support for the old flag settings,
it is best to disallow the deprecated flags, and later on phase out the
flags completely.
Re-enables test_divulgence_of_token in sandbox semantic tests.
Fixes#157.
* purge LedgerFlags entirely...
...since we only support one version of them anyway, and clearing them
* updated release notes
* Implement legacy DAR support, #308
if MANIFEST.MF is not present filter zip entries by .dalf extension,
expect one main DALF and one optional *-prim* DALF, fail in other cases
* Implement legacy DAR support, #308
if MANIFEST.MF is not present filter zip entries by .dalf extension,
expect one main DALF and one optional *-prim* DALF, fail in other cases
* comments, naming
* Fetch status.proto from remote, simplify JS gRPC codegen
Fetch the `status.proto` file (part of the standard gRPC distribution)
from a distribution channel. _Moreover_, use the recently introduced
`proto_gen` rule to simplify how the gRPC code for the Node.js bindings
are generated (and remove the need to have `google/rpc/status.proto`
locally in the repository.
* Add plugin_runfiles option to proto_gen
This allows use to add additional files to the bazel sandbox so that
plugins can refer to them. This will subsequently be used by the
protoc-gen-doc plugin.
Also, pass the plugin options via --name_opt parameter.
* Add missing status.proto dependency /language-support/java and /ledger
* Build proto docs using the proto_gen rule
To make this work, I had to turn on the bazel build flag
`--protocopt=--include_source_info` because we cannot turn enable this
flag only for specific build rules.
* Make /ledger-api/grpc-definitions:docs public again
* Revert to the old style of passing plugin arguments to --name_out=options:path
* Suppress output of unzipping
* Fix link for google.rpc.Status in proto-docs
* Change contract key maintainers to have template argument in scope
More precisely, change the contract key type class we desugar the surface
syntax to such that the maintainers function has the template argument
in scope instead of the key. The compiler rewrites all references to the
template argument into references to the key and fails whenever that is
not possible.
* Add TemplateKey class to desugaring module
* Make template key check slightly more lenient
* Fix TODO descriptions
* decode key expressions in scala correctly. fixes#177
replaced a `foldRight` with a `foldLeft`, projections were decoded
backwards.
* remove ContractKeysSimple -- superseded by ContractKeys
