* daml-lf: rename Map to TextMap in archive proto
+ in Scala/haskell AST
* a bit more renamming
* Update compiler/daml-lf-tools/src/DA/Daml/LF/TypeChecker/Serializability.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* fix test
* Apply suggestions from code review
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Add InsertOrdSet
A set with deterministic ordering.
* Add party to participant mappings as inputs to kvutils submissions
* Add InsertOrdSetTest, remove fromSet. Sort packages and parties.
* Apply suggestions from code review
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Add AbstractSet and SetLike traits to InsertOrdSet. Remove (Non)EmptyInsertOrdSet.
* Expose conversion from Ast.Type to iface.Type
This allows me to get rid of the duplicated conversion logic for DAML
script. The reason for why I can’t use the higher level APIs provided
by the interface reader is that the type of the script identifier can
be a function which is not serializable and therefore does not show up
in the interface. However, I only want to translate the type of the
argument of that function which is serializable.
* Update daml-lf/interface/src/main/scala/com/digitalasset/daml/lf/iface/reader/InterfaceReader.scala
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
This uses the format for LF values that we already use elsewhere.
There is one annoying part in this PR where I had to duplicate the
logic for converting to the types used in the interface reader since
it is not exposed but hopefully we can get rid of this soon in a
separate PR.
fixes#3470
* comparison query parser given scalar parser
- written in half-error-propagation style to better suit other potential
error features
* factor dupes in RangeExpr
* text range parsing
* interpreting Range into in-memory predicate; points out bad dedupe from earlier
* make reuse of the scalar extractors much nicer
* express date, time, int64 as factored-out range exprs
* express numeric as factored-out range expr
* factor \&/ usage
* refactor LF value extractors for reuse in range queries
* factor mkRange usage
* totally deconstruct the int64 and text cases
* totally deconstruct the date and timestamp cases
* totally deconstruct the numeric case
* document comparison queries
* use Utf8.Ordering for text comparison queries
* int64 range query tests
* more int64 range query tests
* date, string, numeric range query tests
* include line # in query test successes table
* timestamp range query tests
* add release note
* remove duplicate changelog entry from #3425
The code still needs a fair amount of cleanup but it seems to work and
there is a test so I’d like to do the cleanup in-tree after merging
the current state
* Use TreeMap for storing transaction nodes
Going forward we require deterministic serialization for transaction nodes.
This PR switches the `Transaction.nodes` to use a TreeMap instead of HashMap,
which gives us deterministic serialization order for the transaction nodes.
An alternative would be to sort the transaction nodes when serializing, but
I feel avoiding HashMap altogether is the safer option as this may bite us
elsewhere as well (e.g. tests, new serialization code, etc.).
* Address review
* Remove unused import
* Add GenMap to DAML-LF 1.dev
* Update LF spec
* remove type constraints again
* Revert "remove type constraints again"
This reverts commit 2081e5b8ef.
* Fix a messed up merge
* bazel: 0.28.1 --> 1.1.0
* bazel-watcher sha256
* Fix missing line in patch
* proto_source_root --> strip_import_prefix
See https://github.com/bazelbuild/bazel/issues/7153 for details.
* Update rules_nixpkgs
Required to avoid errors of the form
```
ERROR: An error occurred during the fetch of repository 'node_nix':
parameter 'sep' may not be specified by name, for call to method split(sep, maxsplit = None) of 'string'
```
and
```
ERROR: An error occurred during the fetch of repository 'node_nix':
Traceback (most recent call last):
File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
_execute_or_fail(repository_ctx, <3 more arguments>)
File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
fail(<1 more arguments>)
Cannot build Nix attribute 'nodejs'.
Command: [/Users/runner/.nix-profile/bin/nix-build, /private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/node_nix/nix/bazel.nix, "-A", "nodejs", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
src/main/tools/process-tools.cc:173: "setitimer": Invalid argument
```
* Update rules_scala
* .proto has been removed, use [ProtoInfo] instead
See
https://docs.bazel.build/versions/1.1.0/be/protocol-buffer.html#proto_library
* python3_nix add nix_file attribute
To avoid the following error
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/BUILD:66:1: //:nix_python3_runtime depends on @python3_nix//:bin/python in repository @python3_nix which failed to fetch. no such package '@python3_nix//': Traceback (most recent call last):
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
_execute_or_fail(repository_ctx, <3 more arguments>)
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
fail(<1 more arguments>)
Cannot build Nix attribute 'python3'.
Command: [/home/aj/.nix-profile/bin/nix-build, "-E", "import <nixpkgs> { config = {}; overlays = []; }", "-A", "python3", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
error: anonymous function at /home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs.nix:3:1 called with unexpected argument 'config', at (string):1:1
```
* rules_haskell unnamed string.split(_, maxsplit = _)
The keyword argument may no longer be named.
* string.replace(_, _, maxsplit = _) may not be named
* Move proto sources from deps to data
Fixes
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/daml-lf/archive/BUILD.bazel:150:1: in deps attribute of scala_test rule //daml-lf/archive:daml_lf_archive_reader_tests_test_suite_src_test_scala_com_digitalasset_daml_lf_archive_DecodeV1Spec.scala: '//daml-lf/archive:daml_lf_1.6_archive_proto_srcs' does not have mandatory providers: 'JavaInfo'. Since this rule was created by the macro 'da_scala_test_suite', the error might have been caused by the macro implementation
```
* Define sha256 for haskell_ghc__paths
Bazel 1.1.0 fails on missing hashes.
* Disable --incompatible_windows_native_test_wrapper
* //compiler/daml-extension don't modify sources
Modifying sources in-place can cause issues on Windows, where build
actions are not sandboxed and changes on sources can affect other build
steps.
* bazel-genfiles --> bazel-bin
The bazel-genfiles symlink has been removed since Bazel 1.0.
See https://github.com/bazelbuild/bazel/issues/8651
* Mark dev_env_tool repository rule as configure
See
https://docs.bazel.build/versions/1.1.0/skylark/lib/globals.html#repository_rule
* Move data deps into data attribute
* Mark dev_env_tool as local = True
* Manually fetch @makensis_dev_env
* Fix the import paths of {value,transaction,blindinginfo}.proto
This cleans the imports in daml_kvutils.proto:
-import "daml-lf/transaction/src/main/protobuf/transaction.proto";
+import "com/digitalasset/daml/lf/transaction.proto";
This shouldn't be a breaking change to the outside as the proto files
themselves are not published, only the java libraries generated from them.
* Update missed deps
When a package has dependencies, `ConcurrentCompiledPackages#addPackage` returns
a `ResultNeedPackage` on the dependency.
This return value was ignored in `Result#needPackage`. As a consequence,
the package would never be added to the map of compiled packages in
this way.
This is problematic for the reinterpretation of `Fetch` nodes, when
performed on an `Engine` with a clean cache. The package would never get
loaded, but the interpretation would commence and then blow up. See
the modified test in `EngineTest.scala`.
The problem would not appear for other types of nodes, because the other
`CommandPreprocessor#preprocess*` functions (e.g., preprocessCreate)
also invoke `translateValue`, which correctly chains the result of `addPackage`.
* Update bazel-common to fix javadoc issues
Specifically, to fix the following error
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
javadoc_library(name = 'rs-grpc-bridge_javadoc')
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```
* Define Maven deps using rules_jvm_external
* Pin artifacts
* Remove bazel-deps generated targets
* Remove bazel-deps
* Switch to rules_jvm_external targets
* update bazel documentation
* pom_file: There are no more bazel-deps targets
* BAZEL-JVM.md `maven_install` typo
* Add TemplateTypeRep to AnyContractId
* Define Trigger.ContractId t
* Use Trigger.ContractId t
* Implement fromCreated and fromArchived
* instance MapKey TemplateTypeRep
* More efficient ACS access using Map TemplateTypeRep
* ./fmt.sh
* toString and fromString for Identifier
* Replace Identifier by TemplateTypeRep
* TheContractId --> AbsoluteContractId
https://github.com/digital-asset/daml/pull/3245#discussion_r338033546
* speedy: limit allocation of trivial SValue constants
* daml-lf: limit allocation of constants for Value, Exp, Type
* weaken numeric equality in ledger service test
* Properly fill eventId for active contracts
This gets rid of the last remaining bit that assumes
contractId==eventId.
Fixes#65.
Contributes to #2068.
* Do not conflate eventId and contractId in the daml-lf interpreter
* Do not treat contractId as eventId in Ledger.scala
* Remember the transaction that divulged a contract.
* In this scope we can treat divulged contracts the same as disclosed ones
* revert a few more syntactical changes to make the overall diff smaller
* retain the same behavior on the scenario service api
* fix unreleased after rebase
* Generalize AnyTemplate type to Any in DAML-LF
See #3131 for the motivation for this. The tl;dr is that we need
something like AnyTemplate for choice types as well.
Since the protobuf was already more general in anticipation of such a
change, this change only changes the internal AST on the Haskell and
Scala side.
Since AnyTemplate change has never made it out of 1.dev, I updated the
changelog in the LF spec instead of adding a new entry.
* Update daml-lf/spec/daml-lf-1.rst
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* windows debugging
* more windows debugging
* clean expunge
* don’t cat the config file
* remove comment on type equality
* windows …
* gnah
* foobar
* foobar
* does anything ever work?
* reenable caching
* Do not build daml-lf-ast separately
* engine: create persisted value for numeric
* engine: update info test
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* engine: weaken scale constraints when transalting value
* Address Stephen's comments
* Address one of Stefano's comments
* daml-lf: fix transaction test
* Intern all strings and dotted names in the DAML-LF protobuf encoding
On one of our huge DAML code bases this has brought down the size of the
DALF by 60% and cut the memory usage during decoding by 50%.
The `@QUERY-LF` tests in the damlc integration tests check that the
interning actually works.
* Add scala decoder support for interned strings
* Memoize versionisOlderThan. Speeds up decoding by 6x.
* Properly fix the performance problem with version ordering
LanguageVersion.ordering was a `def` and not `val` causing us to
recreate the ordering tables every time.
* Clean up //daml-lf/archive:decode-test
* Use int32 for interning indices
* Adapt QUERY-LF tests
* Strip a few useless field_ prefixes
* Add test for uniqueness in interning tables
* Drop field_ prefix from jq queries
* Fix Scala encoder
* Fix bond-trading tests
* Rename decode-test to decode-tester
* Address @cocreature's comments
* daml-lf: proper versioning for reading internedStrings
* daml-lf repl: revert change
* Numeric implementation
* Dealing with all sorts of numeric literals.
* Fix DA.Generics
* Reduce code duplication with IF_NUMERIC
* Simplify Prelude with IF_NUMERIC
* Fix daml-lf validation for MUL_NUMERIC and DIV_NUMERIC
* ADT for json-api's LF value query predicates
* expand predicates into primitive cases
* expand more cases for the predicate reader
* instantiating type references
* record case
* get super-reductionist with ValuePredicate so we can tease out the proper model by operations
* just put the literal predicates in the ADT
* more primitive predicates
* properly build RecordSubset predicates
* tests for ValuePredicate LF value predicate functions
- false cases fail because outer layer still stubbed
* outer RecordSubset always used
* tests for comparing literal predicate and ApiCodecCompressed parsing
* map querying
* Adding ListMatch
* removing List test cases covered by LF-JSON encoding tests
* Optional support; precompile ListMatch elements
* use go instead of toFunPredicate
- deals with potential subst introduction as we make VP more abstract
* test cases for optional queries
* Adding VariantMatch
* Fixing comments
* some precompiling and fusion
* Adding value predicate to contracts/search endpoint, WIP
* Adding value predicate to contracts/search endpoint, WIP
* add filtering to ContractsService#filterSearch
* Adding contracts/search result filtering based on the constructed predicates
* cleanup
* make error throwing in ValuePredicate a little nicer
* scalafmt
* cleanup
* follow JSON format rules for numeric queries
* test that apiValueToLfValue . lfValueToApiValue = arr id
* API value -> LF value discards scale; account for that in ValuePredicate
- fixes queries that match numerics
* document query
* release note
* test for contracts/search with query
* more tests for contracts/search with query
* More tests and cleanup
* more cleanup
* better test
* Adding disclaimer
* records, not documents
* better error message in the fromCon case
* Add Any type and to_any/from_any primitives to protobuf
Following a suggestion by Rémy, the protobuf representation is more
general and is associated with an arbitrary type instead of a
typecon. This allows us to easily extend this later to a full Any
type.
I’ve still called the type in the protobuf Any instead of Haskell’s
Dynamic since I find AnyTemplate more clear than DynamicTemplate and
having AnyTemplate and Dynamic seems confusing.
Right now, the decoder enforces that the type is a TypeCon.
* Fix some mistakes in the spec
* Update daml-lf/spec/daml-lf-1.rst
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update daml-lf/spec/daml-lf-1.rst
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update daml-lf/spec/daml-lf-1.rst
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update daml-lf/spec/daml-lf-1.rst
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Add evaluation rule for to_any_template
* Update daml-lf/spec/daml-lf-1.rst
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* daml-lf: add CAST_NUMERIC and SHIFT_NUMERIC internally
* daml-lf: add CAST_NUMERIC and SHIFT_NUMERIC to archive proto
* daml-lf: update spec with CAST_NUMERIC and SHIFT_NUMERIC
* update release notes
* fix spec
* Address comments from Fran and Gerolf
* fix unrel
* Implement AnyTemplate DAML-LF type on the Scala side
This is the first part of
https://github.com/digital-asset/daml/issues/2876. The PR adds
AnyTemplate to Speedy and to the internal expression representation
and adapts all the relevant infrastructure (e.g., the typechecker) and
the tests.
It does not yet change the protobuf representation, the Haskell side
or the spec. I’ll update the spec together with changing the protobuf.
* Add comments to SBToAnyTemplate and SBFromAnyTemplate
* Address some comments from Remy
* Only allocate TBuiltin(BTAnyTemplate) once
* Preload packages to engine during upload
* Improve logging in KeyValueCommitting and add timing information
* Fix scenario service tests now that logging is done in interpreter
This is a first step towards DAML triggers. At the moment, triggers
can consume (very simplified) create and archive events via the Ledger
API, update a state based on that and emit log mesages at each update.
All of this is likely to change significantly in the future, so I
would prefer to not focus too much on minor details for now.
As a test, I added a simple trigger that tracks active contract ids.
* Add a method to initialize speedy from a non-update expression
We need something like this for DAML triggers. You can kind of fake it
since all the individual fields of Machine are exposed but `initial`
is private (for good reasons as it sets ctrl to null) which makes this
rather annoying to do.
* Base other initialization methods on fromSExpr
* add Numeric.java
* ledger-api: rename `decimal` field to `numeric` in value protobuf
* Address Gerolf's comment
* ledger-api: add missing renammings
* ledger-api: relax syntax of numbers that can be sent as numerics
* extractor: fix
* leger-api: change format of number though ledger api
* daml-lf: fix numeric regexp
* ledger: fix tests
* engine: factorize value translation out of CommandPreprocessor
* ledger-api-test: add semantic equality for events
* engine: fix package resolution for ValueTranslator
* Initial version for configuration submissions and authorization
* Refactor config submission based on review. Add tests.
* Cleanup test output and comments
* kvutils test-suite
- Add unit tests for kvutils
- Thread the input state into Commit monad
- Fix a bug in handling rejected transactions (from previous refactoring)
* Address PR review
- Add submissionId to configuration changes.
- Revert the dedup of the ParticipantNotAuthorized message. This did not buy much
and we need to duplicate it anyway when modularizing kvutils.
Currently, the `SExpr` class has a method `apply`, which is basically an
alias for `SEApp`. Using this method makes the code harder to read, at
least of me. After all, ASTs are _data_ and not _control_. Thus, having
a DSL for them seems a bit overkill.
* daml-lf: rename data.assert function to data.assertRight
* daml-lf: implement internal Numeric type
* Apply suggestions from Gerolf
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Address Gerolf's comments
in particular add tests for round and toLong
* Make stack traces work with cached top level values
For stack traces to work properly in the presence of cached top level values,
we need to cache the stack trace together with the value and restore the
stack trace when we get the value from the cache.
We also need to push location information under the monadic. Otherwise, the
location information will be removed from the continuation stack before the
monadic action is _executed_.
We also change one test case to use `fail` instead of `error` since `fail` is
more tricky than `error` (due to the lazy expression embedding in DAML-LF).
Unfortunately, the test did not work woth `fail` in the past because of the
issues fixed in this PR.
* Explain special cases when pushing location information
* Show function names in stack trace on failing scenario
So far, we've only shown the location of the function but not its name.
Now, we add the name of the function as well.
* daml-lf: make match2's fallback lazy, and add tuple2 syntax for better inference
* lf-value-json: when decoding, branch on type, *then* on value
- restores totality checking for interface ADT
- makes alternate input formats obvious at a glance
- lets us restructure in the future to cache type branch lookups
* lf-value-json: support numbers for int64s
* lf-value-json: test failures and more timestamp cases
* lf-value-json: adapt timestamp decoding to new rules
* lf-value-json: cleanup
* lf-value-json: decode Number as decimal
* lf-value-json: forbid yet another timezone format
* lf-value-json: don't copy
* lf-value-json: replace {Some, {None optional format with nest-sensitive [] format
* lf-value-json: round decimals rather than rejecting, in either string or number form
* lf-value-json: support output to JsNumber for int64s and decimals, separately
- not dependent on JavaScript safe integer range, because that is not
the purpose of this option
* lf-value-json: uncomment some now-tested rounding cases
* allow None record fields to be omitted in object syntax
* release notes
* Print stack traces in the scenario on failure
Currently, we only print the last source location, which is not
particularly helpful for debugging. Now, we put all source locations we
encounter during execution on the continuation stack and print them when
a scenario fails. This PR does not print the names of entered functions
or choices. We leave this for a future PR.
* Address Moritz' comments
* daml-lf: base interface reader on archive decoder
* fix call to InterfaceReader.readInterface
+ cosmetic changes
* daml-lf: decoder can process only serializable data
* New acceptance test tool prototype
* Fix transaction filter helper
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r308081386
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r308082432
Moreover, creates specific helpers for test DAML code.
* Move helpers from session to context
Remove a circular dependency between session and context, and limits the
usage of implicits.
* Run tests with ledgers out of process
* Prepare for drop in
* Temporary timeout increase
* Move bulk of the logic to the test runner
* Have a map of tests for CLI integration
* Timeout per-test, runner uses context, renaming
* Allow usage of time service, integrate semantic tests
* Integrate CLI SSL configuration
* Integrate failure expectation CLI option
* Integrate with failure output verbosity CLI option
* Integrate with test listing and picking CLI options
* Integrate with timeout scaling CLI option
* Integrate with command TTL CLI option
* Review CODEOWNERS
* Drop previously unimplemented CLI parameters
* Expand test todo list to encompass latest additions
* Deduplicate some testing infrastructure, swap in new test tool
* Remove unused rule
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311045444
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311051974
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311035287
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311055290
* WIP
* Make gRPC thread factory spawn non-deamon threads
* Remove debug prints, complete divulgence test (missing assertion messages)
* Try to detect assertion failure line number
* Add Sandbox binary that starts an ephemeral postgres instance
This allows us to use the client_server_test bazel macro to run against
the sandbox backed by postgres.
Fixes#1543
* Add Sandbox binary that starts an ephemeral postgres instance
This allows us to use the client_server_test bazel macro to run against
the sandbox backed by postgres.
Fixes#1543
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311400695
* Remove TlsConfiguration duplicate, move original to commons
* Make assertion more readable
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Fix build errors introduced in previous commit
- exclude time service tests from reference-v2
- add copyright header to TlsConfiguration
* Complete divulgence test and address review comments
Restores comments from original divulgence tests and strives to be as
close to them as possible.
- Address https://github.com/digital-asset/daml/pull/2282#discussion_r311429169
- Address https://github.com/digital-asset/daml/pull/2282#discussion_r311431156
- Address https://github.com/digital-asset/daml/pull/2282#discussion_r311431913
* Delete old divulgence tests
* Restore methods to fetch contracts by template identifier
* Run conformance tests on in-memory sandbox
* Run conformance tests on the persistent sandbox
* Remove workflow identifier workaround
* New contract keys test (#2452)
* Set channel type for LedgerSession
* Add ContractKeysIT
* Run tests against reference-server v2
* Removed the old ContractKeysIT. However, ...
... I cannot remove ContractKeysChecks completely, because we have
another test path for DARs created prior to DAML-LF 1.6, which doesn't
have the submitter == maintainer restriction for contract keys.
* Use codegen instead of custom code
* Add missing copyright headers
* Drop sandbox conformance test suite (seems to not work on Windows CI)
* Make time simpler
* Test tool tests to run only semantic tests
* Make CI behave more consistently
* Add missing files to run tests on Postrgres
* Fix flaky divulgence test, introduce readable identifiers
* Increase sandbox conformance tests timeout
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r312363246
* Reintroduce test resource extraction
* Read semantic tests from the classpath rather than Bazel
* Use codegen types for ContractKeys integration test
allocateParty returns Primitive.Party and create and exercise also
accept Primitive.Party. This way we don't have to keep a String and
Primitive.Party representation around.
* Bump timeout for sandbox conformance tests
* Remove dependency on Bazel runfiles
* use Arbitrary for TypedValueGenerators to get existing 'sized' uses
* test Option[Option[Long]] specifically
* derive Arbitrary ValueMap solely by implicit means, for the sized
* make ApiCodecCompressed Cid-generic
* first sample JSON tests
* navigator adaptation to Cid genericity
* scalafmt
* Table-ify JSON tests
* more navigator adaptation
* adapt http-json to Cid-agnostic JSON codec
* stem the tide of namespace pollution wrought by the "mixin to import" style
* show line # of failed JSON cases
* naturally limit the nesting factor of generated types
* move SortedLookupList generation to data-scalacheck library
* more sample JSON cases from design doc
* add more parameters to ApiCodecCompressed
- AbsoluteContractId seems like it will give rise to incoherence
- and there are going to be two parameters
- so we might as well just parameterize the whole codec
* test canonical and equivalent encoded forms
* test lists
* test very very nested optionals
* many more optional-related tests
* new data-scalacheck library for ImmArray, FrontStack, et al
* add checkLaws functions as library to data-scalacheck
* make checkLaws more configurable at use points
* Update ghc-libs to use new template desugaring
* Replace old template typeclasses with generic-friendly ones
* New template desugaring doc
* Fix tests
* Fix damldoc tests regarding Archive choice
* Update visualisation code to not traverse master dictionary
* Additional class method stubs to template instance declaration for upgrades
* Increase stack limit for bond trading compilation test
* Update hlint version
* Read stakeholders through the active contract service
Fixes#2070
Also standardizes the use of American English spelling for `serialize`
and `deserialize`. Standardize. Serialize. Americanize.
* Remove Haskell bindings test workaround
* Address https://github.com/digital-asset/daml/pull/2316#discussion_r308193661
* Cleanup
* WIP
* first integration test + fixture
* minor cleanup
* Implementing ContractService.lookup
* Reverting back to endpoints.all (all2 did not work)
* Cleanup
* replace ApiValue ADT with aliases to daml-lf/transaction Value ADT
* porting rest of navigator to LF Value ADT
* Command Service WIP
* CommandService WIP
* porting more of navigator to LF Value ADT
* last error, not first
* rename ApiValueImplicits file
* special conversion features for ImmArray and FrontStack
- just .to[ImmArray] or .to[FrontStack] any random collection
* finish porting most of navigator main code
* use numeric indices for record field name fallback when pretty-printing
* tuples are not serializable
* use numeric indices for label fallback in JSON verbose encoding
* make traverseEitherStrictly more likely to preserve the seq's class
* to shortcut for ImmArraySeq .to[ImmArraySeq]
* compiling, passing navigator backend tests
* test traverseEitherStrictly more, er, strictly
* pass scalacopts through to scaladoc
* deal with unused warning
* remove unneeded function
* simpler error reporting, more private functions in ApiCodecCompressed
* move slowApply to FrontStack, test it so it actually works
* remove unneeded toStrings; better error from impossible ValueTuple case
* scalafmt FrontStackSpec
* support alternative, label-free record JSON encoding
* Adding domain.CreateCommand + corresponding json formats and dummy json format for lav1.value.Record
* CommandService.create should be done... need to test it
* TODO added
* Cleanup
* move ApiCodecCompressed, ApiValueImplicits, and some aliases to new lf-value-json package
* Using tagged TemplateId type instead of Identifier + exercise command WIP
* adapt navigator to moved pieces
* start defining scalacheck extension to ApiCodecCompressedSpec
* CommandService.exercise + introducing CommandMeta
* Adding command endpoints, can't test them yet, need lf value json formats
* fuse some list operations
- suggested by @stefanobaghino-da; thanks
* blue error message
* Minor fixes after merging librify-navigator-json-compressed, #2136
* experiment with an inductive case in TypedValueGenerators
* finish a List case for TypedValueGenerators; it's revealing
* Introducing API value to LF value converter,
CommandsValidator takes IdentifierResolverLike instead of IdentifierResolver
* cleanup
* remove accidentally readded duplicate aliases
* start tying knots in TypedValueGenerators
* verbatim copy ApiCodecCompressedSpec to lf-value-json
* shift some tests from navigator to lf-value-json
* test Optional and Map for ApiCodecCompressed
* heavier random testing of ApiCodecCompressed
* remove unused dependencies from lf-value-json
* adding value json writer
* cleanup
* Revert "cleanup"
This reverts commit 2e4d153f
* fixing the build
* cleanup
* cleaning up imports
* JsValue to API value is done, needs a test
* cleanup
* use scalac -Ypartial-unification in http-json
* simplify some Traverse instances
* factor CreateCommand and ExerciseCommand traverse instances
* Command create integration test WIP
* Command create integration test WIP, got rid of the JsonReader and JsonWriter for the values, converting values explicitly
* Extracting DomainJsonDecoder and DomainJsonEncoder
* LfV refactoring
* Create command serialize/deserialize test works
* cleanup
* resolving conflicts
* More json encode/decode tests
* logging
* command/create passes integration test now
* Adding readme
* grammar
* TODO added
* GetActiveContractsResponse encoding
* ideintifier conversion renaming
* PackageService resolveTemplateId returns domain.TemplateId now
* Resolving LF Identifier instead of Template ID, this should also work for Exercise command decoding
* cleaning up a bit
* daml-lf: show type in TypedValueGenerators-driven errors
* exercise command json encoding/decoding works
* command/exercise IOU_Transfer integration test passes now
* avoid filter for Gens; makes many contract ID gens not fail
* test ApiCodecCompressed against 100 random types, 20 random values each
* Updating README instructions
* improving error handling, failed futures, get logged and reported to the user now as 500
* [ROUTING DSL] Removing routing DSL, it did not work
* getting rid of HttpEntity.Strict match + cleanup
* fixing the merge conflict
* updating README
* use Show.shows instead of new Show
* List(_) isn't checked, but Seq(_) is slightly safer
* improving test assertions
* Adding /contracts/lookup implementation
* http-json: use ImmArraySeq instead of List; use toRightDisjuction
* http-json: .toList.toSet is shorter than fold
* http-json: replace .leftMap.map with .bimap
* http-json: use subst instead of reimplementing JsonFormat
* http-json: remove unused ExceptionHandler
* http-json: safer == comparison
* Adding two test cases for expected errors
* Adding BazelRunfiles.rlocation magic that supposed to handle windows path for bazel dependencies
* http-json: import, not extend
* daml-lf: prepare archive proto for Numeric
* replace DECIMAL by NUMERIC in messages and fields
* add nat kind and nat type
* add builtins CAST_NUMERIC and SHIFT_NUMERIC
* daml-lf: remove new builtins from archive proto
* daml-lf: some more doc about numeric
* daml-lf: fix spec
* Support for contract keys to kvutils
* Fix handling of CreateEvents with contract keys in semantic tester
* Add ContractKeysIT to tests and fix issues related to fetchByKey
* Address code review
* ledger: refactor validateValue from CommandsValidator to new ValueValidator
* daml-lf: add a matchable class for leaf Values except ValueContractId
* ledger: use IdentifierResolverLike for CommandsValidator
* daml-lf: add Traverse[FrontStack]
* navigator: define readRecordArgument and readArgument in terms of ledger-api-common ValueValidator
* navigator: clean up fillInVariantTI
* navigator: replace ledger-api Value writer with LfEngineToApi from ledger-api-common
* navigator: use traverse
* navigator: remove dead sequenceMap
* navigator: adapt to new ValueValidator structure
* ledger: dealias validateValue call
* navigator: a good use case for traverseEitherStrictly
* move ApiCodecCompressed, ApiValueImplicits, and some aliases to new lf-value-json package
* adapt navigator to moved pieces
* start defining scalacheck extension to ApiCodecCompressedSpec
* experiment with an inductive case in TypedValueGenerators
* finish a List case for TypedValueGenerators; it's revealing
* remove accidentally readded duplicate aliases
* start tying knots in TypedValueGenerators
* verbatim copy ApiCodecCompressedSpec to lf-value-json
* shift some tests from navigator to lf-value-json
* test Optional and Map for ApiCodecCompressed
* heavier random testing of ApiCodecCompressed
* remove unused dependencies from lf-value-json
* replace ApiValue ADT with aliases to daml-lf/transaction Value ADT
* porting rest of navigator to LF Value ADT
* porting more of navigator to LF Value ADT
* last error, not first
* rename ApiValueImplicits file
* special conversion features for ImmArray and FrontStack
- just .to[ImmArray] or .to[FrontStack] any random collection
* finish porting most of navigator main code
* use numeric indices for record field name fallback when pretty-printing
* tuples are not serializable
* use numeric indices for label fallback in JSON verbose encoding
* make traverseEitherStrictly more likely to preserve the seq's class
* to shortcut for ImmArraySeq .to[ImmArraySeq]
* compiling, passing navigator backend tests
* test traverseEitherStrictly more, er, strictly
* pass scalacopts through to scaladoc
* deal with unused warning
* remove unneeded function
* simpler error reporting, more private functions in ApiCodecCompressed
* move slowApply to FrontStack, test it so it actually works
* remove unneeded toStrings; better error from impossible ValueTuple case
* scalafmt FrontStackSpec
* support alternative, label-free record JSON encoding
* fuse some list operations
- suggested by @stefanobaghino-da; thanks
* blue error message
* loading Dars from ZipInputStreams
formatting and cleanup
fixing DarReaderTest not compiling
fmt
using IOUtils to read the ZipInputStream
fixing CodeGen
fixing broken legacy DAR loading
fixed v2 ReferenceServer
cleanup
* fixing rebase conflicts
* introduced a fromFile method to avoid having to do boiler-plating where the input is a File
* addressing leaking resources
* cleanup
* inlining closures
* closing the darStream
daml-lf-archive-java is put together from the proto sources with the
compiled java classes. However, the logic that picks the java classes
was dependent on the output of $(locations :daml_lf_proto_lib_java),
because it always would pick the first item.
The new logic filters by name rather than position.
Fixes#2058.
* check that submitter is in maintainers when looking up keys
Fixes#1866. Note that this limitation applies both for `lookupByKey`
and `fetchByKey` -- anything involving retrieving a key is affected.
* add UNTIL-LF to run tests up to a certain version of DAML-LF
* name targets for DAML tests better
* add notes about DAML-LF changes
* commit Test.daml with DAML-LF 1.5 rather than compiling it on the fly
* add scenario tests for #1866
* add warnings about future key behavior in docs
* use flag rather than version when executing
* Allow shadowing of type variables in DAML and DAML-LF
We relax the DAML-LF type checker to allow for shadowing of type variables.
This does not need big changes since the substitution we use already avoids
name capture. The test for alpha equivalence converts to de Bruijn indices
on the fly and is hence not a problem either.
We inline type synonyms during the conversion from GHC Core to DAML-LF. This
requires alpha renaming. The cheapest way to get this, is to use the unique
names of type variables instead of their surface names.
This fixes#1915.
* Fix Scala test
* Relax memory constraints for bond-trading test
* lf-encoder: minor fix for let expression
* extractor: add test for enum types
* navigator: a small test for enum type
* daml-lf: create value version 5 for enums
* formatting
* daml-lf: fix version timeline
* extractor: fix enum support
* Update rules_haskell and static GHC
Remove patches that have been upstreamed or are no longer required.
Update still required patches to match the new rules_haskell version.
Previously we patched rules_haskell to coerce GHC into using static
Haskell libraries in most places. In particular we moved hs-libraries
entries into extra-libraries entries in the package configuration files.
A much cleaner approach is to compile GHC with a static RTS, then GHC
will by itself choose to load static Haskell libraries.
* Remove haskell_cc_import
* da-hs-daml-cli -> daml-cli
* da-hs-damlc-app -> damlc-app
* Introduce contract key to exercise node
Track the contract key in NodeExercises. This is in
preparation for adding the key into exercise events, and
for support for contract keys in kvutils, where we need
to be able to compute the outputs of the submission, e.g.
the contract key that would be unset by a consuming exercise.
This introduces transaction version 8.
* Move release note on contract keys in exercises to unreleased.rst
* Address Stephen's review
- Drop changelog entry for this change as it is not user facing
- Add info on version 8 to transaction.rst. Fix typo in version 7.
- Reorder VersionTimeline as per review
- Only compare the NodeExercise 'key' if it is set in the original
transaction (in order to compare transactions in a backwards
compatible manner)
* Unbork unreleased.rst
* specify the new fields for interning package IDs
* percolating intern tables
* crawl packages for package IDs as a pre-phase: generic prep
* stub case for interned_id in Scala packageref reader
* HasPackageRefs instances for the rest of the ast
* make intern table and use when encoding PackageRefs in v1
* don't need where
* stub out decode for interned package IDs
* no benefit to using uint32 instead of uint64
* percolate in encode one step
* interned case for decoding PackageRefs
* naming details
* intern table decoder
* finish propagating the intern table in encoder
* encode the package ID table
* document the vital assumption of encodeInternedPackageIds
* propagate the intern table through the LF decoder
- done by stacking ReaderT on top of Decode internally,
as discussed with @hurryabit
* daml-lf-proto requires mtl
* stub out interned case in Scala LF decoder
* stub interface decoder function
* get the interned table to most places in InterfaceReader
* support for interned package IDs in Scala decoder
* use ImmArraySeq instead of Vector for Scala intern decode table
* adding that ghc extension didn't make sense
* implement interned ID decoding for InterfaceReader
* scenario service won't have interned package IDs
* test the interned ID resolution in Scala by examining the proto -> AST in detail
* proper precondition for the dev phase of interned IDs testing
* better error reporting for malformed DALFs in intern test
* just import Data.Int
- suggested by @neil-da; thanks
* pass around the lookup function instead of the vector in decoder
- suggested by @neil-da; thanks
* remove derivations for types deleted in e63b012d2d
* rename VersionAware to EncodeCtx
- suggested by @hurryabit; thanks
* rename MDecode to MonadDecode
- suggested by @hurryabit; thanks
* pass a function through the encoder instead of a set
- based on suggestions by @hurryabit and @neil-da; thanks
* daml-ghc test that interned IDs are generated
- suggested by @hurryabit; thanks
* adapt to 5b480c99ec#1844
* add scenario Module decoding to Decode.OfPackage
* use purely data-driven decoding in scenario service in Scala
- decouples scenario service from LF decoder implementation
* make DecodeV1 companion private
* make extension to LFv2 more obvious
I find the current names very confusing and unintuitive.
I'm aware that there are other primitives called `TO_TEXT_X`. But there the
`X` is meant to be a type index to `TO_TEXT`. This doesn't make much sense in
the context of `TO_TEXT_CODE_POINTS` though since there's no type named
`CodePoints`.
* store archive size in `DarReader`
* rename `SandboxTemplateStore` to `SandboxPackageStore`
* store package info in `SandboxPackageStore`
* introduce package upload / read to write / index services
not tested yet, just a ton of plumbing
* WIP test the package service
* Fix build errors after rebase
* Move packages service to v2
* Ledger API client uses ledger API types
* Fix ReflectionIT
* Correctly handle uploading invalid dar files
* Fix reading DAR entry file sizes
* Improve package management IT
* Improve handling of duplicate packages
* Fix language-support build
* Use unique party and command names
* Rename lfpackage to language
* Rename SandboxPackageStore to InMemoryPackageStore
* Remove getCurrentTime ledger method
* Improve package management IT
* Move InMemoryActiveContracts and InMemoryPackageStore
* Use case object for UploadDarResult.Ok
* Address review comments
* Update release notes
Fixes#1311
* Use BazelRunfiles in test
* encode ast in protobuf
* daml-lf parser: make defaultPackageId and languageVersion parametric
* daml-lf rewritting of AST
* test ast encoder
* copyright
* test function type encoding
* daml-lf add parameter to parser implicits
* damlfl-as stands for "damllf assembler"
* move encoder in its own private package
* Allow slashes in ledger strings
Using '/' as the divider is a natural choice for some ledger backend
implementations and in application/workflow etc. identifiers.
* Update value.proto to include slash in LedgerStrings
* Update release notes
ImmArray, FrontStack, and BackStack violated the contract between equals
and hashCode. ImmArray had a wrong implementation of hashCode, and
FrontStack and BackStack had no implementation at all.
Fixes#1623.
* working generated representation type
* from/to method generation compiles
* moved GHC.Generics to DA.Generics
* removed strictness metainfo
* added test
* some cleanup
* deal with empty record constructors
* more fine grained testing
* Drop support for DAML-LF 1.3 from damlc
This means `damlc` now only supports versions allowing arbitrary expressions
as contract keys. This is very helpful for the upcoming new template
de/re-sugaring underpinning generic templates.
* Unpin a few DAML-LF target versions in tests
* Fix golden tests
* Adapt changelog to rebase
* Rename DAML-LF scenario tests to 1.dev
* Update tests to --target 1.dev
* Fix release notes
* Back parseInt/parseDecimal by DAML-LF primitives when available
DAML-LF 1.5 introduces two new primitives `FROM_TEXT_INT64` and
`FROM_TEXT_DECIMAL`. We translate `parseInt` and `parseDecimal` to
these primitives when compiling to DAML-LF 1.5 or later.
This fixes#1398.
* Fix DAML-LF decoder (Scala)
* Freeze DAML-LF 1.dev into DAML-LF 1.5
In other words, we release DAML-LF 1.5.
This is required for generic templates (#1387).
* description of FROM_TEXT_INT64 & FROM_TEXT_DECIMAL
* amend version history
add ``FROM_TEXT_INT64`` and ``FROM_TEXT_DECIMAL`` in the specification changelog
* typos
* Fix markup in DAML-LF spec
* Add release notes
* Drop support for DAML-LF 1.1 from damlc
Part of #853.
* Fix docs test
* Add show party test back
* Update release notes
* Remove target version from Java codegen test
* Isolate daml-on-x prototype
* Implement TransactionService of index api.
Towards the first milestone of the daml-on-x integration steps
* fix compiler error after last-minute refactoring
* formatting
* remove unnecessary dependency from api-server-damlonx
* Fix some inconsistencies.
* ghc compiles generics
* more general kind conversion
* rename varKind to tyVarKind for clarity
* added a comment regarding a check for origin of Meta constructor in conversion
* added reference to original Generics.hs and git commit hash
* added copyright header
* fixing doc generation
* update dar reader test
* no PolyKinds in generics
* functional dependencies and small fixes
* convert MetaData, MetaSel, MetaCons types
* Make the actors optional in DAML-LF's exercise instruction
If they are not present, the controllers will be filled in. Surface DAML
does this currenty anyway by fetching the contract and computing the
choice controllers before each `exercise`. This change will allow for
getting rid of the additional `fetch` preceding each `exercise`.
The compiler does not use the new form yet. I will do this in a separate
PR together with tests for the new behaviour.
This fixes#1347.
* Fix DAML-LF type checker test
* Check presence of actors for old DAML-LF versions in decoder
This is a preparation for making the actors in DAML-LF's exercise
instruction optional (see #1347).
This is technically a breaking change but since `exerciseExplicit` has never
been exposed via the documentation, we don't add a release note.
* Rename Value's ContractId to VContractId
* daml-lf: a bit more about PartyId
* daml-lf: Concatenable MatchingStringModule
* daml-lf make clear type used for Scenarios only
* daml-lf create ContractId, LedgerId, TransactionId
* sandbox-sql conversion util
* LedgerName -> LedgerString
* futher type cleanup in the sandbox
* daml-lf add test for LedgerString
* fixing tests
* a bit more safety in the DB
* Address Stephen's comments
* fix rebase
* More fixes for StringModule
* change length of LedgerString (256 -> 255)
* Lift restriction on serializable contract ids in DAML-LF 1.dev
In DAML-LF 1.dev, make `ContractId a` serializable whenever `a` is
serializable. This is part 2 of #1277.
* Reformat Scala
* Add changelog entry to daml_lf_1.proto
* Fix package valdiation for DAML-LF 1.4
Currently, complex contract keys are are not accepted for DAML-LF 1.4
although they should because we forgot to roll the version when freezing
DAML-LF 1.4.
@bitonic we need a way forward to avoid this in the future.
* Fix test
* Format Scala
* Add more tests for (un)serializable contract ids
I want to lift this restriction in a subsequent PR for the next version
of DAML-LF. Let's first make sure we have a correct implementation for the
current version though.
* fix test
* in serializability tests, check the modules are properly typed
The foldWithPathState is a fold over the transaction that allows
keeping extra state along the path of the traversal.
We add computePerPartyProjectionRoots in preparation for the
work on privacy-aware kvutils, where transaction nodes are stored
separately rather than in one blob. This function allows us to compute
the projected transaction roots for each party mentioned in the
transaction. We require this information to establish the order
of the root nodes when the size and form of the transaction is
blinded by assigning random node ids.
* Warn if there are any missing methods in type classes in DAML
Helps with #1194
* Make missing methods a fatal warning
* Update release-notes.rst
* Fix a test that had no signatories
* Update docs/source/support/release-notes.rst
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Relax syntactic restriction on contract keys in DAML-LF 1.dev
We lift the syntactic restriction that contact keys must be built using
only record constructions and projections entirely when compiling to
DAML-LF 1.dev. To make this more useful, we also search all sub-expressions
of `maintainer` in `key` during our rewriting of `maintainer` for using
`this` to using `key`.
As one of our next steps we should bring `key` into scope in `maintainer`
and perhaps deprecate the use of `this` at some point in the future.
* Fix versioning
* Adapt package validation to complext contract keys
* Added agreement_text field to the CreatedEvent in Ledger API.
* Changed java bindings + java codegen
* Changed utilities for scala codegen
* Made necessary changes in Sandbox to propagate the agreement text from ContractInst to the CreatedEvent
* Made changes to the navigator to show the agreement text in the contract details page when it is set and not empty
Fixes#1110
* Store divulgence info for contracts
* Store divulgence info for contracts
* some documenting refactors
* Use fail instead of sys.error in test
* Add release note
* Rename migration files
* Fix typo
* Fix updating divulgence info in scenarios
The ACS was comparing differently formatted contract IDs.
Fixes#1166
* stub away the components of LedgerValue
* rewrite v1 api converters to produce transaction Value ADT
* a needless 'case'
* shortcut methods for ImmArraySeq and tests
* porting extractor's JSON encoders to transaction Value
- JSON object ordering now matches record field order, instead of being
its reverse
- Includes a new encoding for records missing labels, to a list of
2-tuples
* porting extractor main transaction Value
- JSON date/time is just <num>s since epoch, maybe revisit
* adapt to Decimal newtype
* use daml-lf/data stringification for decimals in JSON
* snap aliases for Cid-less LedgerValue subtypes
* snap aliases for Cid-containing LedgerValue subtypes
* remove needless indirection
* test new missing-label Record JSON encoding
* write ImmArray traverse with syntax extension
* remove RecordField; move Name parsing to former RecordField converter
