Commit Graph

155 Commits

Author SHA1 Message Date
Stephen Compall
4cd419623a
replace traverseU and sequenceU with traverse and sequence (#6594)
* replace traverseU and sequenceU with traverse and sequence

- with -Ypartial-unification on, the extra Unapply typeclass lookup is
  unnecessary

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* limit imports; we only need *> and void
2020-07-07 15:01:58 +00:00
Stephen Compall
15350a7bc2
disable warts.Any and remove most suppressions (#6132)
* disable Any wart

* first pass removal of Any suppressions for false positives

* second pass removal of Any suppressions for false positives

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* third pass removal of Any suppressions for false positives

* fourth pass removal of Any suppressions for false positives

* reformat newly single-suppressions into single lines

- suggested by @SamirTalwar-DA; thanks
2020-05-28 16:53:41 +00:00
Stephen Compall
78205afa0d
use 2.13-style collections library syntax in json-api (#6075)
* register scala-collection-compat with java deps list

* add scala-collection-compat to http-json deps

* remove breakOut throughout http-json, replaced with view/to or iterator/to

* use scala 2.13-style `to` calls in http-json

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* use 2.13-style to in lf-value-json

* some fused size comparisons

* remove low-hanging breakOuts in daml-lf

* regenerate maven_install.json for scala-collection-compat

* regenerate maven_install.json for scala-collection-compat

* regenerate maven_install.json for scala-collection-compat
2020-05-27 22:27:25 +00:00
Stephen Compall
1be6b3640f
equalz Scalatest matcher in new daml-lf/scalatest-tools library (#1200)
* equalz Scalatest matcher in new daml-lf/scalatest-tools library

* equalz typing tests

* a 'should' replacing design

* a 'MatcherFactory1' design

- this fails because the TC parameter should be a type member to avoid
  scala/bug#5075 but it is not

* MatcherFactory1 with chained Lub+Equal typeclass

- requires partial-unification at point of use, which is not great

* LubEqual's extra tparam is probably unneeded

* better LtEqual

* demonstrate that HK LubEqual's resolve with DMT should + MatcherFactory

* remove unneeded 3rd param from LubEqual, again

* update dependency specs and license headers

* allow use with should, shouldNot in some cases, preserving the shouldx/shouldNotx alternatives

* move Equalz to libs-scala/scalatest-utils

* rename bzl targets and place in com.daml.scalatest package

* add scalatest-utils to release

* move *SpecCheckLaws, Unnatural to scalatest-utils

* missed scalacheck dep in scalatest-utils

* downstreams of *SpecCheckLaws now get them from scalatest-utils

* test equal-types case as well

* update LF documentation

CHANGELOG_BEGIN
CHANGELOG_END

* whitespace error
2020-05-27 17:25:43 -04:00
Remy
93a01f7e12
DAML-LF: enforce size limit for Contract ID suffixes. (#5913)
Following the Contract ID Specification, we limit the size of contract ID suffixes to 94.

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-11 15:32:17 +02:00
Leonid Shlyapnikov
a79377cb2d
Improve DAML LF Party, PackageName, PackageId fromString error messages (#5855)
* Add description to daml.lf.data.StringModule implementations

Make sure DAML LF Party deserialization picks up the added description.

changelog_begin
changelog_end

* addressing code review comments. @S11001001, thanks.
2020-05-06 10:02:49 -04:00
Remy
aed97a5d1f
Engine: Contract IDs collecter for values, transactions… (#5805)
We add a method to collect contract ID for value, transaction ...

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-06 10:52:40 +02:00
Stephen Compall
2b23b41391
JSON decoding support for GenMap (#5223)
* add GenMap to the "all types" test generators

* report bad GenMap format with DeserializationError, not MatchError

* document GenMap JSON

* notes on missing features

* enable -Xsource:2.13 in transaction

* make an Order instance for Value resolvable, but unimplemented

* use the skeleton from SValue ordering to make a Value ordering skeleton

* add Party Order

* add Order instance for SortedLookupList

* add Order for FrontStack, deriving everything

* factor the Order lookup, and tie a knot in the recursive Value instances

* we're going to need this Iterator thing again

* replacing Order#contramap with version that supports equalIsNatural

* use new equalBy, orderBy for FrontStack, SortedLookupList, ImmArray

* _2 comparator, upgrade Name Equal to an Order

* incorporate lookup for enums, variants into Value order; record/struct cases

* Enum/Variant comparison

* looking up the singleton implicitly won't work for non-`object`s, alas

* test Order laws for values of all primitive types

* test Order laws for record and variant types

* test Order laws for enum types

* test that enum strings are not compared

* use checkLaws for Value Equal as well

* test that enums match order to constructor rank

* factor genAddend and genAddendNoListMap

* reintroduce Order for TypedValueGenerators

* more addend order

* record, variant order cases

* record cons order

* deriving Order while decoding from JSON

* make ApiCodecCompressed's Cid codec based on the typeclass

* test how the Value ordering and the underlying projected value orderings line up

- hint: they don't, yet
- this is also a template for how we'll check the fidelity with SValue
  ordering

* test how the Value ordering and SValue ordering line up

- hint: they don't, yet

* typed Arbitrarys have access to Order

* produce proper ValueGenMap

* inj requires Order, sometimes

- we encode this as "all the time" but there is a type-level unification
  approach to remove this requirement in some cases

* make inj a function

* test that order doesn't matter for JSON decoder

* use Utf8 order for TVG text; don't pretend that base equal works

* sort JSON GenMaps, and check for duplicates

* make injarb use IntroCtx

* remove stray import

* Order instances for Bytes, Hash, AbsoluteContractId

* require Order[Cid] to decode JSON to LF values

* clean up map reordering test

* remove unused Instant instance

* fake Order instance no longer needed, valid instance defined

* test parity of global AbsoluteContractId order and SContractId order

* bazel fmt

* test AbsoluteContractId Order lawfulness

* test duplicate key detection

CHANGELOG_BEGIN
- [JSON API] Prepare full support for the planned GenMap primitive type.
  See `issue #5031 <https://github.com/digital-asset/daml/issues/5031>`_.
CHANGELOG_END
2020-05-05 10:27:37 -04:00
Stephen Compall
d4086080ea
proper GenMap Gen with key Value order (#5526)
* add Order instances to various types in LF data, transaction

* define scope-dependent Order for Value; test compatibility with SValue order

* support Order in TypedValueGenerators

* generate properly-ordered ValueGenMaps in TypedValueGenerators

* factor genAddend and genAddendNoListMap

* test Value.orderInstance fidelity with well-typed Orders

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* shrink Maps for better debugging

* line up base List and GenMap orders with svalue

* unhandled cases in custom comparator for ApiValueToLfValueConverter tests

- hazards of not using match2

* adapt to PureCompiledPackages changes
2020-04-24 17:57:12 -04:00
Stefano Baghino
f81efb3d9d
Add new participant schema contracts table (#5541)
* Add new participant schema contracts table

- full implementation (both read and write path)
- fixes #5514

changelog_begin
[Ledger API Server] Fixed a bug that prevented validating transactions
containing reference to contracts that did not originate on the local
participant. See `issue #5514 <https://github.com/digital-asset/daml/issues/5514>`_.
changelog_end

* Format fixes and addressing comments

- address https://github.com/digital-asset/daml/pull/5541#discussion_r408806775
- address https://github.com/digital-asset/daml/pull/5541#discussion_r408806964
- address https://github.com/digital-asset/daml/pull/5541#discussion_r408812745

* Address https://github.com/digital-asset/daml/pull/5541#discussion_r408815185

* Address https://github.com/digital-asset/daml/pull/5541#discussion_r408813067

* Fix mistake in comment

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@daml.com>

* Fix compilation error

* Add data migration

Tested by https://github.com/stefanobaghino-da/test-5541-data-migration

* Address https://github.com/digital-asset/daml/pull/5541#discussion_r409386378

* Address https://github.com/digital-asset/daml/pull/5541#discussion_r409385174

* Address https://github.com/digital-asset/daml/pull/5541#discussion_r409367606

* Simplify migration

* Address review

- address https://github.com/digital-asset/daml/pull/5541#discussion_r410344423
- address https://github.com/digital-asset/daml/pull/5541#discussion_r410339914

* Update ledger/sandbox/src/main/resources/db/migration/postgres/V26_0__Contracts_new_schema.sql

Co-Authored-By: Gerolf Seitz <gerolf.seitz@daml.com>

* Fix migration for privacy-aware ledgers

* Remove leftover query

* Fix migration hashes

Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
2020-04-17 20:07:20 +00:00
Stefano Baghino
19b3d1a02b
Revert participant_contracts write path (#5535)
changelog_begin
changelog_end
2020-04-11 10:00:27 +02:00
Stefano Baghino
21037bfd39
participant_contracts write path (#5522)
* participant_contracts write path

Writes contracts alongside events for interpretation and validation.

changelog_begin
changelog_end

* Fix self-referencing foreign key

* Address https://github.com/digital-asset/daml/pull/5522#discussion_r406289145

* Compute divulgence only if necessary

* Do not attempt at persisting witnesses for transient contracts

* Drop new tables when resetting
2020-04-09 19:33:46 +00:00
Samir Talwar
bd8e3286c6
daml-lf: Tiny performance improvements. (#5493)
* daml-lf/data: Improve performance of `areEqual`.

`areEqual` constructs a one-element `FrontStack`, which should be cheap,
but turns out to be expensive because it constructs builders along the
way. This provides overloads for `apply` that construct the FrontStack
directly, rather than jumping through hoops.

This also changes the behavior of `FrontStack#equals` to check the
length first, which makes the tests work but might also help
performance.

* daml-lf/data: Merge `length` and `len`.

In `FrontStack`, `BackStack`, and `ImmArray`.

No reason to have `def length = len` when `len` can just be public.

* daml-lf/interpreter: Allocate the right-sized list for arguments.

We spend a fair amount of time in `ArrayList#add`; this lessens that by
making sure than when we clone the `args` array for function
application, we leave some room for adding the arguments.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-lf/data: Revert accidentally-lengthened imports.

Co-Authored-By: Stephen Compall <stephen.compall@daml.com>

* daml-lf/interpreter: Rename `functionArgs` to `extendedArgs`.

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-04-08 16:28:42 +00:00
Remy
af5b8e553f
Speedy: optimize value translation (#5379)
CHANGELOG_BEGIN
CHANGELOG_END
2020-04-07 17:17:13 +00:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Gerolf Seitz
329320bad9
Organize maven coordinates (#5272)
* Use com.daml as groupId for all artifacts

CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END

* Add 2 additional maven related checks to the release binary

1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId

* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
2020-04-01 11:41:18 +02:00
Remy
0e1ca7df7c
Engine: ensure freshness of local cid discriminator (#5277)
* Engine: ensure freshness of local cid discriminator

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-31 18:06:18 +02:00
Stefano Baghino
746b774e82
Add canonical string representation for identifiers (#5270)
* Add canonical string representation for identifiers

An identifier is represented as a string as a package identifier and a qualified name separated by a colon.

changelog_begin
changelog_end

* Simplify explicit type signature for HexString

Thanks @remyhaemmerle-da

Co-Authored-By: Remy <remy.haemmerle@daml.com>

* Fix hashing test expected output

* Fix key hasher test expected output

Co-authored-by: Remy <remy.haemmerle@daml.com>
2020-03-30 12:55:11 +00:00
Remy
1b37f6c482
DAML-LF: redesign absolute contract ids (#5207)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 19:07:42 +01:00
Remy
20218e4175
DAML-LF: restrict length of parties (#5237)
CHANGELOG_BEGIN
* [DAML-LF]. *Breaking* Restrict length of parties
CHANGELOG_END
2020-03-27 14:52:43 +01:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
Stephen Compall
8d2c13cf43
generating GenMaps for testing (#5115)
* building a GenMap generator

* move Party Order to accessible place; test whether Name Equal is still in use

* add Order instance for SortedLookupList

* switch to Map for genMap's Inj; more Order instances

* remove all Order from TypedValueGenerators

* Revert "add Order instance for SortedLookupList"

This reverts commit 03a59a8249.

* moving the Equal instance means scenario-interpreter no longer direct-deps scalaz

* add a test using TypedValueGenerators GenMap

* remove Party Order

* refmt bazel

* remove stray import

* followup dep change to moving the Equal[Name] instance

* add changelog

CHANGELOG_BEGIN
CHANGELOG_END

* Name equal instance appears to be no longer used, but keeping anyway
2020-03-25 10:28:06 -04:00
Remy
8fb86b7133
Engine: create type for immutable byte array (#5140)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 09:04:52 +01:00
Remy
c2070170b3
Engine: add rank to Enum and Variant SValue (#5048)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 14:37:44 +01:00
Remy
198973f23b
DAML-LF: introduce HexString to represents base16 array encoding (#4952)
* DAML-LF: introduce the string type HexString that represents base16 array encoding

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-12 22:04:36 +01:00
Stefano Baghino
1bf1a8e8e9
Add diff function between Relations (#4962)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-12 14:41:22 +01:00
Stefano Baghino
6f59832e45
Add lazy flattening function to Relation (#4922)
* Add lazy flattening function to Relation

CHANGELOG_BEGIN
CHANGELOG_END

* Restrict tests to non-empty relations and address https://github.com/digital-asset/daml/pull/4922#pullrequestreview-372050511
2020-03-10 17:39:45 +00:00
Moritz Kiefer
c68dd19ade
Decode and validate package metadata in Scala (#4653)
This adds the code for decoding and validating package metadata
that is now emitted by damlc.

changelog_begin
changelog_end
2020-02-21 18:20:42 +00:00
Remy
8de42ce062
[Sanbox] use new ContractId scheme (#4533)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-17 13:30:47 +00:00
Remy
8394f4ba7d
[DAML-LF] Version new contractId in Proto values (#4460)
* version new contractId

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-13 20:45:46 +01:00
Remy
4137ff510b
[Engine] Adding a specific type for ParticipantId (#4504)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-13 17:15:34 +01:00
Remy
e02e5d61bc
Engine: fork method to resolve contractIds (#4411)
* engine: fork method to resolve relative contractId

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-10 16:52:06 +01:00
Samir Talwar
784a90b1a0
Fix the flaky ledger-on-sql test on Windows. (#4402)
* kvutils: Use durations in participant state tests.

Makes it way easier to read than `addMicros(…)` calls.

* daml-lf/data: Fix warnings in Time.scala.

* daml-lf/data: Add `Timestamp#add`.

* kvutils: An MRT of (start time + 1 second) is too short for CI.

Increasing the MRT to (now + 10 seconds).

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-05 11:32:13 +00:00
Remy
93a613ff42
daml-lf: cleanup StringIds (#4221)
clean up String Identifier in daml-lf 

- Separate LedgerString from ContractIdString
- Drop TransactionIdString from daml-lf 
- Create a new ContractIdString (aka. ContractIdStringV1) for new contractId comparable with  relative contractIds

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-04 09:30:53 +01:00
Remy
2f3ff2f0d3
daml-lf: drop TransactionIdString (#4343)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-03 14:52:33 +00:00
nickchapman-da
62d592ec62
support type synonyms in scala (#4101)
* Support DAML-LF type synonyms in scala.

CHANGELOG_BEGIN
CHANGELOG_END

* dont create synonymns in GenerateSimpleDalf

* extend DAML-LF parser to support type synonyms

* test: expand type synonyms correctly
2020-01-23 10:49:28 +00:00
Remy
b6e848b576
Engine: Add node and contract Discriminator
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-22 18:49:30 +01:00
Remy
21a17fefaa
daml-lf: new cryptographic hash for values (#3849)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-16 22:11:57 +01:00
Nikolai Cherkezishvili
49b90bfb36
Engine: Improve performance of critical parts (#3947) 2020-01-09 19:18:55 +11:00
Andreas Herrmann
f33e79c787
Remove unused dependencies to da_scala_library (#3938)
* Inline all scala_library dependencies

* Run //:buildifier-fix

* TMP scala_library_suite --> scala_library

* da_scala_library: Enable unused dependency checker

* scala_library: Enable unused dependency checker

* //daml-lf/data:data

* //daml-lf/engine:engine

* //ledger-api/rs-grpc-akka:rs-grpc-akka

* //ledger/participant-state:participant-state

* //ledger/ledger-api-client:ledger-api-client

* //scala-protoc-plugins/scala-logging:scala-logging-lib

* //ledger/ledger-api-scala-logging:ledger-api-scala-logging

* //ledger/ledger-api-common:ledger-api-common

* //ledger-service/utils:utils

* //ledger-service/jwt:jwt

* //ledger/ledger-api-auth:ledger-api-auth

* //extractor:extractor

* //daml-assistant/scala-daml-project-config:scala-daml-project-config

* //language-support/codegen-common:codegen-common

* //language-support/scala/codegen:codegen

* //language-support/codegen-main:codegen-main-lib

* //ledger-service/db-backend:db-backend

* //ledger-service/http-json:http-json

* //daml-lf/scenario-interpreter:scenario-interpreter

* //ledger/sandbox:sandbox

* //navigator/backend:navigator-library

* //daml-assistant/daml-sdk:sdk-lib

* //daml-lf/data-scalacheck:data-scalacheck

* //daml-script/test:test-lib

* //ledger/ledger-api-common:ledger-api-common-scala-tests-lib

* //ledger/test-common:test-common

* //ledger/sandbox:sandbox-scala-tests-lib

* //extractor:extractor-scala-tests-lib

* //language-support/java/bindings:bindings-java-tests-lib

* //language-support/java/bindings-rxjava:bindings-java-tests-lib

* //language-support/scala/bindings-akka-testing:bindings-akka-testing

* //language-support/scala/codegen-testing:codegen-testing

* //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app

* //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app-testing

* //language-support/scala/codegen-testing:codegen-testing-testing

* //ledger-api/sample-service:sample-service

* //ledger-api/rs-grpc-akka:rs-grpc-akka-tests-lib

* //ledger/ledger-api-test-tool:ledger-api-test-tool-lib

* //ledger/ledger-api-test-tool:ledger-api-test-tool-tests

* //ledger/participant-state/kvutils:kvutils

* //ledger/sandbox:ledger-api-server

* //ledger/sandbox-perf:sandbox-perf-lib

* //navigator/backend:navigator-tests-library

* UNDO scala_library_suite --> scala_library

This reverts commit ab3eb1ae23139e2ec431ab4551fbb0371e0354e1.

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-06 18:14:21 +01:00
Gary Verhaegen
878429e3bf
update copyright notices to 2020 (#3939)
copyright update 2020

* update template
* run script: `dade-copyright-headers update .`
* update script
* manual adjustments
* exclude frozen proto files from further header checks (by adding NO_AUTO_COPYRIGHT files)
2020-01-02 21:21:13 +01:00
Leonid Shlyapnikov
0b172da49a Add ExceptionOps to extract exception details safely (#3822)
* Add ExceptionOps to extract exception details safely

`Throwable.getMessage` can return `null` which caused unexpected NPE
in JSON API error handling

* Fix formatting

* Remove show instance

* Addressing code review comments,

thanks Stefano!
2019-12-11 18:38:44 +00:00
Remy
94aea562d6 scala codegen: add GenMap support (#3522)
* scala codegen: add GenMap support

* scala-codegen: use InsertOrdMap as underlying of Generic Map binding

* Address Stephen's comments

* make TextMap a strict subtype of immutable.Map to avoid overlap with GenMap

Otherwise, attempted GenMap encodings like
`Value.encode(InsertOrdMap("foo" -> "bar"))` (and any contract
containing such a structure) are either ambiguous or, worse, yield the
wrong result.

* ensure better return types for InsertOrdMap and similar wrappers' operations

* if is an expression

* adapt various primitive encodings to newtype TextMap

* easy cleanups of the TextMapApi

* proper alias for deprecated Map

* update deprecation releases

* Revert "proper alias for deprecated Map"

This reverts commit e85aa85b960c4bf5c4f9624896183ec6e2182bba.

* remove useless invisible deprecated notice

* add generic map tests for scala codegen

* please restart CI
2019-12-11 09:22:38 +00:00
Jussi Mäki
01be23fa82
Implement GenericSetTemplate and ImmutableSetFactory for InsertOrdSet (#3557)
This ensures we retain InsertOrdSet when transforming it.
2019-11-21 16:08:08 +01:00
Jussi Mäki
ff03ed5f3f
Add party to participant mappings as submission inputs (#3456)
* Add InsertOrdSet

A set with deterministic ordering.

* Add party to participant mappings as inputs to kvutils submissions

* Add InsertOrdSetTest, remove fromSet. Sort packages and parties.

* Apply suggestions from code review

Co-Authored-By: Remy <remy.haemmerle@daml.com>

* Add AbstractSet and SetLike traits to InsertOrdSet. Remove (Non)EmptyInsertOrdSet.
2019-11-19 18:03:58 +01:00
Martin Huschenbett
3f2f69224c Fix comment about Party type in DAML-LF Engine (#3496) 2019-11-17 12:20:42 +00:00
Remy
4d36c01682 Daml lf genmap (#3453)
* shortcup for SBMapEmpty

* daml-lf: add GenMap in the interpreter.
2019-11-15 10:36:41 +00:00
Remy
79383bd090 fix InsertOrdMap (#3474)
* fix InsertOrdMap

* Address Samir first comment

* Address Samir second comment

* more tests
2019-11-14 18:06:47 +00:00
Remy
b7e2c17863
cleanup InsertOrdMap (#3466)
* cleanup InsertOrdMap
2019-11-14 16:27:21 +01:00
Stephen Compall
c36696f3a7 json-api: in-memory comparison query (#3405)
* comparison query parser given scalar parser

- written in half-error-propagation style to better suit other potential
  error features

* factor dupes in RangeExpr

* text range parsing

* interpreting Range into in-memory predicate; points out bad dedupe from earlier

* make reuse of the scalar extractors much nicer

* express date, time, int64 as factored-out range exprs

* express numeric as factored-out range expr

* factor \&/ usage

* refactor LF value extractors for reuse in range queries

* factor mkRange usage

* totally deconstruct the int64 and text cases

* totally deconstruct the date and timestamp cases

* totally deconstruct the numeric case

* document comparison queries

* use Utf8.Ordering for text comparison queries

* int64 range query tests

* more int64 range query tests

* date, string, numeric range query tests

* include line # in query test successes table

* timestamp range query tests

* add release note

* remove duplicate changelog entry from #3425
2019-11-12 23:02:33 +00:00
Remy
7c427119e1 DAML-LF add Type Representation value (#3326)
* daml-lf: update spec with type-rep

* daml-lf: update proto with type-rep

* daml-lf: update scala side with TypeRep

* daml-lf: update compiler side with TypeRep

* Get triggers to compile

* Add featureTypeRep to allFeatures

* Apply suggestions from code review

Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* daml-lf: add builtin for TypeRep equality

* Address Andrea's comments

* formatting

* Fix triggers

* Fix template typerep tests
2019-11-04 17:00:55 +00:00
Andreas Herrmann
2bd1db490a
Replace bazel-deps by rules_jvm_external (#3253)
* Update bazel-common to fix javadoc issues

Specifically, to fix the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
        File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
                javadoc_library(name = 'rs-grpc-bridge_javadoc')
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
                dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```

* Define Maven deps using rules_jvm_external

* Pin artifacts

* Remove bazel-deps generated targets

* Remove bazel-deps

* Switch to rules_jvm_external targets

* update bazel documentation

* pom_file: There are no more bazel-deps targets

* BAZEL-JVM.md `maven_install` typo
2019-10-28 13:53:14 +01:00
Andreas Herrmann
dc2f10ebe6 Use TemplateTypeRep in DAML Trigger API (#3245)
* Add TemplateTypeRep to AnyContractId

* Define Trigger.ContractId t

* Use Trigger.ContractId t

* Implement fromCreated and fromArchived

* instance MapKey TemplateTypeRep

* More efficient ACS access using Map TemplateTypeRep

* ./fmt.sh

* toString and fromString for Identifier

* Replace Identifier by TemplateTypeRep

* TheContractId --> AbsoluteContractId

https://github.com/digital-asset/daml/pull/3245#discussion_r338033546
2019-10-23 13:56:59 +00:00
Remy
f3b98ca31d DAML LF: limit allocations for trivial constants (#3212)
* speedy: limit allocation of trivial SValue constants

* daml-lf: limit allocation of constants for Value, Exp, Type

* weaken numeric equality in ledger service test
2019-10-18 15:00:49 +00:00
Remy
fbad37d560 Engine: update daml-lf value proto (#3001)
* engine: create persisted value for numeric

* engine: update info test

* Apply suggestions from code review

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* engine: weaken scale constraints when transalting value

* Address Stephen's comments

* Address one of Stefano's comments

* daml-lf: fix transaction test
2019-10-07 19:27:23 +00:00
Stephen Compall
22a5b17ea0 json-api: "table scan"-style query, literal equality and record subsets (#2826)
* ADT for json-api's LF value query predicates

* expand predicates into primitive cases

* expand more cases for the predicate reader

* instantiating type references

* record case

* get super-reductionist with ValuePredicate so we can tease out the proper model by operations

* just put the literal predicates in the ADT

* more primitive predicates

* properly build RecordSubset predicates

* tests for ValuePredicate LF value predicate functions

- false cases fail because outer layer still stubbed

* outer RecordSubset always used

* tests for comparing literal predicate and ApiCodecCompressed parsing

* map querying

* Adding ListMatch

* removing List test cases covered by LF-JSON encoding tests

* Optional support; precompile ListMatch elements

* use go instead of toFunPredicate

- deals with potential subst introduction as we make VP more abstract

* test cases for optional queries

* Adding VariantMatch

* Fixing comments

* some precompiling and fusion

* Adding value predicate to contracts/search endpoint, WIP

* Adding value predicate to contracts/search endpoint, WIP

* add filtering to ContractsService#filterSearch

* Adding contracts/search result filtering based on the constructed predicates

* cleanup

* make error throwing in ValuePredicate a little nicer

* scalafmt

* cleanup

* follow JSON format rules for numeric queries

* test that apiValueToLfValue . lfValueToApiValue = arr id

* API value -> LF value discards scale; account for that in ValuePredicate

- fixes queries that match numerics

* document query

* release note

* test for contracts/search with query

* more tests for contracts/search with query

* More tests and cleanup

* more cleanup

* better test

* Adding disclaimer

* records, not documents

* better error message in the fromCon case
2019-09-23 16:08:24 -04:00
Remy
d540f7aab0 Daml-LF: change Numeric max scale from 38 to 37 (#2969)
* daml-lf: numeric maxScale change from 38 to 37

* daml-lf: update spec

* udpate protobuf comments
2019-09-23 12:39:28 +00:00
Remy
a46f1c041a Daml-LF: make Numeric scale type safe (#2958)
* daml-lf: make numeric scale type safe

* navigator: fix scale

* ledger: fix scale in tests

* Update daml-lf/data/src/main/scala/com/digitalasset/daml/lf/data/NumericModule.scala

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Update ApiCodecCompressedSpec.scala

* Update ApiCodecCompressedSpec.scala
2019-09-18 19:33:52 +00:00
Remy
220a03c9e8 Daml-LF: make MUL_NUMERIC and DIV_NUMERIC multi-scale (#2921)
* daml-lf: Make MUL_NUMERIC and DIV_NUMERIC multi-scale

* update release notes

* compiler: fix with type change
2019-09-17 14:32:49 +00:00
Remy
3ee9b83fb1 daml-lf: adapt interface reader to Numeric (#2666) 2019-09-02 10:02:56 +02:00
Remy
20649cf78e ledger-api: rename decimal field to numeric in value Proto (#2688)
* add Numeric.java

* ledger-api: rename `decimal` field to `numeric` in value protobuf

* Address Gerolf's comment

* ledger-api: add missing renammings

* ledger-api: relax syntax of numbers that can be sent as numerics

* extractor:  fix

* leger-api: change format of number though ledger api

* daml-lf: fix numeric regexp

* ledger: fix tests
2019-08-29 22:51:33 +00:00
Remy
834ca8d7ae
engine: fix string to numeric conversions (#2681) 2019-08-28 16:20:06 +02:00
Remy
997212c8f4
daml-lf: decommission internal Decimal type in favor of Numeric. (#2594)
* daml-lf: ValueDecimal -> ValueNumeric
* Apply suggestions from Stephen's code review
   Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
* extractor: add comment about Numeric SQL format
2019-08-22 18:07:47 +02:00
Remy
daf10bf247
daml-lf: make LF parser numeric compatible (#2587) 2019-08-19 16:31:33 +02:00
Remy
832be124e0 engine: SDecimal -> SNumeric (#2577) 2019-08-16 16:17:56 +00:00
Remy
6920a6be11
daml-lf: base implementation of Decimal on top of Numeric (#2569)
* daml-lf: base implementation of Decimal on top of Numeric
2019-08-16 15:33:10 +02:00
Remy
e8995ca0ec Daml-lf: Core type Numeric (#2556)
* daml-lf: rename data.assert function to data.assertRight

* daml-lf: implement internal Numeric type

* Apply suggestions from Gerolf

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Address Gerolf's comments

in particular add tests for round and toLong
2019-08-16 07:40:12 +00:00
Martin Huschenbett
803fee057f Show function names in stack trace on failing scenario (#2516)
* Show function names in stack trace on failing scenario

So far, we've only shown the location of the function but not its name.
Now, we add the name of the function as well.
2019-08-14 09:16:07 +02:00
Stephen Compall
0d72f84fe8
new JSON formats for some DAML-LF data (#2463)
* daml-lf: make match2's fallback lazy, and add tuple2 syntax for better inference

* lf-value-json: when decoding, branch on type, *then* on value

- restores totality checking for interface ADT
- makes alternate input formats obvious at a glance
- lets us restructure in the future to cache type branch lookups

* lf-value-json: support numbers for int64s

* lf-value-json: test failures and more timestamp cases

* lf-value-json: adapt timestamp decoding to new rules

* lf-value-json: cleanup

* lf-value-json: decode Number as decimal

* lf-value-json: forbid yet another timezone format

* lf-value-json: don't copy

* lf-value-json: replace {Some, {None optional format with nest-sensitive [] format

* lf-value-json: round decimals rather than rejecting, in either string or number form

* lf-value-json: support output to JsNumber for int64s and decimals, separately

- not dependent on JavaScript safe integer range, because that is not
  the purpose of this option

* lf-value-json: uncomment some now-tested rounding cases

* allow None record fields to be omitted in object syntax

* release notes
2019-08-13 17:00:08 -04:00
Gary Verhaegen
99ea93168d
update copyright notices (#2499) 2019-08-13 17:23:03 +01:00
Stephen Compall
8625d2e81b
dedupe specification of LF value, transaction versions in Scala (#2372)
* daml-lf: dedupe specification of value, transaction versions in Scala

- also test that some language Dev is in the last entry

* adapt LfVersionsSpec
2019-08-01 16:48:11 -04:00
Stephen Compall
52efc0966c
daml-lf: new data-scalacheck library for ImmArray, FrontStack, et al (#2342)
* new data-scalacheck library for ImmArray, FrontStack, et al

* add checkLaws functions as library to data-scalacheck

* make checkLaws more configurable at use points
2019-08-01 11:10:56 -04:00
Stephen Compall
5a707960d2
navigator: replace Value<->ledger-api conversion with ledger-api-common functions (#2292)
* ledger: refactor validateValue from CommandsValidator to new ValueValidator

* daml-lf: add a matchable class for leaf Values except ValueContractId

* ledger: use IdentifierResolverLike for CommandsValidator

* daml-lf: add Traverse[FrontStack]

* navigator: define readRecordArgument and readArgument in terms of ledger-api-common ValueValidator

* navigator: clean up fillInVariantTI

* navigator: replace ledger-api Value writer with LfEngineToApi from ledger-api-common

* navigator: use traverse

* navigator: remove dead sequenceMap

* navigator: adapt to new ValueValidator structure

* ledger: dealias validateValue call

* navigator: a good use case for traverseEitherStrictly
2019-07-26 14:24:34 -04:00
Stephen Compall
9367d6658f
Move navigator's ApiCodecCompressed to new lf-value-json library (#2136)
* move ApiCodecCompressed, ApiValueImplicits, and some aliases to new lf-value-json package

* adapt navigator to moved pieces

* start defining scalacheck extension to ApiCodecCompressedSpec

* experiment with an inductive case in TypedValueGenerators

* finish a List case for TypedValueGenerators; it's revealing

* remove accidentally readded duplicate aliases

* start tying knots in TypedValueGenerators

* verbatim copy ApiCodecCompressedSpec to lf-value-json

* shift some tests from navigator to lf-value-json

* test Optional and Map for ApiCodecCompressed

* heavier random testing of ApiCodecCompressed

* remove unused dependencies from lf-value-json
2019-07-16 16:38:33 -04:00
Stephen Compall
1a93db264d
navigator: Use LF Value ADT in place of ApiValue ADT (#2053)
* replace ApiValue ADT with aliases to daml-lf/transaction Value ADT

* porting rest of navigator to LF Value ADT

* porting more of navigator to LF Value ADT

* last error, not first

* rename ApiValueImplicits file

* special conversion features for ImmArray and FrontStack

- just .to[ImmArray] or .to[FrontStack] any random collection

* finish porting most of navigator main code

* use numeric indices for record field name fallback when pretty-printing

* tuples are not serializable

* use numeric indices for label fallback in JSON verbose encoding

* make traverseEitherStrictly more likely to preserve the seq's class

* to shortcut for ImmArraySeq .to[ImmArraySeq]

* compiling, passing navigator backend tests

* test traverseEitherStrictly more, er, strictly

* pass scalacopts through to scaladoc

* deal with unused warning

* remove unneeded function

* simpler error reporting, more private functions in ApiCodecCompressed

* move slowApply to FrontStack, test it so it actually works

* remove unneeded toStrings; better error from impossible ValueTuple case

* scalafmt FrontStackSpec

* support alternative, label-free record JSON encoding

* fuse some list operations

- suggested by @stefanobaghino-da; thanks

* blue error message
2019-07-16 10:53:17 -04:00
Stephen Compall
096e4c02ef
shortcut to ImmArray#map from ImmArraySeq#map (#2076) 2019-07-10 11:54:39 -04:00
Francesco Mazzoli
6cc5510dae purge all Map#mapValues from daml-lf codebase (#1864)
fixes #1861.
2019-07-04 15:08:43 +00:00
Robert Autenrieth
c14b909c71 Implement package management API in the sandbox (#1610)
* store archive size in `DarReader`

* rename `SandboxTemplateStore` to `SandboxPackageStore`

* store package info in `SandboxPackageStore`

* introduce package upload / read to write / index services

not tested yet, just a ton of plumbing

* WIP test the package service

* Fix build errors after rebase

* Move packages service to v2

* Ledger API client uses ledger API types

* Fix ReflectionIT

* Correctly handle uploading invalid dar files

* Fix reading DAR entry file sizes

* Improve package management IT

* Improve handling of duplicate packages

* Fix language-support build

* Use unique party and command names

* Rename lfpackage to language

* Rename SandboxPackageStore to InMemoryPackageStore

* Remove getCurrentTime ledger method

* Improve package management IT

* Move InMemoryActiveContracts and InMemoryPackageStore

* Use case object for UploadDarResult.Ok

* Address review comments

* Update release notes

Fixes #1311

* Use BazelRunfiles in test
2019-06-19 12:31:53 +00:00
Remy
11b602da03 Scala Daml-lf encoder (#1715)
* encode ast in protobuf

* daml-lf parser: make defaultPackageId and languageVersion parametric

* daml-lf rewritting of AST

* test ast encoder

* copyright

* test function type encoding

* daml-lf add parameter to parser implicits

* damlfl-as stands for "damllf assembler"

* move encoder in its own private package
2019-06-18 17:32:21 +00:00
Jussi Mäki
9c0a956e1f
Allow slashes in ledger strings (#1705)
* Allow slashes in ledger strings

Using '/' as the divider is a natural choice for some ledger backend
implementations and in application/workflow etc. identifiers.

* Update value.proto to include slash in LedgerStrings

* Update release notes
2019-06-17 12:41:52 +02:00
Gerolf Seitz
8e144420c4
Correctly implement hashCode for ImmArray, FrontStack, BackStack (#1641)
ImmArray, FrontStack, and BackStack violated the contract between equals
and hashCode. ImmArray had a wrong implementation of hashCode, and
FrontStack and BackStack had no implementation at all.

Fixes #1623.
2019-06-13 22:40:58 +02:00
Remy
0591075187 cleanup daml-lf scala packages (#1581)
* cleanup daml-lf scala packages

* Address Stephen's Comments

* update maven coordinates of language package
2019-06-12 15:55:48 +00:00
Remy
e115c39b67 make colon valid fro party literals + spec cleanup (#1467) 2019-06-04 15:50:02 +00:00
Remy
0d25b73d1f daml-lf add builtin to (un)pack string in code points (#1480)
, namely

* FROM_TEXT_CODE_POINTS: Text -> [Int64]
* TO_TEXT_CODE_POINTS: [Int64] -> Text
2019-06-04 14:06:25 +00:00
Remy
153a68d72d DAML Engine. Improve error message for arithmetic overflow (#1429)
* minor cleanup

* daml-lf improve message of arithmetic errors
2019-05-28 20:09:02 +00:00
Gerolf Seitz
9f21b6c669 Transaction service (#1404)
* Isolate daml-on-x prototype

* Implement TransactionService of index api.

Towards the first milestone of the daml-on-x integration steps

* fix compiler error after last-minute refactoring

* formatting

* remove unnecessary dependency from api-server-damlonx

* Fix some inconsistencies.
2019-05-27 15:36:57 +00:00
Remy
294abecf5d daml-lf remove LedgerIdString (#1376) 2019-05-24 12:08:28 +00:00
Remy
4f18b1afa7 DAML-LF internal type safety (#1192)
* Rename Value's ContractId to VContractId

* daml-lf: a bit more about PartyId

* daml-lf: Concatenable MatchingStringModule

* daml-lf make clear type used for Scenarios only

* daml-lf create ContractId, LedgerId, TransactionId

* sandbox-sql conversion util

* LedgerName -> LedgerString

* futher type cleanup in the sandbox

* daml-lf add test for LedgerString

* fixing tests

* a bit more safety in the DB

* Address Stephen's comments

* fix rebase

* More fixes for StringModule

* change length of LedgerString (256 -> 255)
2019-05-24 09:53:29 +00:00
Stephen Compall
5709fd0474
extractor: use daml-lf transaction Value ADT instead of bespoke ADT (#1117)
* stub away the components of LedgerValue

* rewrite v1 api converters to produce transaction Value ADT

* a needless 'case'

* shortcut methods for ImmArraySeq and tests

* porting extractor's JSON encoders to transaction Value

- JSON object ordering now matches record field order, instead of being
  its reverse
- Includes a new encoding for records missing labels, to a list of
  2-tuples

* porting extractor main transaction Value

- JSON date/time is just <num>s since epoch, maybe revisit

* adapt to Decimal newtype

* use daml-lf/data stringification for decimals in JSON

* snap aliases for Cid-less LedgerValue subtypes

* snap aliases for Cid-containing LedgerValue subtypes

* remove needless indirection

* test new missing-label Record JSON encoding

* write ImmArray traverse with syntax extension

* remove RecordField; move Name parsing to former RecordField converter
2019-05-15 20:21:09 -04:00
Remy
c7df212d42 Daml lf type safty (Decimal) (#1098)
* daml-lf: make Decimal type safe

* daml-lf: create Utf8String type

* daml-lf: cleanup in data package

* Address Stephen Comments

* daml-lf: remove UTF8String
2019-05-14 18:10:45 +00:00
Stephen Compall
c6a382db70 warn on use of 'return' Scala keyword; remove most usages (#956)
*  enable Return wartremover wart

* remove return keyword from various places in daml-lf

* remove return keyword from various places in ledger

* simpler ImmArray equals

* move traverseEitherStrictly to point of use

* remove return from ledger-api-server-example
2019-05-13 17:10:29 +00:00
Remy
2e3a87934b Daml lf type safty (ChoiceName, VarName, FieldName, ConstructorName) (#983)
* daml-lf: make DefinitionRef more typesafe

* daml-lf: Identifier -> DefinitionRef

* daml-lf: remove unsafe apply and copy methods from DottedName

* daml-lf: create identifier

* daml-lf: make ChoiceNames Identifiers

* daml-lf: cleanup TVar

* daml-lf: FieldNames & VariantConstructors -> Identifiers

* bazel fmt

* daml-lf: VarName -> Identifier

* daml-lf: drop return inside Ref.scala

* daml-lf Identifier -> Name

* daml-lf DefinitionRef -> Identifier

* daml-lf make iface more type safe
+ address Francesco's comments

* daml-lf: remove unsafe unapply from MatchingStringModule

* fix navigator

* Address Stephen's Comments
2019-05-13 11:17:12 +00:00
Remy
67772baf61 sandbox: correct KeyHasherSpec (#995) 2019-05-08 15:00:16 +00:00
Remy
0489c6e0a5
Daml lf type safty (Party & PackageId) (#761)
* daml-lf: split SimpleString into Party and PackageId

* daml-lf remove parameter from DefinitionRef
2019-05-06 20:40:43 +02:00
Stephen Compall
78bf1b878c
simplify, fix recursion in transaction equal instances (#903)
* Map case of Equal[Value] was not properly recursive

* protect SortedLookupList from equals incoherence

* simplify ImmArray creation

* clean up boilerplate in Equal[Value] definition

* use match2 for isReplayedBy
2019-05-06 13:59:28 -04:00
Stephen Compall
0ba1e5b386 increase ledger-api IT timeout; daml-lf, language-support/scala: remove long-deprecated symbols (#741)
* remove long-deprecated symbols from daml-lf and language-support scala libraries

* remove useless usage of 2-parameter TemplateId constructor

* grow ledger-api-integration-tests to account for usual 300s-ish runtime

* buildifier
2019-04-26 21:05:26 +00:00
Remy
433184626d Ledger Api: drop ledger api domain values in favor of LF-values (#649) 2019-04-25 23:15:12 +00:00
Remy
680239b5cf daml-lf fix and cleanup Decimal.fromString (#606) 2019-04-25 00:22:45 +00:00
Remy
de69ed61db
Leger Api. Simplify Ledger Api domain (#577)
* drop domain.Identifier in favor of Ref.Identifier
* drop domain.Party in favor of Ref.Party
2019-04-17 17:03:23 +02:00
Francesco Mazzoli
6af84051ee
fix various conversion functions from string to Decimal (#439)
* fix various conversion functions from string to Decimal

Fixes #399.

This fixes a critical bug -- since:

* The DAML-LF spec specifies that the scale of `Decimal` is 10 --
    that is, there are at most 10 digits past the decimal point:
    <79bbf5c794/daml-lf/spec/value.rst (field-decimal)>.
* However, the code converting from the string representation that
    we get on the wire was _not_ performing this check. This is due
    to two reasons:

    - `Decimal.check` is a function that checks whether a given
        `Decimal` is within the upper and lower bounds of what the
        DAML-LF spec specifies, but crucially it does _not_ check that
        the scale is not exceeded:
        <79bbf5c794/daml-lf/data/src/main/scala/com/digitalasset/daml/lf/data/Decimal.scala (L31)>.
        This behavior is correct in some cases (more on that later),
        but not in others. Crucially, `Decimal.fromString`, which was
        supposed to check if a decimal string literal is valid, used
        this function, which means that it accepted string literals
        containing numbers out of the required scale, rounding them to
        make them fit within the scale. This function has been renamed
        to `Decimal.checkWithinBoundsAndRound`, and a new function
        `Decimal.checkWithinBoundsAndWithinScale` has been added, which
        fails if the number provided has data not within the scale.
        `Decimal.fromString` now uses
        `Decimal.checkWithinBoundsAndWithinScale`.

    - `ApiToLfEngine.parseDecimal` assumed that `Decimal.fromString` _did_
        fail when passed numbers that were in any way invalid, and
        moreover did _not_ use the result of `Decimal.fromString`, but rather
        re-parsed the string into an unconstrained `BigDecimal`:
        <79bbf5c794/ledger/ledger-api-common/src/main/scala/com/digitalasset/platform/participant/util/ApiToLfEngine.scala (L96)>.
        The reason for the code to work this way is that in the past
        it was responsible for converting decimal strings both for the
        current engine but also for an older version of the engine which
        handled decimals of a different type. Both issues have been fixed.

* Therefore, `Decimal`s with scale exceeding the specified scale
    made it into the engine, and contracts could be created containing
    this invalid value.

* Once on the ledger, these bad numbers can be used to produce extremely
    surprising results, due to how `Decimal` operations are
    implemented. Generally speaking, all operations on `Decimal`
    first compute the result and then run the output through
    `Decimal.checkWithinBoundsAndRound`. The reason for this behavior
    is that we specify multiplication and division as rounding their
    output. Consider the case where the bad value 0.00000000005 made
    it to the engine, and is then added to 100. The full-precision
    result will be 100.00000000005, which after rounding becomes 100.
    Therefore, on a ledger where such invalid values exist, it is not
    the case that `y > 0 ==> x + y != x`, and so on.

Thanks a bunch to @briandbecker for the excellent bug report.

* fix failing test using to much precision
2019-04-14 13:49:28 +02:00