Commit Graph

110 Commits

Author SHA1 Message Date
Stephen Compall
c6a382db70 warn on use of 'return' Scala keyword; remove most usages (#956)
*  enable Return wartremover wart

* remove return keyword from various places in daml-lf

* remove return keyword from various places in ledger

* simpler ImmArray equals

* move traverseEitherStrictly to point of use

* remove return from ledger-api-server-example
2019-05-13 17:10:29 +00:00
Gabor Aranyossy
496cf5069f
Re-revert) "Ledger API Server refactor" (#1109)
* Revert "Revert "Ledger API Server refactor (#1077)" (#1096)"

This reverts commit e2fa13e62b.

* fixing race conditions why resetting the ledger
2019-05-13 17:23:03 +02:00
Gerolf Seitz
8158269b7d
Remove ExercisedEvent in Event oneof. (#1067)
* Remove ExercisedEvent in Event oneof.

The Event message is only used in the Transaction message. Flat
transactions do not contain exercised events, but only created and
archived events. Therefore we can remove the ExercisedEvent from the
Event oneof, without breaking transport compatibility.

HOWEVER: The Java Bindings used to use the data.Event class for both flat
transactions and transaction trees. To properly represent the actual
event types in the two transaction structures,
1) Event is now and interface and is only used in the Transaction class.
2) there is a new interface TreeEvent, which is used in the TransactionTree class.

* CreatedEvent implements Event and TreeEvent
* ExercisedEvent implements TreeEvent
* ArchivedEvent implements Event

Some "pathological" cases where an occurrence of an exercised event
would have resulted only in an exception, are now removed (see change in
LedgerApiV1.scala).

Fixes #960.
2019-05-13 14:36:13 +02:00
Remy
2e3a87934b Daml lf type safty (ChoiceName, VarName, FieldName, ConstructorName) (#983)
* daml-lf: make DefinitionRef more typesafe

* daml-lf: Identifier -> DefinitionRef

* daml-lf: remove unsafe apply and copy methods from DottedName

* daml-lf: create identifier

* daml-lf: make ChoiceNames Identifiers

* daml-lf: cleanup TVar

* daml-lf: FieldNames & VariantConstructors -> Identifiers

* bazel fmt

* daml-lf: VarName -> Identifier

* daml-lf: drop return inside Ref.scala

* daml-lf Identifier -> Name

* daml-lf DefinitionRef -> Identifier

* daml-lf make iface more type safe
+ address Francesco's comments

* daml-lf: remove unsafe unapply from MatchingStringModule

* fix navigator

* Address Stephen's Comments
2019-05-13 11:17:12 +00:00
Gabor Aranyossy
e2fa13e62b Revert "Ledger API Server refactor (#1077)" (#1096)
This reverts commit 17060b6225.
2019-05-13 10:31:14 +00:00
Gabor Aranyossy
17060b6225
Ledger API Server refactor (#1077)
* adding necessary @volatiles

* services are factored out

* no need for ResetService to stop the services before

* removing not needed callback methods

* mostly renaming things

* some further cleanup
2019-05-13 10:51:36 +02:00
Gabor Aranyossy
4abc18d8a1
avoiding linear searching for transactions from genesis (#994)
* avoiding linear searching for transactions from genesis

* validating transaction ids

* one more failing test nailed down

* fixing off by 1 error

* docs + release notes

* fixing EventIdFormatterSpec

* fixing broken validation test
2019-05-10 11:28:14 +02:00
Francesco Mazzoli
de54e8f60f
dynamically check type of contract ids, fixes #1005 (#1037)
Up to now, the engine blindly assumed that contract ids pointed to
contracts of the right type. However, this assumption is faulty:
contract ids coming from the Ledger API cannot be type checked
in command translation since we need access to the contract itself
to do so.

This caused some seriously surprising / broken behavior: one could
send an exercise command with the wrong template id and still go
through, or break internal invariants about the type of choices.

This commit fixes this by checking that the type of the contract
instances we fetch is correct at runtime.

cc @hurryabit @dajmaki @remyhaemmerle-da @S11001001 @meiersi-da
2019-05-09 17:11:05 +02:00
Jussi Mäki
66934f8a1e
Fix log output of ledger-api-integration-tests (#1016)
The logback-test.xml file was not included as a resource.
2019-05-09 17:09:32 +02:00
Jussi Mäki
17438382b8 Remove ledger-api-server-example (#1017)
* Remove ledger-api-server-example

This code does not serve any purpose anymore as it has been replaced
by ledger/participant-state and friends.

* Remove ledger-api-server-example from Windows build
2019-05-09 17:04:38 +02:00
Gabor Aranyossy
b72377a950
cleaning up LedgerApiServer (#1036)
* cleanup/refactoring of LedgerApiServer

* fixed LedgerResource

* fixed ResetService to be great again
2019-05-09 15:21:03 +02:00
Gerolf Seitz
662a21f493
Fix LedgerResource after concurrent changes being merged to master (#1033) 2019-05-09 13:01:49 +02:00
Gabor Aranyossy
b6b9a08c82
increasing a perf test timeout due to flakiness (#1028) 2019-05-09 11:26:13 +02:00
Gerolf Seitz
c89f3bdebe
Sandbox: Respect MRT (#990)
Transactions with a record time > maximum record time are now rejected
with a timeout error instead of being committed to the ledger.
2019-05-09 11:10:54 +02:00
Gabor Aranyossy
82edab41f8
making sql queue depth configurable -> making back-pressure tests more reliable (#1027)
* making sql queue depth configurable -> making back-pressure tests more reliable

* fixing SqlLedgerSpec
2019-05-09 10:42:22 +02:00
Francesco Mazzoli
5b1e82b4ef run ledger-api-integration-tests exclusively (#1018)
might help with #1012
2019-05-08 17:43:01 +00:00
Remy
67772baf61 sandbox: correct KeyHasherSpec (#995) 2019-05-08 15:00:16 +00:00
Jussi Mäki
a23000446b
Check contract visibility in reference index service (#947)
* Use Sandbox's ActiveContracts in reference index service

This properly handles contract key updates and adds the required
checks for contract activeness and visibility.

Could consider moving the ActiveContracts.scala into another library,
but since we're soon starting the work to bring sandbox onto the
daml-on-x stack we might postpone that.

* reformat

* Exclude logback.xml from sandbox the library

This allows users of sandbox as a library to specify their
own logging configuration.
2019-05-08 14:36:28 +02:00
Stefano Baghino
f9839a7e4c
Re-order fields in records if all labels are provided (#989)
* Re-order fields in records if all labels are provided

Fixes #988

* Add line to release notes

* Fix line on release notes

* Document change on the Ledger API definition

* Simplify pattern matching

* Improve and fix tests

Integration tests now look for the "Missing field label" rather then the
"Mismatching record label" when inspecting errors. Furthermore, a test
has been added that ensures that repetitions in labels still raise an
error.

* Add comment

Comment about relationship between fields length check and fields reordering based on labels

Co-Authored-By: stefanobaghino-da <43749967+stefanobaghino-da@users.noreply.github.com>

* Improve Ledger API docstring

Explicitly state the semantics of labeled fields (i.e. repeated keys are not accepted)

Co-Authored-By: stefanobaghino-da <43749967+stefanobaghino-da@users.noreply.github.com>
2019-05-08 12:54:53 +02:00
Jussi Mäki
0dc49ea811
Fixes to kvutils when used as external workspace (#984)
* Export daml_kvutils_java_proto and add (un)packDamlLogEntryId

Export needed when using kvutils via bazel from another workspace.

* Use 'String' for participant state's LedgerId

Ref.PackageId is the wrong thing, and likely MatchingStringModule
isn't very usable from Java.
2019-05-07 17:55:11 +02:00
gleber
e419420205
Improve error messages of Ledger API Test Tool. (#949)
* Drop LedgerTestingHelpers.asyncFromTimeService as unused.

* ledger-api-test-tool: Add clues to asserts in the main code paths.

This ensures that users get more informative errors for the currently used
semantic tests.

* semantic tester: add operation description to an underlying ledger.

This allows to provide richer error messages to users of LedgerApiTestTool.
2019-05-07 16:01:26 +02:00
Gabor Aranyossy
1a4a571488
making batch queries nicer (#968) 2019-05-07 15:34:26 +02:00
gleber
1225b45ce5
Extend test durations on CI for Ledger API Test Tool driven test. (#944)
* Extend test durations on CI for Ledger API Test Tool driven test.

This introduces a command-line argument to scale timeouts used in the test.

* ledger-api-its: Make FutureTimeouts.timeout duration scaled inside.

Also include more information in the error message.
2019-05-07 14:52:49 +02:00
Moritz Kiefer
55d5fa5dea
Upgrade to nixpkgs 19.03 (again) (#962)
* Upgrade to nixpkgs 19.03 (#871)

* Fix nix-build restart

Without || true we will never get to the next line.
2019-05-07 13:55:30 +02:00
Remy
0489c6e0a5
Daml lf type safty (Party & PackageId) (#761)
* daml-lf: split SimpleString into Party and PackageId

* daml-lf remove parameter from DefinitionRef
2019-05-06 20:40:43 +02:00
Gabor Aranyossy
dd1ac5067b
adding a simple test to hedge against existing sql migration files (#943)
* simple test to hedge against existing sql migration files

* copyright

* adding better failure message

* adding intructions on generating the digest file
2019-05-06 17:34:56 +02:00
Nick Smith
d725d50be9 add result of exercise to transaction -- fixes #479 (#583)
Modify the DAML Engine, Ledger API and Sandbox to pass the result of the
exercise as a field of the transactions.
2019-05-06 14:33:44 +02:00
Jussi Mäki
32a168a999
Index service refactoring (#928)
* Add configuration to LedgerInitialConditions

* Wait until index state is established in reference index service

This fixes a race condition where the daml-on-x server has been
brought up before the index service can serve it.

* Remove ledgerId checks from index service

Now that we have decided on non-changing ledger identifiers, remove
the ledgerId checks from index service and simplify the interface.

* Reformat and fix leftover ??? from DamlOnXTransactionService
2019-05-06 13:59:06 +02:00
Gabor Aranyossy
990a8ea5db
Dispatcher cleanup (#855)
* dispatcher cleanup

* doc improvements

* extracting SubSource from Dispatcher

* moving tests
2019-05-06 11:43:51 +02:00
Moritz Kiefer
9deb4ae7b2 Revert "Upgrade to nixpkgs 19.03 (#871)" (#917)
This reverts commit 3d8acde916.

For some reason that commit seems to have resulted in a lot of
"unexpected end of file" errors during cache downloads. I do not know
what is going on here or how to fix it so let’s revert it for now.
2019-05-05 09:10:20 +00:00
Moritz Kiefer
3d8acde916 Upgrade to nixpkgs 19.03 (#871) 2019-05-04 20:01:48 +00:00
gleber
c85503159b ledger-api-test-tool: uniquify command ids. (#884)
Add 'commandIdMangler' lambda to SemanticTester which forms command identifiers,
which can be passed via a constructor.
2019-05-03 16:46:15 +00:00
Francesco Mazzoli
e8261f2fd9
clarify the role of party sets in various locations (#766)
* clarify role of party sets in various locations

* use 1.dev for tests...

...so that we're sure we're testing the latest value / tx versions.
2019-05-03 17:09:59 +02:00
gleber
e057265af4 LedgerApiTestTool: exit with success when doing '--extract' (#889) 2019-05-03 14:41:55 +00:00
Gerolf Seitz
ecf6ece8c4
CommandService returns useful data for successful submissions (#875)
Submitting a command via the CommandService now returns either the
transaction id (SubmitAndWaitForTransactionId), the flat transaction
(SubmitAndWaitForTransactionResponse), or the transaction tree
(SubmitAndWaitForTransactionTreeResponse).

This means that users don't have to wade through the transaction stream
to retrieve the resulting transaction. This is particularly useful in
combination with #479.

Fixes #406
2019-05-03 16:01:41 +02:00
gleber
b3e2e10897 scenario-tester: Allow to mangle names used in scenarios before their executions. (#795)
* Allow to mangle names used in scenarios before their executions.

This allows to run a scenario against a long-running server repeatedly and avoid
clashes between runs, since each party is unique (up to the randomness used).

* ScenarioRunner: add test for partyNameMangler
2019-05-03 13:04:23 +00:00
Robert Autenrieth
44e43aa620 Reject running sandbox with --scenario and -w (#840)
Scenarios are always run in static time mode.
This change prevents users from thinking the scenario
was run in wallclock mode.
2019-05-03 10:38:03 +02:00
Gerolf Seitz
f4d8e134e3
Add TransactionService methods for looking up flat transactions (#830)
This change is needed in preparation of #406, where we want to return a
transaction tree and flat transaction after a SubmitAndWaitForTransaction(Tree).
2019-05-03 09:03:12 +02:00
Jussi Mäki
746c69255e
Move out classes from participant-state package object (#849)
Having the classes specified in the package object makes the
unusable from Java.
2019-05-02 17:16:27 +02:00
Francesco Mazzoli
cf89366ac6 respect contract visibility when looking up contract keys (#753)
* remove outdated flags from comments

* update tests in preparation for visibility rules

* check visibility of contract keys, fixes #751

* add release notes

* include visibility check in scenario runner

* stephen's suggestions

* fix type error in api server example

* scalafmt

* bring TransactionFiltration.scala up to speed with disclosure

see
<faf1cde1a1/ledger-api/grpc-definitions/com/digitalasset/ledger/api/v1/event.proto>
for an in-depth description. fixes #794.

* add tests for contract key visibility

* move a lot of ledger testing utils out of command checks

* move a lot of ledger testing utils out of command checks

* remove useless checks regarding divulgence

* add tests regarding event witnesses

* update release notes

* re-introduce safety when disclosing
2019-05-02 14:02:30 +00:00
Gabor Aranyossy
c501d64264
fixing bug for mishandling potential offset gaps (#802)
* hedging against possibly gaps in offsets

* doing the math right

* adding some notes

* extracting pipeline logic
 Please enter the commit message for your changes. Lines starting

* extracting "run until" logic to be testable

* testing _run until_ logic

* adding release note

* copyright

* adding link to PR

* changing lookUpByTransactionId to use TransactionPipeline

* fixing issue identified by Extractor tests
2019-05-02 15:00:41 +02:00
gleber
77edbba8d3 reference-server: Fix offset handling for empty resulting transaction streams. (#793)
* reference-server: Add invariant that observed offset is monotonic.

* reference-server: Fix offset handling for empty resulting transaction streams.

If the requested transaction stream (by range) returns empty, code - previously
- would reset 'currentOffset' to None, which was breaking invariants for the
next transaction stream request. Ensure that 'currentOffset' is maintained in
such scenario.
2019-04-30 14:50:17 +00:00
gleber
f6a29f3f69 Drop unused LedgerTestingHelpers from //ledger/sandbox. (#760)
Looks like only LedgerTestingHelpers from //ledger/ledger-api-integration-tests/
is used.
2019-04-30 13:20:13 +00:00
Gabor Aranyossy
43ce45c84e
changing parameters table to have typed entries (#787)
* chaning parameters table to have typed entries

* finishing thought on parameters table
2019-04-30 14:16:15 +02:00
Jussi Mäki
2328914ba4 Update participant-state-kvutils based on feedback (#780)
- Make `getLedgerInitialConditions` a `Source` to mirror `stateUpdates`.
  This makes it easier to implement in Java.

- Expose transaction submission outputs with `KeyValueSubmission.transactionOutputs`
  for ledger implementations that require knowledge of outputs up-front.

- Fix typo in unpackDamLLogEntry.
2019-04-30 09:14:20 +00:00
Gabor Aranyossy
f8a5fdc493
SQL performance improvements (#752)
* document performance bottleneck source

* making transaction ingestion parallel

* dump

* reverting test parameters

* updating ledger end in a race-safe way

* ignoring ledger entry persistence result

* fixing faulty update logic

* fixing tests relying on invalid semantics

* Update ledger/sandbox/src/main/scala/com/digitalasset/platform/sandbox/stores/ledger/sql/dao/PostgresLedgerDao.scala

Co-Authored-By: gaboraranyossy-da <gabor.aranyossy@digitalasset.com>

* Update ledger/sandbox/src/main/scala/com/digitalasset/platform/sandbox/stores/ledger/sql/SqlLedger.scala

Co-Authored-By: gaboraranyossy-da <gabor.aranyossy@digitalasset.com>
2019-04-29 21:22:01 +02:00
mziolekda
dc17c30d76 implement TLS in reference daml-on-x server (#739)
* implement TLS in reference daml-on-x server

* add tls configuration to LedgerApiTestTool

* add ssl tests to the reference ledger server
2019-04-29 17:01:26 +00:00
Francesco Mazzoli
71f61b0691
sign off / document first version of SQL schema. (#759) 2019-04-29 18:23:02 +02:00
Jussi Mäki
a8be6442bf
DamlOnX: release kvutils (#763)
* Move package comment to correct place

* Update kvutils package documentation

* Release participant-state-kvutils
2019-04-29 16:05:22 +02:00
Gabor Aranyossy
8e54bf2e67
documenting persistence with Postgres in SDK docs (#722)
* documenting persistence with Postgres in SDK docs

* revealing jdbcurl flag as it's docuemnted now

* updating release notes

* Update docs/source/tools/sandbox.rst

Co-Authored-By: gaboraranyossy-da <gabor.aranyossy@digitalasset.com>

* Update docs/source/tools/sandbox.rst

Co-Authored-By: gaboraranyossy-da <gabor.aranyossy@digitalasset.com>

* Update docs/source/tools/sandbox.rst

Co-Authored-By: gaboraranyossy-da <gabor.aranyossy@digitalasset.com>

* Apply suggestions from code review

Co-Authored-By: gaboraranyossy-da <gabor.aranyossy@digitalasset.com>

* changed release note entry as suggested

* Apply suggestions from code review

Co-Authored-By: gaboraranyossy-da <gabor.aranyossy@digitalasset.com>
2019-04-29 15:22:05 +02:00