Commit Graph

3632 Commits

Author SHA1 Message Date
Stefano Baghino
ceeb7c3487
Re-use AkkaBeforeAndAfterAll in AbstractSandboxFixture (#5132)
* Re-use AkkaBeforeAndAfterAll in AbstractSandboxFixture

CHANGELOG_BEGIN
CHANGELOG_END

* Do not expose the Akka execution context directly
2020-03-23 11:56:47 +00:00
Samir Talwar
3d6ecc13b6
Sandbox: Configurable metrics output. (#5113)
* sandbox: Clean up `MetricsReporting` a little.

Make sure it closes both reporters, and avoid starting things in a
constructor.

* sandbox: Add hidden options for enable metrics reporting.

* sandbox: Add a disambiguating name to the DB connection/thread pools.

CHANGELOG_BEGIN
- [Sandbox] DB connection pool metrics names have changed slightly, from
  ``daml.index.db.connection`` to ``daml.index.db.connection.sandbox``.
- [Ledger Integration Kit] DB connection pool metrics names have changed
  to disambiguate the StandaloneApiServer from the
  StandaloneIndexerServer. The former now has a ``.ledger-api-server``
  suffix, and the latter now has a ``.indexer`` suffix.
CHANGELOG_END

* sandbox-next: Use the same metrics registry for the API and indexer.

* sandbox: Give a useful error message on an invalid metrics reporter.

And simplify the error messages.

With the arguments `--client-auth=foo --metrics-reporter=foo`, we now
get the output:

```
Error: Option --client-auth failed when given 'foo'. Must be one of
  "none", "optional", or "require".
Error: Option --metrics-reporter failed when given 'foo'. Must be one of
  "console", or "csv:PATH".
Try --help for more information.
```

* sandbox: Pull out more helpers in `MetricsReporting`.

* sandbox: Rename MetricsReporter classes so they don't clash.

* sandbox: Wrap the `name` parameter in a `ServerName` tagged string.

For safety. Yours, not mine.

* sandbox: Push metrics to Graphite with `--metrics-reporter=graphite`.

* sandbox: Make `MetricsReporter.Graphite` singly-lazy, not doubly-.

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* sandbox: Replace `ServerName` with `ServerRole`.

* sandbox: Fix usage of `ServerRole.Testing` in `LedgerResource`.

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-03-23 10:59:13 +00:00
Stefano Baghino
c00af40fe5
Cleanup: InfiniteRetries is used only in sandox-perf (#5131)
* Cleanup: InfiniteRetries is used only in sandox-perf

CHANGELOG_BEGIN
CHANGELOG_END

* Fix import
2020-03-23 10:49:11 +00:00
Stefano Baghino
97ec70e508
Add missing truncations from reset service and fix perf tests (#5129)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 10:36:32 +00:00
Gary Verhaegen
95d9cdb567
faster hlint with fmt.sh --diff (#5114)
This PR adds new behaviour to the existing --scalafmt-diff flag to also
use a diff-based approach for hlint. Consequently, the flag is renamed
to the simpler and more generic `--diff`, in a backwards-compatible way.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 11:22:02 +01:00
Gerolf Seitz
f9fb888cc4
Allow for predictable contractIDs and transactionIDs (#5112)
- New seeding type "Static" that uses a fixed seed for generating new seeds.
This is only used in Sandbox and Sandbox-next.

- Remove the fuzzing for submission time in Engine.scala

- DAML-on-SQL: Create new log entry IDs from a provided SeedService.
This allows for generating deterministic transaction IDs in
Sandbox-Next.

Fixes #5107

CHANGELOG_BEGIN
[Sandbox] Add contract-id-seeding=static to allow for predictable contract IDs. This is useful for documentation,
to be able to refer to a specific contract ID instead of having to write "note down the contract ID you see on the screen. we will use it later."
[DAML-on-SQL] Derive the next log entry ID using the provided SeedService. This allows us to also deterministically create transactionIds in static time mode together with `--contract-id-seeding=static`. This should only be used for demos or documentation.
CHANGELOG_END
2020-03-23 10:55:40 +01:00
Samir Talwar
3e95bb7bec
Sandbox: Simplify the ApiCommandService a little. (#5125)
* sandbox: Remove a few levels of indentation in `ApiCommandService`.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Get rid of the `LowLevelCommandServiceAccess` trait.

It only has one implementation. Just use it.

* sandbox: Remove unused type aliases from ApiCommandService.
2020-03-23 08:55:25 +00:00
Samir Talwar
28c78e5978
sandbox | kvutils/app: Re-use the Materializer in the JdbcIndexer. (#5124)
* kvutils/app|sandbox: Rename `system` to `actorSystem`.

* sandbox: Pass a materializer in to the StandaloneIndexerServer.

There's no need for it to construct one when the caller always has one
available.

CHANGELOG_BEGIN
CHANGELOG_END

* recovering-indexer: Use `SubmissionId` instead of `LedgerString`.

Where appropriate.

* sandbox: Use the materializer implicitly in JdbcIndexer.
2020-03-23 08:55:19 +00:00
Oliver Seeliger
45acfec218
Ledger api server indexer initializes proper ledger_end after restart (#5122)
by reading parameters.ledger_end column instead of parameters.external_ledger_end.

CHANGELOG_BEGIN
[Ledger API Server]: Upon restart, ledger api server continues consuming unconsumed events rather than
all events from beginning.
CHANGELOG_END

Closes #5121
2020-03-23 09:53:48 +01:00
Gerolf Seitz
e82b8d743b
Add missing truncate for participant_events (#5117)
The missing truncate caused "Duplicate Submissions" after a reset in
between the performance tests.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 09:06:02 +01:00
Shayne Fletcher
2525a4610a
Update hlint (#5123)
changelog_begin
changelog_end
2020-03-22 16:30:11 +00:00
Shayne Fletcher
bf22d8db59
Fix package name (#5120)
changelog_begin
changelog_end
2020-03-22 12:01:56 -04:00
Stephen Compall
f2b068f0b5
remove List and TextMap support from the JSON query language (#5099)
* remove List and TextMap support from the JSON query language

CHANGELOG_BEGIN
- [JSON API - Experimental] Removed support for maps and lists from the query language.
  See `issue #4855 <https://github.com/digital-asset/daml/issues/4855>`_.
CHANGELOG_END

* GenMap won't be supported in general, remove FIXME

- thanks @leo-da

* more documentation on unsupported query types

- as suggested by @hurryabit; thanks
2020-03-20 16:22:22 -04:00
Andreas Herrmann
4f7c52c981
DAML script runner takes CompiledPackages (#5110)
* daml-script runner from compiledPackages

* Factor out script parameter conversion

* Use ADT for DAML script (Action & Function)

* run --> runwithClients

* DAMl script Runner `run` convenience function

* REPL use pure script runner constructor

* DAML script runner holds script itself

* scalafmt

* comments

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-20 17:52:54 +00:00
Stefano Baghino
f67c32b002
Integrate transaction lookup on new schema in Ledger API (#5104)
* Integrate transaction lookup on new schema in Ledger API

Re-wires all transaction lookups to the new schema

CHANGELOG_BEGIN
CHANGELOG_END

* Always return the agreement text

CHANGELOG_BEGIN
[Ledger API Server] The metric 'daml.index.lookup_transaction' has been
replaced by 'daml.index.lookup_flat_transaction_by_id' and
'daml.index.lookup_transaction_tree_by_id', which record the same events
but with more granularity regarding the type of lookup.
CHANGELOG_END

* Ensure agreement text invariant in a single place

* Do not compare the order in which witness parties appear in an event

* Hide command identifier from non-submitters in transaction trees

* Fix time assigned to transaction to be the ledger effective time and not the record time

* Store transactions from initial state into the new schema
2020-03-20 17:22:42 +00:00
Leonid Shlyapnikov
ffabc2b3e5
Bumping up http-json tests size to large, adding some extra logging (#5109)
* Bumping up http-json tests size to large, adding some extra logging

* Removing the logging from prod code, adding some in tests.

changelog_begin
changelog_end
2020-03-20 16:48:11 +00:00
Moritz Kiefer
be7f0802c6
Bump ghcide (#5103)
This includes some completion improvements.

changelog_begin
changelog_end
2020-03-20 16:43:59 +00:00
Samir Talwar
1ffd23a6a2
Sandbox: Make JdbcLedgerDao create its own execution context. (#5091)
* sandbox: Don't let just anyone construct a DbDispatcher.

It's the job of the JdbcLedgerDao.

* sandbox: Clean up the DbDispatcher a little.

* sandbox: JdbcLedgerDao now creates its own execution context.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Make `defaultNumberOfShortLivedConnections` private.
2020-03-20 15:30:35 +00:00
Moritz Kiefer
18033271de
Remove bzip2 library (#5105)
This used to be a dependency of our zip library that you couldn’t
disable. However, in the meantime it has gotten a flag to disable
it and we don’t actually use it. Given that the upstream URL is dead
atm now seems like a good time to kill it.

changelog_begin
changelog_end
2020-03-20 15:05:33 +01:00
Gary Verhaegen
b45c7aed69
fix 0.13.55 release notes typo (#5074)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-20 13:25:23 +00:00
Stefano Baghino
febace7391
Add lookup for transaction tree by id (#5089)
* Add lookup for transaction tree by id

CHANGELOG_BEGIN
CHANGELOG_END

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395156203

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395163051

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395165561

* Address multiple comments

- https://github.com/digital-asset/daml/pull/5089#discussion_r395185084
- https://github.com/digital-asset/daml/pull/5089#discussion_r395185290
- https://github.com/digital-asset/daml/pull/5089#discussion_r395185461

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395157643

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395160343

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395482878

* Use refactored testing utilities

* Add test for partial transaction visibility
2020-03-20 13:28:11 +01:00
Stefano Baghino
a7c01e1b6f
Refactor transaction generation and add generation of more complex transaction (#5101)
Makes storing and generating test transaction more composable (store just takes a generated transaction)

- Add a nonTransient utility method to retrieve contracts that have been created but not consumed as part of a transaction
- Add an addChildren utility method to add children to a transaction to allow to create more complex test transactions
- Add a transaction generator that uses the aforementioned addChildren to create a more complex transaction

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-20 12:03:18 +01:00
Martin Huschenbett
7037f4b6a8
Getting started guide: Invoke daml start directly (#5102)
* Getting started guide: Invoke daml start directly

Since we no can configure arguments we wish to pass to DAML Sandbox
when running `daml start` in `daml.yaml`, we don't need the mysterious
`daml-start.sh` script anymore. That's why
https://github.com/digital-asset/create-daml-app/pull/78
removes the script from `create-daml-app`.

This PR adjust the getting started guide accordingly.

This fixes #4295.

CHANGELOG_BEGIN
CHANGELOG_END

* Fix expected message when daml start successful
2020-03-20 10:57:23 +00:00
Samir Talwar
6f9b516251
Sandbox: Remove duplication in JdbcIndexerFactory. (#5096)
* sandbox: Remove duplicate parameters in JdbcIndexerFactory.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: `InitializedJdbcIndexerFactory` is just a ResourceOwner now.

* sandbox: Rename helper methods in `JdbcIndexerFactory`.

* sandbox: Make `JdbcIndexer#initialized` look like it does something.
2020-03-20 09:45:40 +00:00
Robin Krom
7a9648c714
language: react: tests for useReload. (#5095)
This adds tests to see that the `useReload` hook of @daml/react works.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 22:25:28 +01:00
Shayne Fletcher
78d34dd9ca
Experiment with 'eitherDecode' (#5098)
changelog_begin
changelog_end
2020-03-19 20:07:19 +00:00
Stefano Baghino
d966c17f0b
Refactor events (#5097)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 19:18:47 +01:00
Moritz Kiefer
be90649186
Support partial patterns in DAML repl (#5093)
* Support partial patterns in DAML repl

This PR improves the support for partial patterns in DAML repl by
making sure that they fail on the line itself rather than some
subsequent line and avoids the partial pattern match warnings on all
following lines.

changelog_begin
changelog_end

* Fix tests
2020-03-19 17:28:40 +00:00
Martin Huschenbett
8bd2172569
daml2ts: Remove "itself" from progress output (#5088)
I've received some feedback that the "itself" in
```
Generating <some hash> as itself
```
is confusing. Also, the first noun in each line said what is being
_processed_ not what is being _generated_. I've fixed that too.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 18:05:54 +01:00
Moritz Kiefer
c2c0557970
Support complex patterns in DAML REPL (#5087)
* Support complex patterns in DAML REPL

Previously, we only supported simple variable patterns in DAML
repl. This PR extends this to support pretty much all patterns. The
main complexity here is in handling shadowing correctly but I’m
reasonably confident the solution here is sensible.

What doesn’t work properly yet is partial patterns. I’ve added a
comment describing the issue and a potential solution. Given that this
isn’t a regression apart from maybe a slightly worse error message, I
think it makes sense to merge this PR and fix that separately.

changelog_begin

- [DAML REPL - Experimental] You can now use more complex patterns in
  statements, e.g., ``(x,y) <- pure (1,2)``.

changelog_end

* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/Repl.hs

Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/Repl.hs

Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* Improve error messages

* update documentation

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2020-03-19 16:17:50 +00:00
Robin Krom
b7f36d5e10
streamQuery tests (#5059)
* language:daml-react: tests for useStreamQuery/useStreamFetchByKey

This adds tests for `useStreamQuery` and `useStreamFetchByKey` of the
`@daml/react` library.

CHANGELOG_BEGIN
CHANGELOG_END

* renamed hooks.test.ts -> index.test.ts
2020-03-19 16:42:34 +01:00
Samir Talwar
82f2685e89
sandbox: Fix warnings by making test variables private. (#5086)
And move them to a companion object, just in case.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 14:12:06 +00:00
Remy
1216fd1a6f
Engine: fix missing usage of Enum Svalue rank (#5083)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 14:27:13 +01:00
Robert Autenrieth
60a845ec72
Add new protobuf fields for new ledger time model (#5036)
* Add new protobuf fields for new ledger time model

Fixes #4344.

CHANGELOG_BEGIN
CHANGELOG_END

* Adapt haskell bindings
2020-03-19 13:53:49 +01:00
Martin Huschenbett
d9f2eed632
daml2ts: Simplify generated build config (#5085)
We don't use TypeScript's project references. Hence there's no point in
passing `--build` to `tsc`. Let's stop doing that hence.

By default, the TypeScript compiler type checks the `.d.ts` of all
dependencies of a project. This is painfully slow. For the packages
generated by `daml2ts` it also doesn't make a lot of sense since we control
all dependencies except for `@mojotech/json-type-validation`, which is
written in TypeScript itself and hence has very sane typings anyway.

This default behaviour can be disabled by setting `skipLibCheck` to `true`.
Doing that decreases the compilation time of _every_ single package generated
by `daml2ts` for the DAVL project from ~10s to ~2s. Let's get that time back!

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 12:21:05 +00:00
Moritz Kiefer
13816c75b3
Support uploading DARs to the trigger service (#5084)
* Support uploading DARs to the trigger service

This PR adds a new `upload_dar` endpoint that accepts a DAR as a
multi-part form request and adds it to the list of compiled packages.

I’ve also made the DAR passed in on startup optional now given the new
endpoint.

There is no endpoint for deleting a DAR so far but there is none on
the ledger API either so I think this not particularly urgent.

changelog_begin
changelog_end

* Address review comments
2020-03-19 13:02:04 +01:00
Andreas Herrmann
8527ec4c4b
Infer daml-script package id from script (#5076)
* Factor out common identifier generation

For `DA.Types`, `DA.Internal.Any`, and `Daml.Script`.

* Factor out Script type for DAML scripts

* Adapt DAML script test runners

* Adapt REPL

CHANGELOG_BEGIN
CHANGELOG_END

* ./fmt.sh

* Avoid `unapply`

addressing
https://github.com/digital-asset/daml/pull/5076#discussion_r394526881

* Pure Script.fromIdentifier

* Pure Script.fromDar

* Simplify test script discovery

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-19 11:59:06 +00:00
Nemanja
8dc99b583b
Follower terminology in the docs (#5067)
* Follower terminology in the docs

Changed the docs to a more twitter/Instagram like terminology where we use 'follower' relationship instead of 'friend'. Resolves #5015

CHANGELOG_BEGIN
CHANGELOG_END

* Update docs/source/getting-started/index.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/index.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/index.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/app-architecture.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/app-architecture.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Update docs/source/getting-started/first-feature.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-03-19 12:00:09 +01:00
Shayne Fletcher
9fbd3da302
Use type application for 'decode' (#5081)
changelog_begin
changelog_end
2020-03-19 06:53:04 -04:00
Remy
c097266edc
java-binding: fix protobuf testing generator (#5082)
* exclude com.daml.ledger.javaapi.data.ValueSpec test

* Revert "exclude com.daml.ledger.javaapi.data.ValueSpec test"

This reverts commit 9b737bc452.

* java-binding: fix testing Generators

* revert minor change

* changelog

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 09:59:36 +00:00
Stefano Baghino
499fd1b75a
Add method to look up flat transaction by id (#5080)
* Add method to look up flat transaction by id

CHANGELOG_BEGIN
CHANGELOG_END

* Limit visibility of helper
2020-03-19 10:01:09 +01:00
Oliver Seeliger
e50a33019c
StandaloneApiServer facility for lf-seeding opt-out (#5078)
Until all DAML-on-X implementations have a chance to implement
seeding support for daml-lf contract-ids.

Closes #5077

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 08:50:15 +00:00
nickchapman-da
d81caa9568
rework daml_compile bazel rule (#5070)
Avoiding `damlc compile/package` commands (which we would like to deprecate), and replace with plain `damlc build` together with a post dar->dalf extraction step in the couple of places where we actually want the .dalf for testing.

changelog_begin
changelog_end
2020-03-18 23:06:55 +00:00
Leonid Shlyapnikov
29c340fed0
2nd part of stabilizing flaky websocket integration test (#5079)
* 2nd part of stabilizing flaky websocket integration test,

handling empty events, which we may receive a bunch in the end of the test
when there is no activity

changelog_begin
changelog_end

* fixing `Offset.semigroup`, thanks @S11001001!
2020-03-18 17:04:01 -04:00
Remy
3cbba8a444
Engine: Use enum and variant svalues' enum and variant s rank (#5068)
CHANGELOG_BEGIN
CHANGELOG_END

We use the rank recently introduced in the enum and variant svalues for:
* pattern matchin
* value hashCode
* value ordering (so the oredering match the DAML-LF spec)
* value equality
2020-03-18 20:44:17 +01:00
Gary Verhaegen
7b9932a7f3
specify which channels to post releases to (#5075)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 17:22:02 +00:00
Stefano Baghino
856ed80728
Add tests for completion reader in JdbcLedgerDaoSpec (#5072)
* Add tests for completion reader in JdbcLedgerDaoSpec

CHANGELOG_BEGIN
CHANGELOG_END

* Add test for completion, make offset checks more strict

* Fix badly typed check

* Comment reason for weird matcher use

* Address https://github.com/digital-asset/daml/pull/5072#discussion_r394447235

* Address https://github.com/digital-asset/daml/pull/5072#discussion_r394447981

* Address https://github.com/digital-asset/daml/pull/5072#discussion_r394448506

* Address https://github.com/digital-asset/daml/pull/5072#discussion_r394451069
2020-03-18 16:34:36 +00:00
Shayne Fletcher
76a37b9560
Update instructions (#5069)
changelog_begin
changelog_end
2020-03-18 14:22:24 +00:00
Remy
c2070170b3
Engine: add rank to Enum and Variant SValue (#5048)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-18 14:37:44 +01:00
Shayne Fletcher
8a9e5e778a
Notes on updating release version (#5066)
* Notes on updating release version

changelog_begin
changelog_end

* Update release/RELEASE.md

Co-Authored-By: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Corrections

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-03-18 13:06:38 +00:00