Commit Graph

3611 Commits

Author SHA1 Message Date
Moritz Kiefer
37dc2f29d3
Integrate create-daml-app into the assistant (#5152)
* Integrate create-daml-app into the assistant

fixes #4868

changelog_begin

- [DAML Assistant] Add a new ``daml create-daml-app`` command for creating a project based on
  `create-daml-app <https://github.com/digital-asset/create-daml-app>`_.

changelog_end

* Try random things hoping to fix windows

* Try random things hoping to fix things on macos
2020-03-24 18:29:40 +01:00
Andreas Herrmann
3b550b9933
Avoid recompiling loaded packages in REPL (#5157)
* Use MutableCompiledPackages in REPL

This avoids unnecessary recompilation of loaded packages.

CHANGELOG_BEGIN
CHANGELOG_END

* Track compiled definitions explicitly

Compile packages on load directly.

Use `PureCompiledPackages` extended with package for current line in
REPL service `runScript`.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-24 16:53:40 +00:00
Luciano
4610ed5b4f
Getting Started Guide: Update codegen command to SDK 0.13.55 (#5160)
* remove `src` subdir from codgen command

If you don't do this, running `yarn install` will complain:
```
error Couldn't find package "@daml-ts/create-daml-app-0.1.0@0.13.55" required by "create-daml-app@0.1.0" on the "npm" registry.
```
This was with yarn v1.22.4 - can someone please double-check this.

* Update codegen command and some surrounding explanation

Unfortunately also removed a bunch of trailing whitespace due to my
editor settings, which is fine but makes reviewing harder :(

changelog_begin
changelog_end

* Warn that workspaces run build takes ages

Co-authored-by: Rohan Jacob-Rao <rohanjr@gmail.com>
2020-03-24 16:23:06 +00:00
Stefano Baghino
a8a38a099e
Refactor conversion of pairs of offsets (#5159)
This way sharing code between new and old implementation should be easier.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 17:21:59 +01:00
Moritz Kiefer
a50b7b7dbd
Hide "Slf4jlogger started" log statement (#5163)
This just looks ugly and doesn’t help users at all.

changelog_begin
changelog_end
2020-03-24 17:06:14 +01:00
Samir Talwar
b815ab641d
Add the DAML-on-X team as code owners of /ledger/participant-state. (#5162)
* Add the DAML-on-X team as code owners of /ledger/participant-state.

Namely @dajmaki, @fabiotudone-da, @miklos-da, and @mziolekda.

CHANGELOG_BEGIN
CHANGELOG_END

* Add @dasormeter to the list of owners of /ledger/participant-state.

* DAML-on-X and Runtime own kvutils; just Runtime otherwise.

* Sort the owners of /ledger alphabetically.
2020-03-24 15:29:44 +00:00
Rohan Jacob-Rao
c950bf97d1
Guide to UI testing with Jest and Puppeteer (#5119)
* First draft of guide to UI testing

changelog_begin
changelog_end

* Address Nemanja's feedback

* Add copyright headers

* Cut down intro and description of Jest/Puppeteer

* Shorten a bit more
2020-03-24 11:17:41 -04:00
Remy
d834b9b029
Sandbox: make contract id seeding flag public (#5153)
* Sandbox: Reveal contract id seeding flag

CHANGELOG_BEGIN
- [Sandbox] Add support for random contract identifiers.  See section
  `Contract Identifiers Generation` section in
  docs/source/tools/sandbox.rst
CHANGELOG_END
2020-03-24 15:51:23 +01:00
Moritz Kiefer
1942efba23
Allow controlling navigator startup in daml.yaml (#5155)
* Allow controlling navigator startup in daml.yaml

While I’m not entirely convinced the default atm is right, making it
configurable seems like an easier solution than bikeshedding the default.

changelog_begin

- [DAML Assistant] You can now disable starting navigator as part of
  ``daml start`` in your ``daml.yaml`` file by adding ``start-navigator: false``.

changelog_end

* Update docs/source/tools/assistant.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Use --start-navigator=true in the docs

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-03-24 14:04:26 +01:00
Samir Talwar
e4c2df827c
kvutils: Move ProcessTransactionSubmission parameters into #run. (#5151)
* kvutils: Avoid global `inputState` in `ProcessTransactionSubmission`.

* kvutils: Avoid transaction variables in `ProcessTransactionSubmission`.

Globals, bad.

* kvutils: Simplify imports in `ProcessTransactionSubmission`.

* kvutils: Move `ProcessTransactionSubmission` parameters into `#run`.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Move `engine` to the ProcessTransactionSubmission constructor.
2020-03-24 12:59:53 +00:00
Gary Verhaegen
7d665d6163
fix tf config for GCP default (#5158)
It looks like GCP doesn't like not having a "page suffix" set, so it
sets a default. Except somehow Terraform doesn't know it's a default
value, so when trying to plan without the (optional) website value set,
Terraform will always find that the deployed state has changed.

With this change, we set it to a value that doesn't exist and won't
work, but at least Terraform will see that the deployed state matches
the configured one.

Note: this PR is a bit special as far as "changes" go as there will be
nothing to apply: applying current master tries to get rid of this
website.main_page_suffix value, but it's back on the next run. With this
patch, `terraform plan` declares "nothing to apply", so this PR itself
won't (need to) be applied.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 13:33:59 +01:00
Stefano Baghino
337a3a8269
Increase the time allowed for DB-backed reset service to complete tests (#5156)
The database-backed reset service can (understandably) go a bit slower than the one backed by the in-memory ledger.

This should help avoiding flaky tests.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 12:04:19 +00:00
Stefano Baghino
181df8f3ab
Act on Option2Iterable wart (#5154)
Some Option2Iterable ignore annotations are not needed, others were needed for unused methods.

In a few occasions we were ignoring the warning for the very purpose for which is was there,
i.e. avoiding an implicit conversion. I'm all for not verifying this rule if we agree we
don't need it.

For ProcessFailedException it was a bit gratuitous, I changed the way in which the exception
message is built.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 11:25:05 +00:00
Gary Verhaegen
4095538acf
match terraform with reality (#5143)
Our current Terraform setup attempts to create three static files on our
GCS buckets. The issue is that these buckets are configured to
automatically delete files that are older than X days, and there is no
way to exclude specific files from that. Therefore, the created files
disappear after some time, and running `terraform plan` suddenly looks
like the infrastructure has changed.

Moreover, the added value of these three files seems questionable: two
of them provide `index.html` type of functionality for our two caches,
whereas the third is automatically created by `nix` when pushing to the
cache anyway (if it doesn't exist already).

This PR also reduces the cache eviction time for the nix cache to 60
days, as a full year seemed a bit long.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 12:07:16 +01:00
Moritz Kiefer
2b47f6994d
Switch default DAML-LF target to 1.8 (#5127)
changelog_begin

- [DAML Compiler] The default output DAML-LF target version is now
  1.8. You can target 1.7 by specifying ``--target=1.7`` in the
  ``build-options`` field in your ``daml.yaml``.

changelog_end

Fix daml2ts tests
2020-03-24 11:02:13 +01:00
Moritz Kiefer
5cb7fc7c19
Disable bifunctor dlint rules (#5149)
changelog_begin
changelog_end
2020-03-24 10:43:05 +01:00
Stefano Baghino
37b7d855b8
Add test to ensure that the reset truncates all tables (#5142)
* Add test to ensure that the reset truncates all tables

The test can be adjusted over time to accomodate for exceptions (which are already there).

Unfortunataly we have to add a new couple of queries to support both Postgres and H2.

Fixes #5130

CHANGELOG_BEGIN
CHANGELOG_END

* Make loose check on configuration_entries
2020-03-24 09:18:14 +00:00
Remy
8fb86b7133
Engine: create type for immutable byte array (#5140)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 09:04:52 +01:00
Samir Talwar
22aa642a9f
sandbox: Add timers for all read, write, and index service methods. (#5147)
CHANGELOG_BEGIN
- [Sandbox] Added metrics under "daml.services" for various internal
  events.
CHANGELOG_END
2020-03-24 07:03:33 +00:00
Moritz Kiefer
ed67e5c188
Bump portfile retries (#5134)
We have seen a few test failures where the port file was not written
to in time. Apparently this can takes ages on CI for tests that are
not marked exclusive.

changelog_begin
changelog_end
2020-03-23 19:20:33 +01:00
Oliver Seeliger
3df5a6ec5b
Upgrade canton to 0.11.0 (#5144)
in turn based on 13.56-snapshot-20200318

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 18:06:38 +00:00
Shayne Fletcher
a5faa1cdbc
Use local paths in package dependencies (#5137)
changelog_begin
changelog_end
2020-03-23 13:54:56 -04:00
Remy
2dd9c50631
DAML-LF: contract id spec (#5106)
* DAML-LF: contract id spec

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 18:05:48 +01:00
Samir Talwar
e853012626
kvutils/app: Share the metrics registry between indexer and API server. (#5139)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 15:31:21 +00:00
Robert Autenrieth
c477b63a04
Store initial ledger configuration (#5126)
This is required by the "initialize before transaction acceptance"
invariant of the ReadService interface.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 15:39:13 +01:00
Stephen Compall
fb42845959
simplify websocket notes, remove an apparent contradiction (#5116)
- from a discussion with @meiersi-da and @hurryabit on #3878

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 09:59:49 -04:00
Moritz Kiefer
bef129605f
Fix fmt.sh (#5135)
grep returns with an exit code of 1 if there are no matches.

changelog_begin
changelog_end
2020-03-23 13:27:03 +00:00
Stefano Baghino
c26effb235
Expose both JDBC URL and database type in tests, centralize initialization (#5133)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 12:17:24 +00:00
Moritz Kiefer
aa48f3026e
Bump ghcide (#5128)
This PR bumps ghcide, haskell-lsp and haskell-lsp-types. There aren’t
really any important changes in ghcide itself but the haskell-lsp
update includes my fix for crashing completions.

One change in ghcide itself is that NormalizedFilePath got moved to
haskell-lsp. ghcide needs special treatment for empty file paths so we
use `toNormalizedFilePath'` from ghcide instead of
`toNormalizedFilePath` from `haskell-lsp`. I’ve added an hlint rule to
enforce this.

changelog_begin
changelog_end
2020-03-23 12:09:36 +00:00
Stefano Baghino
ceeb7c3487
Re-use AkkaBeforeAndAfterAll in AbstractSandboxFixture (#5132)
* Re-use AkkaBeforeAndAfterAll in AbstractSandboxFixture

CHANGELOG_BEGIN
CHANGELOG_END

* Do not expose the Akka execution context directly
2020-03-23 11:56:47 +00:00
Samir Talwar
3d6ecc13b6
Sandbox: Configurable metrics output. (#5113)
* sandbox: Clean up `MetricsReporting` a little.

Make sure it closes both reporters, and avoid starting things in a
constructor.

* sandbox: Add hidden options for enable metrics reporting.

* sandbox: Add a disambiguating name to the DB connection/thread pools.

CHANGELOG_BEGIN
- [Sandbox] DB connection pool metrics names have changed slightly, from
  ``daml.index.db.connection`` to ``daml.index.db.connection.sandbox``.
- [Ledger Integration Kit] DB connection pool metrics names have changed
  to disambiguate the StandaloneApiServer from the
  StandaloneIndexerServer. The former now has a ``.ledger-api-server``
  suffix, and the latter now has a ``.indexer`` suffix.
CHANGELOG_END

* sandbox-next: Use the same metrics registry for the API and indexer.

* sandbox: Give a useful error message on an invalid metrics reporter.

And simplify the error messages.

With the arguments `--client-auth=foo --metrics-reporter=foo`, we now
get the output:

```
Error: Option --client-auth failed when given 'foo'. Must be one of
  "none", "optional", or "require".
Error: Option --metrics-reporter failed when given 'foo'. Must be one of
  "console", or "csv:PATH".
Try --help for more information.
```

* sandbox: Pull out more helpers in `MetricsReporting`.

* sandbox: Rename MetricsReporter classes so they don't clash.

* sandbox: Wrap the `name` parameter in a `ServerName` tagged string.

For safety. Yours, not mine.

* sandbox: Push metrics to Graphite with `--metrics-reporter=graphite`.

* sandbox: Make `MetricsReporter.Graphite` singly-lazy, not doubly-.

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* sandbox: Replace `ServerName` with `ServerRole`.

* sandbox: Fix usage of `ServerRole.Testing` in `LedgerResource`.

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-03-23 10:59:13 +00:00
Stefano Baghino
c00af40fe5
Cleanup: InfiniteRetries is used only in sandox-perf (#5131)
* Cleanup: InfiniteRetries is used only in sandox-perf

CHANGELOG_BEGIN
CHANGELOG_END

* Fix import
2020-03-23 10:49:11 +00:00
Stefano Baghino
97ec70e508
Add missing truncations from reset service and fix perf tests (#5129)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 10:36:32 +00:00
Gary Verhaegen
95d9cdb567
faster hlint with fmt.sh --diff (#5114)
This PR adds new behaviour to the existing --scalafmt-diff flag to also
use a diff-based approach for hlint. Consequently, the flag is renamed
to the simpler and more generic `--diff`, in a backwards-compatible way.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 11:22:02 +01:00
Gerolf Seitz
f9fb888cc4
Allow for predictable contractIDs and transactionIDs (#5112)
- New seeding type "Static" that uses a fixed seed for generating new seeds.
This is only used in Sandbox and Sandbox-next.

- Remove the fuzzing for submission time in Engine.scala

- DAML-on-SQL: Create new log entry IDs from a provided SeedService.
This allows for generating deterministic transaction IDs in
Sandbox-Next.

Fixes #5107

CHANGELOG_BEGIN
[Sandbox] Add contract-id-seeding=static to allow for predictable contract IDs. This is useful for documentation,
to be able to refer to a specific contract ID instead of having to write "note down the contract ID you see on the screen. we will use it later."
[DAML-on-SQL] Derive the next log entry ID using the provided SeedService. This allows us to also deterministically create transactionIds in static time mode together with `--contract-id-seeding=static`. This should only be used for demos or documentation.
CHANGELOG_END
2020-03-23 10:55:40 +01:00
Samir Talwar
3e95bb7bec
Sandbox: Simplify the ApiCommandService a little. (#5125)
* sandbox: Remove a few levels of indentation in `ApiCommandService`.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Get rid of the `LowLevelCommandServiceAccess` trait.

It only has one implementation. Just use it.

* sandbox: Remove unused type aliases from ApiCommandService.
2020-03-23 08:55:25 +00:00
Samir Talwar
28c78e5978
sandbox | kvutils/app: Re-use the Materializer in the JdbcIndexer. (#5124)
* kvutils/app|sandbox: Rename `system` to `actorSystem`.

* sandbox: Pass a materializer in to the StandaloneIndexerServer.

There's no need for it to construct one when the caller always has one
available.

CHANGELOG_BEGIN
CHANGELOG_END

* recovering-indexer: Use `SubmissionId` instead of `LedgerString`.

Where appropriate.

* sandbox: Use the materializer implicitly in JdbcIndexer.
2020-03-23 08:55:19 +00:00
Oliver Seeliger
45acfec218
Ledger api server indexer initializes proper ledger_end after restart (#5122)
by reading parameters.ledger_end column instead of parameters.external_ledger_end.

CHANGELOG_BEGIN
[Ledger API Server]: Upon restart, ledger api server continues consuming unconsumed events rather than
all events from beginning.
CHANGELOG_END

Closes #5121
2020-03-23 09:53:48 +01:00
Gerolf Seitz
e82b8d743b
Add missing truncate for participant_events (#5117)
The missing truncate caused "Duplicate Submissions" after a reset in
between the performance tests.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 09:06:02 +01:00
Shayne Fletcher
2525a4610a
Update hlint (#5123)
changelog_begin
changelog_end
2020-03-22 16:30:11 +00:00
Shayne Fletcher
bf22d8db59
Fix package name (#5120)
changelog_begin
changelog_end
2020-03-22 12:01:56 -04:00
Stephen Compall
f2b068f0b5
remove List and TextMap support from the JSON query language (#5099)
* remove List and TextMap support from the JSON query language

CHANGELOG_BEGIN
- [JSON API - Experimental] Removed support for maps and lists from the query language.
  See `issue #4855 <https://github.com/digital-asset/daml/issues/4855>`_.
CHANGELOG_END

* GenMap won't be supported in general, remove FIXME

- thanks @leo-da

* more documentation on unsupported query types

- as suggested by @hurryabit; thanks
2020-03-20 16:22:22 -04:00
Andreas Herrmann
4f7c52c981
DAML script runner takes CompiledPackages (#5110)
* daml-script runner from compiledPackages

* Factor out script parameter conversion

* Use ADT for DAML script (Action & Function)

* run --> runwithClients

* DAMl script Runner `run` convenience function

* REPL use pure script runner constructor

* DAML script runner holds script itself

* scalafmt

* comments

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-20 17:52:54 +00:00
Stefano Baghino
f67c32b002
Integrate transaction lookup on new schema in Ledger API (#5104)
* Integrate transaction lookup on new schema in Ledger API

Re-wires all transaction lookups to the new schema

CHANGELOG_BEGIN
CHANGELOG_END

* Always return the agreement text

CHANGELOG_BEGIN
[Ledger API Server] The metric 'daml.index.lookup_transaction' has been
replaced by 'daml.index.lookup_flat_transaction_by_id' and
'daml.index.lookup_transaction_tree_by_id', which record the same events
but with more granularity regarding the type of lookup.
CHANGELOG_END

* Ensure agreement text invariant in a single place

* Do not compare the order in which witness parties appear in an event

* Hide command identifier from non-submitters in transaction trees

* Fix time assigned to transaction to be the ledger effective time and not the record time

* Store transactions from initial state into the new schema
2020-03-20 17:22:42 +00:00
Leonid Shlyapnikov
ffabc2b3e5
Bumping up http-json tests size to large, adding some extra logging (#5109)
* Bumping up http-json tests size to large, adding some extra logging

* Removing the logging from prod code, adding some in tests.

changelog_begin
changelog_end
2020-03-20 16:48:11 +00:00
Moritz Kiefer
be7f0802c6
Bump ghcide (#5103)
This includes some completion improvements.

changelog_begin
changelog_end
2020-03-20 16:43:59 +00:00
Samir Talwar
1ffd23a6a2
Sandbox: Make JdbcLedgerDao create its own execution context. (#5091)
* sandbox: Don't let just anyone construct a DbDispatcher.

It's the job of the JdbcLedgerDao.

* sandbox: Clean up the DbDispatcher a little.

* sandbox: JdbcLedgerDao now creates its own execution context.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Make `defaultNumberOfShortLivedConnections` private.
2020-03-20 15:30:35 +00:00
Moritz Kiefer
18033271de
Remove bzip2 library (#5105)
This used to be a dependency of our zip library that you couldn’t
disable. However, in the meantime it has gotten a flag to disable
it and we don’t actually use it. Given that the upstream URL is dead
atm now seems like a good time to kill it.

changelog_begin
changelog_end
2020-03-20 15:05:33 +01:00
Gary Verhaegen
b45c7aed69
fix 0.13.55 release notes typo (#5074)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-20 13:25:23 +00:00
Stefano Baghino
febace7391
Add lookup for transaction tree by id (#5089)
* Add lookup for transaction tree by id

CHANGELOG_BEGIN
CHANGELOG_END

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395156203

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395163051

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395165561

* Address multiple comments

- https://github.com/digital-asset/daml/pull/5089#discussion_r395185084
- https://github.com/digital-asset/daml/pull/5089#discussion_r395185290
- https://github.com/digital-asset/daml/pull/5089#discussion_r395185461

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395157643

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395160343

* Address https://github.com/digital-asset/daml/pull/5089#discussion_r395482878

* Use refactored testing utilities

* Add test for partial transaction visibility
2020-03-20 13:28:11 +01:00