Commit Graph

3632 Commits

Author SHA1 Message Date
Martin Huschenbett
b1913e6d38
TS libs: Remove deprecated methods/hooks (#5168)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 20:05:27 +01:00
Samir Talwar
93e9dd9663
CODEOWNERS: Add owners for the LATEST file, used to manage releases. (#5191)
Namely @bame-da, @gerolf-da, and @hurryabit.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 20:05:11 +01:00
Shayne Fletcher
9351b7a7b9
Build js packages at code-generation time (#5171)
changelog_begin
changelog_end
2020-03-25 14:13:37 -04:00
Stefano Baghino
1a229ebb7b
Refactor response preparation for flat transactions (#5187)
* Refactor response preparation for flat transactions

This will allow to share code across endpoints

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/5187#discussion_r397996322
2020-03-25 17:53:25 +00:00
Samir Talwar
ef6c09b1bd
Release: 2020-03-25. (#5188)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 18:15:29 +01:00
Stefano Baghino
848cbcc418
Add result set parser as vector (#5181)
* Add result set parser as vector

Use this to not require a linear scan of the results in PaginatingAsyncStream.

Since the offset computation is now more lightweight, it should be safe to run it on the parasitic execution context.

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/5181#discussion_r397818211

* Address https://github.com/digital-asset/daml/pull/5181#discussion_r397900841

* Address https://github.com/digital-asset/daml/pull/5181#discussion_r397899395

* Optimize imports
2020-03-25 16:15:16 +00:00
Robert Autenrieth
a3ddde3a9d
Remove MRT and LET (#5172)
Contributes to #4194.
Closes #4231.
Closes #5022.
CHANGELOG_BEGIN
- [Ledger API] The protobuf fields ledger_effective_time and maximum_record_time have been removed from
  command submission. These fields were previously deprecated following the introduction
  of a new ledger time model. See issue `#4194 <https://github.com/digital-asset/daml/issues/4194>`__.
[Java Bindings] removed the usage of ledgerEffectiveTime and
maximumRecordTime, and instead added minLedgerTimeAbsolute and
minLedgerTimeRelative in CommandSubmissionClient and CommandClient
CHANGELOG_END
2020-03-25 16:50:27 +01:00
Samir Talwar
69220c3f58
kvutils/app: Add timing metrics for read/write/index services. (#5176)
* participant-state{,-index}: Move Timed*Service classes from Sandbox.

CHANGELOG_BEGIN
- [Ledger Integration Kit] Metrics for the various read, write, and index
  services.
CHANGELOG_END

* kvutils/app: Add timing metrics for read/write/index services.

* participant-state: Move metrics-related code to another Bazel package.

* participant-state-metrics: Add to artifacts.yml.

* participant-state-metrics: Move TimedIndexService back into Sandbox.

Cuts down on dependencies like nobody's business.
2020-03-25 15:11:37 +00:00
Samir Talwar
d7d45d1860
sandbox: Set the Netty channel type to fix a warning. (#5179)
Tests would often display this warning:

    io.grpc.netty.NettyChannelBuilder buildTransportFactory
    WARNING: Both EventLoopGroup and ChannelType should be provided or neither should be, otherwise client may not start. Not provided values will use Nio (NioSocketChannel, NioEventLoopGroup) for compatibility. This will cause an Exception in the future.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 14:29:20 +00:00
Stephen Compall
8d2c13cf43
generating GenMaps for testing (#5115)
* building a GenMap generator

* move Party Order to accessible place; test whether Name Equal is still in use

* add Order instance for SortedLookupList

* switch to Map for genMap's Inj; more Order instances

* remove all Order from TypedValueGenerators

* Revert "add Order instance for SortedLookupList"

This reverts commit 03a59a8249.

* moving the Equal instance means scenario-interpreter no longer direct-deps scalaz

* add a test using TypedValueGenerators GenMap

* remove Party Order

* refmt bazel

* remove stray import

* followup dep change to moving the Equal[Name] instance

* add changelog

CHANGELOG_BEGIN
CHANGELOG_END

* Name equal instance appears to be no longer used, but keeping anyway
2020-03-25 10:28:06 -04:00
Moritz Kiefer
46be910ba1
Abstract over ledger interaction in DAML script (#5184)
This PR adds as `ScriptLedgerClient` trait (happy to change the name
if anyone has a better proposal) that abstracts over the interaction
with the ledger. This will allow us to plug in a different
implementation for interacting with the JSON API so we can run DAML
scripts against DABL or other environments where gRPC is not a
workable option. Note that this PR does not yet add the implementation
for interacting with the JSON API. I’ll leave that for a separate PR.

changelog_begin
changelog_end
2020-03-25 15:26:18 +01:00
Stephen Compall
f4525356f6
stray an->a from renaming "add" to "created" (#5170)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 12:42:36 +00:00
Stefano Baghino
5a6ca00567
Document PaginatingAsyncStream (#5177)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 12:26:49 +00:00
Samir Talwar
a8909811dd
sandbox: Add eventuallys to MetricsInterceptorSpec. (#5178)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 11:35:14 +00:00
Samir Talwar
bfe27d23db
sandbox: Capture timing metrics for API server calls. (#5145)
* sandbox: Capture timing metrics for API server calls.

`timer` is a superset of `meter`, so this doesn't lose any existing
behavior; just adds new behavior.

CHANGELOG_BEGIN
- [Ledger API Server] Added timing metrics for all GRPC endpoints.
CHANGELOG_END

* sandbox: Rename SandboxClientResource to GrpcClientResource.

* sample-service: Clean up warnings.

* sandbox: Add tests for MetricsInterceptor.

* sandbox: Split the API metrics interceptor from the naming.

* sandbox: Use `MetricRegistry.name` instead of string interpolation.

* rs-grpc-akka: Restrict the test library to the DAML workspace.

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-03-25 09:56:37 +00:00
Oliver Seeliger
2c0627c87d
Flake fix of TransactionService.TXTreeBlinding for multi-node (#5175)
Added a handful of eventually wrappers.

Closes #5174

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 10:45:29 +01:00
Robert Autenrieth
3f597aae16
New ledger time (#5100)
* Tighten result type

Command execution can't result in a sequencer error

* New helper method for extracting used contracts

* New error clause

* Add a DAO query for the maximum time of contracts

* Implement algorithm for finding ledger time

CHANGELOG_BEGIN
CHANGELOG_END

* fixup ledgerTimeHelper

* Use new ledger time algorithm

* Mark LET/MRT as deprecated

CHANGELOG_BEGIN
- [Ledger API] DAML ledgers have switched to a new ledger time model.
  The ledger_effective_time and maximum_record_time fields of command submission are deprecated,
  the ledger time of transactions is instead set automatically by the ledger API server.
  Ledger time is no longer strictly monotonically increasing, but only follows causal monotonicity:
  ledger time of transactions is greater than or equal to the ledger time of any used contract.
  See `#4345 <https://github.com/digital-asset/daml/issues/4345>`__.
CHANGELOG_END

* Add ledger time skew check

* Remove command updater

LET/MRT are now deprecated, this class is now useless

* Remove old time model validator

* Switch to new time model check: kvutils

* Switch to new time model check: in-memory ledger

* Switch to new time model check: SqlLedger

* Use initial ledger config

* Ignore user provided LET

* Use TimeProvider in submission services

* Use deduplication_time in daml-script runner

- Also remove unnecessary command completion output of CommandTracker.
- Remove usage of maximum record time in CommandTracker.

* Use arbitrary default value for deduplication time

* Use built-in Instant ordering

* Remove obsolete test

* Remove obsolete test: CommandStaticTimeIT

* Refactor test: TransactionMRTCompliance

* Disable test: CommandTrackerFlow timeout

* thread maxDeduplicationTime through to CommandTracker

* Improve test

* Refactor command client configuration

* Deduplication time should always use UTC

* Add missing method in TimedIndexService after rebase

* Put more details into the deduplication error response.

* Use system time for command dedup submittedAt.

* Use explicit UTC time source in command validator

* Revert CommandTracker[Flow] to previous completion-recovering-behavior

* Adapt scala client command config to new config params

Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-03-25 09:28:56 +01:00
Martin Huschenbett
f8e383daf6
daml2ts: Consolidate ESLint exception in generated code (#5167)
Currently, we write exceptions for using namespaces and all exceptions in
`index.ts` at the place where they occur. This is pretty noisy.

After this PR, we write all exceptions at the beginning of the file.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 22:19:00 +00:00
Samir Talwar
2887f28cc1
kvutils: Inject the metrics registry instead of using SharedMetricRegistries. (#5161)
* kvutils: Remove an unnecessary `@SuppressWarnings`.

* kvutils: Reduce the scope of fields and methods in `Committer`.

* kvutils: Inject the metric registry into `KeyValueCommitting`.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Inject the metric registry into the committers.

* kvutils: Inject the metric registry into `ProcessTransactionSubmission`.

* kvutils: Avoid shared metric registries in tests.

* kvutils: Recreate the metrics registry per participant state.

* kvutils: Add trailing commas to parameter lists.

Flagrantly encouraged by @stefanobaghino-da.

* recovering-indexer: Don't re-use the metric registry in tests.
2020-03-24 19:41:19 +00:00
Leonid Shlyapnikov
a6a1e75bc1
Streaming API error handling/reporting improvements (#5141)
* Streaming API error handling

* Stopping the stream after emitting the 1st error

* Cleaning up test cases, stricter type for non empty streaming fetch

* Update docs.

changelog_begin

[JSON API - Experimental] Change Streaming API error format to match synchronous API:
``{"status": <400 | 401 | 404 | 500>, "errors": <JSON array of strings> }``. See #4521

changelog_end

* a few words about warnings in the Streaming API

* a few words about warnings in the Streaming API

* Updating command formatting in the JavaScript example.
2020-03-24 19:32:55 +00:00
Samir Talwar
a56b3fd868
rs-grpc-bridge: Split the test library from the tests, and actually run the tests. (#5166)
* rs-grpc-bridge: Separate the test library from tests.

I don't want _logback-test.xml_ shipping with the
TestExecutionSequencerFactory.

* rs-grpc-bridge: Actually run the tests.

How we managed to miss this, I don't know.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 19:29:35 +00:00
Moritz Kiefer
37dc2f29d3
Integrate create-daml-app into the assistant (#5152)
* Integrate create-daml-app into the assistant

fixes #4868

changelog_begin

- [DAML Assistant] Add a new ``daml create-daml-app`` command for creating a project based on
  `create-daml-app <https://github.com/digital-asset/create-daml-app>`_.

changelog_end

* Try random things hoping to fix windows

* Try random things hoping to fix things on macos
2020-03-24 18:29:40 +01:00
Andreas Herrmann
3b550b9933
Avoid recompiling loaded packages in REPL (#5157)
* Use MutableCompiledPackages in REPL

This avoids unnecessary recompilation of loaded packages.

CHANGELOG_BEGIN
CHANGELOG_END

* Track compiled definitions explicitly

Compile packages on load directly.

Use `PureCompiledPackages` extended with package for current line in
REPL service `runScript`.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-24 16:53:40 +00:00
Luciano
4610ed5b4f
Getting Started Guide: Update codegen command to SDK 0.13.55 (#5160)
* remove `src` subdir from codgen command

If you don't do this, running `yarn install` will complain:
```
error Couldn't find package "@daml-ts/create-daml-app-0.1.0@0.13.55" required by "create-daml-app@0.1.0" on the "npm" registry.
```
This was with yarn v1.22.4 - can someone please double-check this.

* Update codegen command and some surrounding explanation

Unfortunately also removed a bunch of trailing whitespace due to my
editor settings, which is fine but makes reviewing harder :(

changelog_begin
changelog_end

* Warn that workspaces run build takes ages

Co-authored-by: Rohan Jacob-Rao <rohanjr@gmail.com>
2020-03-24 16:23:06 +00:00
Stefano Baghino
a8a38a099e
Refactor conversion of pairs of offsets (#5159)
This way sharing code between new and old implementation should be easier.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 17:21:59 +01:00
Moritz Kiefer
a50b7b7dbd
Hide "Slf4jlogger started" log statement (#5163)
This just looks ugly and doesn’t help users at all.

changelog_begin
changelog_end
2020-03-24 17:06:14 +01:00
Samir Talwar
b815ab641d
Add the DAML-on-X team as code owners of /ledger/participant-state. (#5162)
* Add the DAML-on-X team as code owners of /ledger/participant-state.

Namely @dajmaki, @fabiotudone-da, @miklos-da, and @mziolekda.

CHANGELOG_BEGIN
CHANGELOG_END

* Add @dasormeter to the list of owners of /ledger/participant-state.

* DAML-on-X and Runtime own kvutils; just Runtime otherwise.

* Sort the owners of /ledger alphabetically.
2020-03-24 15:29:44 +00:00
Rohan Jacob-Rao
c950bf97d1
Guide to UI testing with Jest and Puppeteer (#5119)
* First draft of guide to UI testing

changelog_begin
changelog_end

* Address Nemanja's feedback

* Add copyright headers

* Cut down intro and description of Jest/Puppeteer

* Shorten a bit more
2020-03-24 11:17:41 -04:00
Remy
d834b9b029
Sandbox: make contract id seeding flag public (#5153)
* Sandbox: Reveal contract id seeding flag

CHANGELOG_BEGIN
- [Sandbox] Add support for random contract identifiers.  See section
  `Contract Identifiers Generation` section in
  docs/source/tools/sandbox.rst
CHANGELOG_END
2020-03-24 15:51:23 +01:00
Moritz Kiefer
1942efba23
Allow controlling navigator startup in daml.yaml (#5155)
* Allow controlling navigator startup in daml.yaml

While I’m not entirely convinced the default atm is right, making it
configurable seems like an easier solution than bikeshedding the default.

changelog_begin

- [DAML Assistant] You can now disable starting navigator as part of
  ``daml start`` in your ``daml.yaml`` file by adding ``start-navigator: false``.

changelog_end

* Update docs/source/tools/assistant.rst

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Use --start-navigator=true in the docs

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-03-24 14:04:26 +01:00
Samir Talwar
e4c2df827c
kvutils: Move ProcessTransactionSubmission parameters into #run. (#5151)
* kvutils: Avoid global `inputState` in `ProcessTransactionSubmission`.

* kvutils: Avoid transaction variables in `ProcessTransactionSubmission`.

Globals, bad.

* kvutils: Simplify imports in `ProcessTransactionSubmission`.

* kvutils: Move `ProcessTransactionSubmission` parameters into `#run`.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Move `engine` to the ProcessTransactionSubmission constructor.
2020-03-24 12:59:53 +00:00
Gary Verhaegen
7d665d6163
fix tf config for GCP default (#5158)
It looks like GCP doesn't like not having a "page suffix" set, so it
sets a default. Except somehow Terraform doesn't know it's a default
value, so when trying to plan without the (optional) website value set,
Terraform will always find that the deployed state has changed.

With this change, we set it to a value that doesn't exist and won't
work, but at least Terraform will see that the deployed state matches
the configured one.

Note: this PR is a bit special as far as "changes" go as there will be
nothing to apply: applying current master tries to get rid of this
website.main_page_suffix value, but it's back on the next run. With this
patch, `terraform plan` declares "nothing to apply", so this PR itself
won't (need to) be applied.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 13:33:59 +01:00
Stefano Baghino
337a3a8269
Increase the time allowed for DB-backed reset service to complete tests (#5156)
The database-backed reset service can (understandably) go a bit slower than the one backed by the in-memory ledger.

This should help avoiding flaky tests.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 12:04:19 +00:00
Stefano Baghino
181df8f3ab
Act on Option2Iterable wart (#5154)
Some Option2Iterable ignore annotations are not needed, others were needed for unused methods.

In a few occasions we were ignoring the warning for the very purpose for which is was there,
i.e. avoiding an implicit conversion. I'm all for not verifying this rule if we agree we
don't need it.

For ProcessFailedException it was a bit gratuitous, I changed the way in which the exception
message is built.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 11:25:05 +00:00
Gary Verhaegen
4095538acf
match terraform with reality (#5143)
Our current Terraform setup attempts to create three static files on our
GCS buckets. The issue is that these buckets are configured to
automatically delete files that are older than X days, and there is no
way to exclude specific files from that. Therefore, the created files
disappear after some time, and running `terraform plan` suddenly looks
like the infrastructure has changed.

Moreover, the added value of these three files seems questionable: two
of them provide `index.html` type of functionality for our two caches,
whereas the third is automatically created by `nix` when pushing to the
cache anyway (if it doesn't exist already).

This PR also reduces the cache eviction time for the nix cache to 60
days, as a full year seemed a bit long.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 12:07:16 +01:00
Moritz Kiefer
2b47f6994d
Switch default DAML-LF target to 1.8 (#5127)
changelog_begin

- [DAML Compiler] The default output DAML-LF target version is now
  1.8. You can target 1.7 by specifying ``--target=1.7`` in the
  ``build-options`` field in your ``daml.yaml``.

changelog_end

Fix daml2ts tests
2020-03-24 11:02:13 +01:00
Moritz Kiefer
5cb7fc7c19
Disable bifunctor dlint rules (#5149)
changelog_begin
changelog_end
2020-03-24 10:43:05 +01:00
Stefano Baghino
37b7d855b8
Add test to ensure that the reset truncates all tables (#5142)
* Add test to ensure that the reset truncates all tables

The test can be adjusted over time to accomodate for exceptions (which are already there).

Unfortunataly we have to add a new couple of queries to support both Postgres and H2.

Fixes #5130

CHANGELOG_BEGIN
CHANGELOG_END

* Make loose check on configuration_entries
2020-03-24 09:18:14 +00:00
Remy
8fb86b7133
Engine: create type for immutable byte array (#5140)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 09:04:52 +01:00
Samir Talwar
22aa642a9f
sandbox: Add timers for all read, write, and index service methods. (#5147)
CHANGELOG_BEGIN
- [Sandbox] Added metrics under "daml.services" for various internal
  events.
CHANGELOG_END
2020-03-24 07:03:33 +00:00
Moritz Kiefer
ed67e5c188
Bump portfile retries (#5134)
We have seen a few test failures where the port file was not written
to in time. Apparently this can takes ages on CI for tests that are
not marked exclusive.

changelog_begin
changelog_end
2020-03-23 19:20:33 +01:00
Oliver Seeliger
3df5a6ec5b
Upgrade canton to 0.11.0 (#5144)
in turn based on 13.56-snapshot-20200318

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 18:06:38 +00:00
Shayne Fletcher
a5faa1cdbc
Use local paths in package dependencies (#5137)
changelog_begin
changelog_end
2020-03-23 13:54:56 -04:00
Remy
2dd9c50631
DAML-LF: contract id spec (#5106)
* DAML-LF: contract id spec

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 18:05:48 +01:00
Samir Talwar
e853012626
kvutils/app: Share the metrics registry between indexer and API server. (#5139)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 15:31:21 +00:00
Robert Autenrieth
c477b63a04
Store initial ledger configuration (#5126)
This is required by the "initialize before transaction acceptance"
invariant of the ReadService interface.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 15:39:13 +01:00
Stephen Compall
fb42845959
simplify websocket notes, remove an apparent contradiction (#5116)
- from a discussion with @meiersi-da and @hurryabit on #3878

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 09:59:49 -04:00
Moritz Kiefer
bef129605f
Fix fmt.sh (#5135)
grep returns with an exit code of 1 if there are no matches.

changelog_begin
changelog_end
2020-03-23 13:27:03 +00:00
Stefano Baghino
c26effb235
Expose both JDBC URL and database type in tests, centralize initialization (#5133)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-23 12:17:24 +00:00
Moritz Kiefer
aa48f3026e
Bump ghcide (#5128)
This PR bumps ghcide, haskell-lsp and haskell-lsp-types. There aren’t
really any important changes in ghcide itself but the haskell-lsp
update includes my fix for crashing completions.

One change in ghcide itself is that NormalizedFilePath got moved to
haskell-lsp. ghcide needs special treatment for empty file paths so we
use `toNormalizedFilePath'` from ghcide instead of
`toNormalizedFilePath` from `haskell-lsp`. I’ve added an hlint rule to
enforce this.

changelog_begin
changelog_end
2020-03-23 12:09:36 +00:00