Commit Graph

5312 Commits

Author SHA1 Message Date
Remy
f0372979be
Speedy: shortcup failure in SValue ordering. (#7329)
* LF: shortcup failure in SValue ordering.

Before this PR comparison of lists, maps, ans structural record had a
always linear complexity.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-08 16:13:40 +02:00
Moritz Kiefer
d086738ac0
Migrate DAML Intro to DAML Script (#7264)
changelog_begin
changelog_end
2020-09-08 14:28:30 +02:00
Moritz Kiefer
fea21e33a6
Migrate leftover scenario in quickstart to DAML Script (#7338)
We already migrated everything in the Tests directory but forgot about
this one.

changelog_begin
changelog_end
2020-09-08 13:15:45 +02:00
azure-pipelines[bot]
8b942687f8
update compat versions for 1.5.0-snapshot.20200907.5151.0.eb68e680 (#7337)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-09-08 12:56:35 +02:00
Robin Krom
678a8eef71
Replace yarn with npm (#7222)
* replace yarn with npm in docs

CHANGELOG_BEGIN
CHANGELOG_END

* updating assistant and compatibility tests

* moved gitignore entry to toplevel
2020-09-08 12:07:04 +02:00
Robin Krom
b86d213ecb
moved robin down in release rotation because of holidays. (#7339)
CHANGELOG_BEGIN
CHANGELOG_END
2020-09-08 09:06:26 +00:00
Gerolf Seitz
86608c2d98
Update CODEOWNERS (#7336)
reflect recent org changes

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-07 15:50:40 +00:00
Moritz Kiefer
82c4971d91
Release a new snapshot (#7335)
Includes a bugfix for data-dependencies which we want in 1.5.0.

changelog_begin
changelog_end
2020-09-07 16:56:34 +02:00
Remy
8aa34c701c
Speedy: use standard scala equality to check SBuiltinTests result. (#7333)
This reverts changed made in #3416, where the standard scala was
replace by ad-hoc Speedy equality for SValue. As Speedy equality
become more permisive (e.g. it does not check for type constructor of
records, variants, and enumrations or constructor of variants or
enumerations) this is safer to use standard scala eqality for case
class.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-07 16:33:01 +02:00
Robin Krom
3728106958
create-daml-app: make sure errors are displayed (#7332)
When an error happend during decoding of JSON API answers due to missing
templates, the error was just displayed as an empty object '{}'.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-07 16:21:03 +02:00
Samir Talwar
74e7086662
ledger-api-test-tool: Only create sessions once. (#7328)
* ledger-api-test-tool: Wait for futures when constructing LedgerSession.

* ledger-api-test-tool: Wait for futures on `new ParticipantSession`.

* ledger-api-test-tool: If we create two sessions, close one immediately.

* ledger-api-test-tool: Only create sessions once.

CHANGELOG_BEGIN
- [Ledger API Test Tool] Fix a race condition in which multiple
  connections were created to a single participant, and only one was
  shut down properly. This error was likely benign but may cause
  spurious errors to show up when the test tool finishes and shuts down.
CHANGELOG_END

* ledger-api-test-tool: Let ParticipantSessionManager manage connections.

* ledger-api-test-tool: Shrink the configuration surface area.

* ledger-api-test-tool: Use `Future.traverse` to simplify LedgerSession.

* ledger-api-test-tool: Fix the ParticipantSessionManager logger.

* ledger-api-test-tool: Address issues flagged by @stefanobaghino-da.

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* ledger-api-test-tool: Make `SessionParts` a real class.

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-09-07 14:15:52 +00:00
Samir Talwar
eb68e680f6
kvutils: Change the export header from "v3" to "v2". (#7330)
The old v2 is no more. Let's pretend it never happened.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-07 13:52:09 +00:00
Martin Huschenbett
d7f0bc1c7b
DAML-LF spec: Fix two minor issues around structs (#7323)
* DAML-LF spec: Fix two minor issues around structs

We make it explicit that all field names in a structural record need to
be distince. We also fix a small typo in the indices for the
`ExpStructCon` typing rule.

CHANGELOG_BEGIN
CHANGELOG_END

* Address comments regarding distinctness

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-07 15:13:13 +02:00
Remy
8968889a31
LF: drop buggy/useless check in SBCheckPrecond (#7331)
CHANGELOG_BEGIN
CHANGELOG_END
2020-09-07 12:31:27 +02:00
Brian Healey
e2bff87a0c
Provide configurable clock interval to cater for ledger differences with deduplication tests (#7294)
* Extend wait time for dedupe tests to 2x dedupe window

* default clock tick interval of 10s, make it optionally configurable

CHANGELOG_BEGIN
[ledger-api-test-tool] Include new attribute for configurable clock tick
interval
CHANGELOG_END

Signed-off-by: Brian Healey <brian.healey@digitalasset.com>

* address review comments

* Address code review - use Duration

* Add documentation on --ledger-clock-granularity option

* Update documentation

* address review comments

* address warning: implicit numeric widening

* scalafmt

* ProtobufDuration alias rather than ProtoDuration for consistency with namespacing

* underline in docs
2020-09-04 17:19:07 -04:00
Remy
fbeae8ae5e
LF: Use the new Struct data structure for SStruct (#7319)
This PR uses the new data structure introduced in #7220.

Additionally this fixes `svalue.Equality` and
`svalue.Ordering` which were considering <a: x, b: y>
different from <b:y, a:x>.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-04 22:04:07 +02:00
Remy
78e770bf72
LF: Use the new Struct data structure for ValueStruct (#7241)
This PR uses the new data structure introduced in #7220.
Additionnally this fix `Value Equal instance` which was considering
<a: x, b: y> different from <b:y, a:x>.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-04 18:32:39 +02:00
Remy
ee309d1e35
LF: extend tests for comparison builtins (#7314)
CHANGELOG_BEGIN
CHANGELOG_END
2020-09-04 17:13:29 +02:00
Gary Verhaegen
dd37d98458
fix grpcurl (#7324)
It looks like the structure of the nix package has changed when we
updated nixpkgs in #6761, so we need to update the dev-env script to
match.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-04 16:38:49 +02:00
Oliver Seeliger
f2a35ee922
Canton conformance test debug logging and reduced test parallelism (#7322)
* Canton conformance test debug logging and reduced test parallelism

To reign in flakiness of a handful of TransactionServiceIT tests
that had run into timeouts during synchronize

Closes #7316

changelog_begin
changelog_end

* Review feedback from Stefano
2020-09-04 14:25:30 +02:00
Moritz Kiefer
3171bb588a
Migrate trigger assertion library to DAML Script (#7318)
This is clearly a breaking change but triggers are still alpha so we
can get away this (confirmed with Bernhard).

changelog_begin

- [DAML Trigger] Daml.Trigger.Assert now uses DAML Script instead of scenarios.

changelog_end
2020-09-04 13:48:50 +02:00
Remy
724466f77c
Speedy compile: cleanup (#7321)
This PR perform the following changes:
 - reduce function size by spliting them out,
 - rename "translate" function prefix to "compile",
 - replace 'private' by 'private[this]',
 - add 'inline' annotation to function used only once.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-04 11:38:39 +02:00
Moritz Kiefer
0903b1a6be
Filter implicit parameter instances from data-dependencies (#7320)
* Filter implicit parameter instances from data-dependencies

Occasionally GHC likes to float implicit parameters to the top-level
making them look like any other typeclass instances. Implicit
parameters are by design not unique so if we try to reconstruct them
we get an error about a duplicate instance.

I haven’t figured out when exactly GHC floats things to the top-level
but I did verify that this testcase breaks without the fix.

changelog_begin
changelog_end

* Assert that there are 2 implicit instances

changelog_begin
changelog_end
2020-09-04 07:36:37 +00:00
Stephen Compall
0b72b6f716
more trigger service logs; tagged LoggingContext (#7205)
* get a LoggingContext into the TriggerRunnerImpl

* make some implicits more implicitly scoped and explicitly ascribed

* make some private/final markings

* most of JsonFormat[Identifier] is in companion

* experimental LoggingContext with phantom type

* ActorContext#log isn't really doing that much

* more details of LoggingContextOf

* make LoggingContextOf compile

* add trigger message logging, yet without context

* fix parent compile errors

* use Config as the phantom for its own logging extensions

* LocalDateTimeFormat cleanup

* switch TriggerRunner to contextual logging

* add trigger definition ID to logs

* log trigger-submitted commands, fix trigger test compile

* log trigger stopping and DAR uploads

* add context to PostStop/PreRestart logs

* add changelog

CHANGELOG_BEGIN
- [Triggers] More detailed logging of trigger actions and trigger service actions.
  See `issue #7205 <https://github.com/digital-asset/daml/pull/7205>`_.
CHANGELOG_END

* missed copyright header

* switch to Unit, scala/bug#9240 fixed
2020-09-03 17:47:09 +00:00
Remy
c4e21910ab
disble flaky tests in canton-conformance-test (#7315)
* mark canton-conformance-test as flaky

CHANGELOG_BEGIN
CHANGELOG_END

* Address Moritz's review

* fix

* fix

* Disable a handful of TransactionServiceIT changes that flake

rather than making the entire canton suite as flaky

Related to #7316

changelog_begin
changelog_end

* formatting

Co-authored-by: Oliver Seeliger <oliver.seeliger@digitalasset.com>
2020-09-03 15:07:02 +00:00
Remy
d08de60d9c
LF: prevent duplicate fields in Struct (#7299)
CHANGELOG_BEGIN
CHANGELOG_END
2020-09-03 16:07:38 +02:00
Miklos
0bfb4ba1d2
kvutils: Remove deprecated methods [KVL-406] (#7313)
* Removed deprecated signature from WriteService and LedgerWriter.

* CHANGELOG_BEGIN
CHANGELOG_END

* Reverted change.

* Fixed ScalaDoc for KeyValueParticipantState.
2020-09-03 12:12:58 +00:00
azure-pipelines[bot]
deb4648a60
update compat versions for 1.5.0-snapshot.20200902.5118.0.2b3cf1b3 (#7308)
CHANGELOG_BEGIN
CHANGELOG_END

Update exclusions to include new snapshot

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-09-03 14:07:55 +02:00
Moritz Kiefer
0ff3804266
Upgrade bl to address security vulnerability (#7312)
I’ve removed dockerode which we don’t seem to use and which depends on
a bunch of outdated versions in particular bl 1.x transitively.

changelog_begin
changelog_end
2020-09-03 09:49:06 +00:00
dependabot[bot]
53981f1027
Bump bl from 4.0.2 to 4.0.3 in /navigator/frontend (#7309)
Bumps [bl](https://github.com/rvagg/bl) from 4.0.2 to 4.0.3.
- [Release notes](https://github.com/rvagg/bl/releases)
- [Commits](https://github.com/rvagg/bl/compare/v4.0.2...v4.0.3)

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>
2020-09-03 11:28:37 +02:00
Samir Talwar
d4b4e7fe87
ledger-api-test-tool: Actually print the cause of uncaught exceptions. (#7310)
CHANGELOG_BEGIN
CHANGELOG_END
2020-09-03 09:26:46 +00:00
azure-pipelines[bot]
de807f33b6
rotate release duty after 1.5.0-snapshot.20200901.5116.0.4460cb5e (#7297)
@rohanjr is taking care of 1.5.0-snapshot.20200901.5116.0.4460cb5e (#7296), so they get pushed back to the end of the line.

Please do not merge this before #7296.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-09-03 10:04:18 +02:00
Sofia Faro
a938fd7773
Preserve empty variant constructor in data-deps. (#7303)
* Preserve empty variant constructor in data-deps.

This fixes #7207 and adds a regression test. In a
separate PR I'll add a warning for variants with
single argument of unit type and add a changelog
entry.

changelog_begin
changelog_end

* Add special case for single constructor variants

* Add test for special case
2020-09-02 21:16:16 +01:00
Brian Healey
f2707cc54f
Ensure shortIdentifier for ledger api tests do not overlap partially with other shortIdentifiers (#7279)
* uniquely name tests to easily facilitate running them one by one for troubleshooting

CHANGELOG_BEGIN
CHANGELOG_END

Signed-off-by: Brian Healey <brian.healey@digitalasset.com>

* ensure a few further tests do not include a portion of another tests shortIdentifier

* no overlap for CSsubmitAndWaitForTransactionTree
2020-09-02 13:41:07 -04:00
Moritz Kiefer
d9551b6cbd
Migrate skeleton to DAML Script (#7301)
* Migrate skeleton to DAML Script

Also kills the separate setup module since it’s the same thing

changelog_begin
changelog_end

* Fix test

changelog_begin
changelog_end
2020-09-02 19:14:50 +02:00
Leonid Shlyapnikov
6186c5ccfd
Fixing flaky Template ID resolution test (#7307)
* Adding `uniqueModuleEntity`

and making sure that generated domain Template IDs do not
have module entity duplicates, so resolution should work
with no problem.

changelog_begin
changelog_end

* cleaning up
2020-09-02 12:17:37 -04:00
azure-pipelines[bot]
3f1aa593b4
release 1.5.0-snapshot.20200902.5118.0.2b3cf1b3 (#7296)
* release 1.5.0-snapshot.20200901.5116.0.4460cb5e

This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.

@rohanjr is in charge of this release.

CHANGELOG_BEGIN
CHANGELOG_END

* release 1.5.0-snapshot.20200902.5118.0.2b3cf1b3

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-09-02 11:27:57 -04:00
anthonylusardi-da
b3b2a8119c
Update codegen docs to include Javascript (#6354)
* Partial update to codegen docs

* ADD: JS codegen to codegen docs

* CHANGELOG_BEGIN
ADD: JS codegen to codegen docs
CHANGELOG_END

* Make it clear we're talking about codegen

* Add extra reference to JS, JS -> TS, and mention missing Java/Scala classes

* Add placeholder for SDK version
2020-09-02 10:15:47 -04:00
Bernhard Elsner
86974d8afa
Add hints to the DAML Intro docs to encourage downloading the code. (#6839)
* Add hints to the DAML Intro docs to encourage downloading the code.

CHANGELOG_BEGIN
CHANGELOG_END

* Improve hints
2020-09-02 13:28:58 +00:00
Leonid Shlyapnikov
ccecc46e70
JSON API Perf Scenario Runner (#7295)
* Run gatling scenario from the perf runner main

reports are disabled for now, getting a class not found
when generating them

changelog_begin
changelog_end
2020-09-02 09:00:43 -04:00
Samir Talwar
fe7d56f099
kvutils/tools: Split the tools by directory. (#7306)
* kvutils/tools: Split the integrity checkers by package.

The two versions of the integrity checker are intertwined when they
really don't rely on each other at all. This splits them into two
separate packages to make the distinction clear.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils/tools: Split the integrity checkers by directory.

This splits the integrity checkers further into separate directories, to
make it clear they do not interact at all.

* kvutils/tools: Rename "integrity-check" to "integrity-check-v1".

* ledger-on-memory: Recommend ledger exports (v3), not ledger dumps (v1).

* kvutils/tools: Split the benchmarks from integrity-check-v2.
2020-09-02 12:45:56 +00:00
Moritz Kiefer
5e8196061e
Use implode in prettySrcLoc (#7305)
As pointed out by @hurryabit this is a bit nicer. The foldr was just
copied over from GHC.

changelog_begin
changelog_end
2020-09-02 12:43:01 +00:00
Samir Talwar
15cb93eec3
http-json: Use matchers in WebsocketServiceIntegrationTest. (#7302)
This makes diagnosing failures in CI easier, because "true is not false"
is not a very helpful error message.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-02 11:52:05 +00:00
Moritz Kiefer
abb979a4db
Fix flakiness in multi-key tests (#7298)
There are two sources of flakiness and I’ve seen both on CI:

1. We can get more than one offset at the beginning if things are too
   slow. This is addressed by just filtering those out.
2. The stream completes as soon as the input is closed. This is
   addressed by keeping the stream open and closing it with `take`.

Point 2 is a problem for all tests, see #7293, but I’ll leave the
other tests for separate PRs (I’ve also never seen them flake).

You can observe the failures locally if you add a `Thread.sleep` between
creating the stream future and sending the commands.

changelog_begin
changelog_end
2020-09-02 12:50:51 +02:00
Samir Talwar
2b3cf1b3c0
ledger-api-client: Rename maxInboundMessageSize to maxInboundMetadataSize. (#7290)
* ledger-api-client: `maxInboundMessageSize` -> `maxInboundMetadataSize`.

CHANGELOG_BEGIN
- [Scala Bindings] Rename a field in the ``LedgerClientConfiguration``
  to ``maxInboundMetadataSize``, to match the builder Netty channel
  builder. It was incorrectly named ``maxInboundMessageSize``, which is
  a different channel property that configures the maximum message size,
  not the header size.
CHANGELOG_END

* ledger-api-client: Introduce a `maxInboundMessageSize` config property.

We use this a lot; easier if it's in the configuration.

CHANGELOG_BEGIN
- [Scala Bindings] Replace the
  ``LedgerClientConfiguration.maxInboundMessageSize`` property with a
  new one that represents the maximum size of the response body.
CHANGELOG_END
2020-09-02 08:41:34 +00:00
Samir Talwar
0c99f637a0
[KVL-254] kvutils: Export to protobuf, not a custom serialization format. (#7265)
* kvutils: Implement a basic protocol buffer-based export.

* kvutils: Add a header to the protobuf-based export.

So if we decide to go for a v4, it's easier to figure this stuff out.

* kvutils: Use our own header format for export.

The idea is to be somewhat forward-compatible, so relying on protobuf
to always be the format of the future seems a little odd. We may switch.

* kvutils: Construct the importer up-front during integrity checks.

* kvutils: Move the various importers and exporters to versioned packages.

* kvutils: Move the export version to the package object.

* kvutils: Create helpers to construct importers and exporters from paths.

* kvutils: Process either v2 or v3 exports with the integrity checker.

* kvutils: Sort the write set during export.

* kvutils: Switch to v3 of the export.

The integrity checker will work with either version.

CHANGELOG_BEGIN
- [Integration Kit] kvutils-based ledgers will now export the ledger
  using a new serialization format based on Protocol Buffers, which we
  are referring to as "v3". Existing "v2" exports can still be read and
  verified using the integrity checker, which can read both old and new
  versions.
CHANGELOG_END

* kvutils: Aggregate export data in a sorted map.

We need to sort it anyway; might as well do it on input.

* kvutils: Improve a test string.

* kvutils: Use the v3 importer in the replay test.

* kvutils: Remove v2 of the export format.

We're pretty sure no one is using it.
2020-09-02 07:41:25 +00:00
Moritz Kiefer
4460cb5e08
Fix trace statement handling in DAML Script service (#7292)
* Fix trace statement handling in DAML Script service

This PR makes sure that we get both trace statements coming from the
ledger client as well as from the ledger server. In the script service
those are two different Speedy machines so we need to interleave
them. This is a bit messy so let me explain the steps (apologies for
not splitting it up but I think it’s easier to understand the
motivation when it is a single PR):

1. Abstract over the tracelog interface. The ringbuffer doesn’t makes
   sense for our purposes and is annoying to clear.

2. Pass in the respective fields of `Machine` to `Conversions` instead
   of the whole machine. This is both a simpler design (passing around
   mutable objects scares me) and it allows us to assemble the fields
   from different machines.

3. Initialize the ledger machine with a simple ArrayBuffer tracelog.

4. After `submit` and `submitMustFail` copy the tracelog from the
   ledger to the client.

fixes #7280

changelog_begin
changelog_end

* Address review feedback

changelog_begin
changelog_end
2020-09-01 17:15:48 +00:00
Gary Verhaegen
989fdc9bde
fix JSON API multikey stream (#7244)
fix JSON API multikey stream

In the current state, the JSON API only handles multiple keys _from
different templates_. This makes it work for multiple keys from the same
template too.

Extracted from #7066 with the following changes:

- Use of a mutable `HashSet` to test for keys, because perf.
- Addition of a test at the JSON API level.

CHANGELOG_BEGIN

- [JSON API] Fix a bug where streaming multiple keys could silently
  ignore some of the given keys.

CHANGELOG_END

* apply @cocreature's patch

https://gist.github.com/cocreature/d35367153a7331dc15cca4e5ea9098f0

* fix fmt
2020-09-01 18:17:13 +02:00
Moritz Kiefer
09a1b445df
Only include stakeholder contracts in result of query (#7291)
* Only include stakeholder contracts in result of query

This fixes a bug in the script service. We need to filter out divulged
contracts since this should behave exactly like the ACS endpoint on
the ledger API.

changelog_begin
changelog_end

* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/scenario/ScenarioLedger.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

* Update daml-script/runner/src/main/scala/com/digitalasset/daml/lf/engine/script/LedgerInteraction.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

* fmt

changelog_begin
changelog_end

Co-authored-by: Remy <remy.haemmerle@daml.com>
2020-09-01 10:34:11 +00:00
Moritz Kiefer
f8a82ad25f
Add queryContractId to DAML Script (#7289)
* Add queryContractId to DAML Script

I’ve switched between a few different naming options and in the end
settled on this one:

- `lookupContract`, not too bad but misses the connection to `query`
- `queryCid`, we don’t abbreviate this in other places in our API so I
  don’t think we should here.
- `queryContractId`, makes the connection clear and no
  abbreviation. We could also add `queryContractKey` later

changelog_begin

- [DAML Script] Add a `queryContractId` function for querying for a
  contract with the given identifier.

changelog_end

* Fix test

changelog_begin
changelog_end
2020-09-01 10:31:34 +00:00