Commit Graph

4249 Commits

Author SHA1 Message Date
Shayne Fletcher
0e1ac4670c
Add maxInboundMessageSize param to trigger service (#6000)
changelog_begin
- trigger runner supports the `max-inbound-message-size` option
changelog_end
2020-05-15 13:52:59 +00:00
Oliver Seeliger
c0af52950c
Fix ledger api test tool multi-node CommandService test flake (#5998)
by adding synchronize call between an alpha-participant-create and the
corresponding beta-participant-exercise.

changelog_begin
changelog_end

We occasionally run into this in Canton
2020-05-15 13:41:57 +02:00
Moritz Kiefer
2853a20eb9
Make max-inbound-message-size in DAML script/repl configurable (#5996)
Fixes #5592

The CLI syntax and the defaults follow the JSON API here.

changelog_begin

- [DAML Script] The maximum inbound message size can now be configured
using `--max-inbound-message-size``. This matches the flag in the JSON
API.

- [DAML REPL] The maximum inbound message size can now be configured
using `--max-inbound-message-size``. This matches the flag in the JSON API.

changelog_end
2020-05-15 07:22:05 -04:00
Samir Talwar
8cd3cc414e
ledger/metrics: Display graphs with counts alongside mean timings. (#5994)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-15 08:44:34 +00:00
Stefano Baghino
be914d3b26
Measure translation time for full result from DB (#5983)
Furthermore, uses a work stealing thread pool for work JdbcLedgerDao has
to perform outside of the database thread pool. This solves some
flakiness observed in conformance tests against PostgreSQL-backed
ledgers.

changelog_begin
changelog_end
2020-05-15 09:02:02 +02:00
Shayne Fletcher
97fd974381
Add createAndExerciseCmd to triggers (#5990)
changelog_begin
- New function `createAndExerciseCmd` added to DAML triggers
changelog_end
2020-05-14 13:24:01 -04:00
Remy
3eed4d1a22
Ledger: remove all reference to RelativeContractId (#5985)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-14 19:12:32 +02:00
Samir Talwar
f18beb9851
daml-sdk-head: Delete the SDK temporary directory after installation. (#5986)
I'm aggregating a lot of SDKs.

```
$ du -h -d1 $TMPDIR | sort -rh | head -5
1.9G    /var/folders/5x/36ws6_n96qz_b4tbpvfbcth80000gn/T/
501M    /var/folders/5x/36ws6_n96qz_b4tbpvfbcth80000gn/T/tmp.ahBTENEeZw
500M    /var/folders/5x/36ws6_n96qz_b4tbpvfbcth80000gn/T/tmp.S4sB7DIjkA
500M    /var/folders/5x/36ws6_n96qz_b4tbpvfbcth80000gn/T/tmp.Q3VeVV8DIU
405M    /var/folders/5x/36ws6_n96qz_b4tbpvfbcth80000gn/T/tmp.b5ZWnPhLkH
```

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-14 15:33:32 +00:00
Samir Talwar
76a7e6e18d
daml-sdk-head: Standardize the syntax of the script. (#5984)
* daml-sdk-head: Quote all variables.

And use arrays where necessary.

* daml-sdk-head: Use Bash tests (`[[` and `]]`) rather than `test`.

Most importantly, they support the `&&` and `||` operators, because
they're a shell builtin, not a program.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-sdk-head: Refer to `${BASH_SOURCE[0]}` explicitly.

`BASH_SOURCE` is an array. Expanding an array gets the first element, but this
is unclear. We can clarify by explicitly expanding the first element.

* daml-sdk-head: Use `command -v` instead of `which`, as it's more standard.

* daml-sdk-head: Use semicolons judiciously.
2020-05-14 15:06:00 +00:00
Gerolf Seitz
a4f75c3cef
Fix quadratic runtime when reading events (#5981)
Fixes #5980.

CHANGELOG_BEGIN
[Sandbox] Improved performance when loading transactions with many
events. See `issue #5980
<https://github.com/digital-asset/daml/issues/5980>`__.
CHANGELOG_END
2020-05-14 13:31:15 +00:00
Martin Huschenbett
ce98261dbc
Fix a typo in the DAML-LF spec around contract keys (#5975)
In our first design draft, the maintainers of a contract key were
computed from the template payload. Now, they are computed from the
key itself. The rule `EvUpdCreateWithKeySucceed` still reflect the old
behavior but changing a single index from `t` to `v` fixes the issue.

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-14 09:30:52 +00:00
Samir Talwar
57a8d0b37e
CI: Run PostgreSQL once for all Scala tests. (#5919) 2020-05-14 09:06:34 +02:00
Martin Huschenbett
9f43ab95ac
Fix evaluation order for create with contract keys (#5972)
Currently, a call to `create` for a template with a contract key
evaluates the key even before _executing_ the `create`, which is _way_
too early. See #5967 for examples what this implies.

This change moves the evaluation of the key into the execution of the
`create` and therein _after_ the evaluation of the `ensure` clauses,
which is where it belongs according to the DAML-LF specification.
Unfortunately, the specification does not say anything about the
relative evaluation order of the `agreement`, `signatory`, `observer`
and `key` clauses. Thus, I've deciced to evaluate the key after the
other three, which seems to be its natural place given its optionality.

This fixes #5967.

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-13 21:31:32 +00:00
Shayne Fletcher
2b33a4d273
Version the existing paths (#5970)
changelog_begin
changelog_end
2020-05-13 16:09:41 -04:00
Remy
b1c09ce1a0
Update perf tests SHA (#5971)
* update Perfs test sha

* changelog

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-13 19:08:53 +00:00
Remy
deedec9b9f
Engine: Remove optionality of contract ID Seeding. (#5966)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-13 20:29:14 +02:00
Stephen Compall
af52ef3ce9
random IOU create and split testing for websocket streams (#5879)
* Consume language for listing [re]actions in a foldAsync linearly

* eagerly consume as many Futures as possible

* restate one foldAsync test with Consume

* report value that failed to match

* add error locations and new `drain` primitive

* use `drain` to properly handle heartbeats at end of stream

* it doesn't make sense to delay the drain step

* rewrite other websocket test; document syntax

* get rid of StreamState, finally

* split Consume to a separate file

* plan split tests

* functions for consuming SplitSeq

* converting SplitSeq into stream actions

* exercise and check recursively

* add one 100-sized sample to the tests

* don't violate Iou's invariant

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* disable test pending tracking down duplicates

- as discussed with @leo-da
2020-05-13 14:06:11 -04:00
Moritz Kiefer
577cd653d9
Add debugging output to inclusion errors (#5961)
* Add debugging output to inclusion errors

This adds some more debugging output to inclusion errors in Bazel
which should hopefully help us track it down. These are the only call sites
in the Bazel source that can produce them. My suspicion is that it’s
coming from HeaderDiscovery but I’m not entirely sure what is off.

We’ll almost certainly have to add more output once we know which of
those 3 cases we hit but let’s do it step by step.

changelog_begin
changelog_end

* Bump url and hash

changelog_begin
changelog_end
2020-05-13 18:18:17 +02:00
Remy
9019b3a33c
Ledger: Remove optionality of contract ID seeding. (#5965)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-13 17:39:07 +02:00
Stefano Baghino
b0ad8d7a60
Centralize and time serialization (#5946)
changelog_begin
OVERRIDES CHANGELOG ENTRY FROM 2a8c93a614
[Ledger Integration Kit] Added new metrics for
``daml.index.db.*.translation`` to measure the time spent
translating to and from the serialized DAML-LF values when
fetched from the participant index.
changelog_end
2020-05-13 17:06:46 +02:00
Stefano Baghino
e39adbe69e
Cleanup: move unused translation helpers to migrations (#5963)
* Cleanup: move unused translation helpers to migrations

These helpers are no longer used in the main source tree.

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/5963#discussion_r424415151
2020-05-13 16:35:05 +02:00
Gary Verhaegen
9d5e0d5654
1.2.0-snapshot.20200513.4172.0.021f4af3 (#5960)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-13 15:40:18 +02:00
Shayne Fletcher
007ebeb56a
Add http-port option (#5951)
changelog_begin
changelog_end
2020-05-13 09:30:03 -04:00
Remy
203df44498
Scenario service: drop useless resolveRelCid (#5932)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-13 13:00:44 +00:00
Moritz Kiefer
bdb591575e
Forward port release notes for 1.0.1 and 1.1.0 (#5962)
At least for now we still somewhat pretend to have a linear version
history so we want to include this in future releases.

changelog_begin
changelog_end
2020-05-13 14:34:00 +02:00
Oliver Seeliger
cc880e290b
Port PerformanceEnvelope.TransactionSize perf test from Canton (#5955)
* Port PerformanceEnvelope.TransactionSize perf test from Canton

to LedgerApiTestTool --perf-tests

changelog_begin
[Ledger Api Test Tool] Added the TransactionSize performance benchmark test.
changelog_end

* Review feedback from Samir

* Formatting
2020-05-13 12:03:46 +00:00
Moritz Kiefer
294c881a2a
Fix standard change check (#5958)
This check never triggered for changes to LATEST due to the trailing
slash in `has_changed`.

changelog_begin
changelog_end
2020-05-13 13:58:43 +02:00
Remy
021f4af394
Sandbox classic: Emulate legacy contract ID scheme (#5929)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-13 12:56:41 +02:00
Martin Huschenbett
b954361d5b
Make Remy and Martin codeowner of everything interpreter (#5952)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-13 10:51:28 +00:00
Moritz Kiefer
4916a28682
Include create-daml-app tests in compatibility tests (#5945)
This is the first part of #5700

It adds tests that build create-daml-app using `daml build` and then
run the codegen and build the UI. Contrary to our main tests these
also run on Windows. This is actually reasonably simple by first
building the typescript libraries on Linux and then downloading them
on Windows.

There are two parts that are still missing from the tests in the main
workspace:

1. Building the extra feature. This should be fairly easy to add.
2. Running the pupeeter tests. At least MacOS and Linux should be
   reasonably easy. I don’t know what horrors Windows will throw at
   us. This step is what actually makes this a compatibility
   test. Currently it doesn’t actually launch Sandbox and the JSON API.

Since this PR is already pretty large, I’d like to tackle those things
separately.

changelog_begin
changelog_end
2020-05-13 10:39:51 +02:00
Gary Verhaegen
bda565fa44
patching Bazel on Windows (infra bits, no patch yet) (#5918)
patch Bazel on Windows (ci setup)

We have a weird, intermittent bug on Windows where Bazel gets into a
broken state. To investigate, we need to patch Bazel to add more debug
output than present in the official distribution. This PR adds the basic
infrastructure we need to download the Bazel source code, apply a patch,
compile it, and make that binary available to the rest of the build.
This is for Windows only as we already have the ability to do similar
things on Linux and macOS through Nix.

This PR does not contain any intresting patch to Bazel, just the minimum
that we can check we are actually using the patched version.

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-12 23:16:04 +02:00
Shayne Fletcher
221c6db0ae
Rename targets (#5950)
changelog_begin
changelog_end
2020-05-12 16:00:15 -04:00
Stefano Baghino
9ff36a13cf
Extract caching from participant-state as a library (#5949)
* Extract caching from participant-state as a library

This will be used to keep a cache of values to cut on LF translation cost when serving transactions.

changelog_begin
changelog_end

* Add dependency where missing
2020-05-12 19:45:55 +02:00
Edward Newman
0ec0cc335f
Updates to support VMWare variant of Hypervisor for MacOS Build Nodes (#5940)
* Updates to support VMWare vairant of Hypervisor

* Update infra/macos/scripts/rebuild-crontask.sh

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Update infra/macos/scripts/run-agent.sh

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-05-12 09:36:40 -04:00
Shayne Fletcher
6cbe36bf4d
Shutdown protocol (#5936)
changelog_begin
changelog_end
2020-05-12 08:33:05 -04:00
Shayne Fletcher
1007df46d2
script to start ledger and trigger service (#5939)
changelog_begin
changelog_end
2020-05-12 08:32:34 -04:00
Remy
1cd412fa83
Sandbox: addapt scenario loader to the new contract ID scheme (#5942)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-12 13:45:14 +02:00
Stefano Baghino
7e0377dd76
Offload transaction write preparation from DB thread (#5930)
* Offload transaction write preparation from DB thread

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/5930#discussion_r423049118

* Solve compilation issue

* Move type into companion object

* Address https://github.com/digital-asset/daml/pull/5930#discussion_r423170046
2020-05-11 22:51:32 +00:00
Gary Verhaegen
3899a59a11
switch back to hosted macOS nodes (#5935)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-11 22:59:33 +02:00
Moritz Kiefer
84dbce9eba
Fix typescript tests (#5934)
The new version of eslint has broken stuff, see the linked issue.

changelog_begin
changelog_end
2020-05-11 19:37:53 +00:00
Davin K. Tanabe
60b2649dc2
Change references to replicateM to replicateA in dlint rules; remove some Haskell-based rules that do not yet apply in DAML. (#5933)
changelog_begin
dlint: Change references to replicateM to replicateA; remove some Haskell-based rules that do not currently apply in DAML.
changelog_end
2020-05-11 13:32:51 -04:00
Sofia Faro
1bd117bcec
Inline typeclass projections. (#5759)
* Inline typeclasses and apply projections.

This draft PR:

* adds a Subst module for substitution within LF expressions.
  This implementation piggybacks on the existing type
  substition. But there are not enough tests yet.
* passes World data into the simplifier, in order to have
  have the ability to inline functions (selectively)
* inlines typeclass dictionaries and projection functions
* beta reduces type lambdas (which are just ignored by
  speedy afaik)
* beta reduces lambdas that are passed units or dictionaries
* runs the simplifier twice so it has a change to
  perform the inlining, reduction, AND projection
  -- this is probably avoidable by sequencing the
  simplifier steps in a specific order, but running
  the simplifier twice is simple enough for now.
* together, these fix #5748 (see result on a toy module)

TODO:

* add lots of tests for Subst
* run this on a benchmark to see how big of a difference it makes.
* reduce the jankiness of running the simplifier twice.

Results:

DAML input:

```
module Main where

hello : Int
hello = 10 + 30
```

Original DAML-LF output:

```
module Main where
    @location(7:0-7:5)
    def hello : Int64 =
      a284919a95c4a515cd1efac0d89be302d0e9d61e692a2176128c871ad8067e36:GHC.Num:+
        @Int64
        a284919a95c4a515cd1efac0d89be302d0e9d61e692a2176128c871ad8067e36:GHC.Num:$fAdditiveInt
        10
        30
```

Current DAML-LF output:

```
 module Main where
    @location(7:0-7:5)
    def hello : Int64 = ADD_INT64 10 30
```

changelog_begin
changelog_end

* Update copyright header

* Lint

* More direct typeclass simplification

* extendWorldSelf + getTypeClassDictionary

* Fix visual

* Fix visual comment

* Disable cross-module inlining for incremental builds

* Cleanup subst/freevars.

* copyright header

* Alpha equivalence for LF expressions.

* copyright header

* lots of tests

* fix comment

* Apply review suggestions
2020-05-11 16:48:27 +00:00
Remy
93a01f7e12
DAML-LF: enforce size limit for Contract ID suffixes. (#5913)
Following the Contract ID Specification, we limit the size of contract ID suffixes to 94.

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-11 15:32:17 +02:00
Stefano Baghino
dd6af4e276
Centralize index DB metrics naming (#5927)
* Centralize and review index DB metrics naming

Closes #5922

changelog_begin
changelog_end

* Remove unnecessary factory

* Fix compilation issue
2020-05-11 12:59:02 +00:00
Bernhard Elsner
1615a79aea
Update Contract Key docs (#5923)
* Update Contract Key docs

CHANGELOG_BEGIN
CHANGELOG_END

* Also update the Updates section

CHANGELOG_BEGIN
CHANGELOG_END

* Update docs/source/daml/reference/contract-keys.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Incorporate feedback

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-05-11 12:18:19 +00:00
Moritz Kiefer
c3f6d85192
Bump timeout in create-daml-app tests (#5928)
At least with the official azure CI nodes, this seems to run into timeouts.

changelog_begin
changelog_end
2020-05-11 11:40:47 +00:00
nickchapman-da
4823053441
Add instrumentation and lightweight tracing to Speedy CEK machine (#5819)
- Disabled by default.
- Enabled with flags: `enableInstrumentation` and `enableLightweightStepTracing'

The instrumentation:
- tracks #pushes and maximum-depth for the `kont`-stack and the `env`ironment
- counts #steps
- classifies  each step:
    - for `CtrlExpr`, what kind of expression is it?
    - for `CtrlValue`, what kind of continuation is at the head of the `kont`-stack?
- the information is printed when the machine finalizes

The lightweight tracing shows an abbreviated output line per step:
- the control expression / value
- the size of the environment
- the depth of the `kont`-stack

changelog_begin
changelog_end
2020-05-11 11:10:02 +01:00
Andreas Herrmann
35fe61ed06
Update Bazel 2.0.0 --> 2.1.0 (#5651)
* Update Bazel 2.0.0 --> 2.1.0

CHANGELOG_BEGIN
CHANGELOG_END

* Update rules_haskell

For Bazel 2.1.0 compatibility.

* nix: grpc: patch glibc version incompatibility

* Fix hlint warnings

* CcToolchainInfo.compiler

https://docs.bazel.build/versions/2.1.0/skylark/lib/CcToolchainInfo.html#compiler

* CcToolchainInfo.ar_executable

* texlive: add missing transparent package

* Leave a debug note on the docs target

* Ignore FutureWarning in //docs:docs-no-pdf

* Isolate PR

This is to avoid any interference with pipelines prior to the Bazel
update.

* Remove isolation

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-05-11 11:47:54 +02:00
Stefano Baghino
2a8c93a614
Offload value deserialization from DB threadpool and add metrics (#5917)
* Offload value deserialization from DB threadpool and add metrics

Fixes #5909

CHANGELOG_BEGIN
[Ledger Integration Kit] Added new metrics for
``daml.index.db.*.deserialization`` to measure the duration of
the translation of the serialized DAML-LF values when fetched
from the participant index.
CHANGELOG_END

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422282582

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422282901

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422283768

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422285920

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422285674
2020-05-11 10:41:49 +02:00
Gary Verhaegen
9b476416b8
switch back to Azure-provided macos nodes (#5920)
This is temporary. It looks like the macOS nodes are dead; @nycnewman is
looking into it, but in case he doesn't fix it in time, at least we
have a backup plan so we're not completely blocked on Monday.

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-11 09:28:40 +02:00