Commit Graph

2963 Commits

Author SHA1 Message Date
Leonid Shlyapnikov
cb465837ef
Fixing package reload scheduling, thanks @lima-da for providing the fix (#4254)
CHANGELOG_BEGIN

[JSON API - Experimental]
Fixing package reload scheduling. The bug was introduced when upgraded
to the most recent Akka library. Also using `scheduleWithFixedDelay` and
awaiting for the reload future to complete. This is to guarantee that we
don't trigger multiple reloads simultaneously. Package reload is
thread-safe though.

CHANGELOG_END
2020-01-30 10:46:38 -05:00
Stefano Baghino
fabd6d9f2c
Refactor getTimeModel (#4284)
Avoid unnecessary flatmapping/indentation, prepare default response once

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-30 15:19:43 +00:00
Samir Talwar
eaf17cae73
ledger-on-posix-filesystem: Lock the entire ledger with a file lock. (#4249)
* ledger-on-posix-filesystem: Run `scalafmt`.

* ledger-on-posix-filesystem: Just use file channel locks. Seriously.

* ledger-on-posix-filesystem: Rename `FileSystemLock#run` to `apply`.

* ledger-on-posix-filesystem: Conformance tests now run quickly.

So we can stop constraining the number of concurrent test runs.

* ledger-on-posix-filesystem: Lock the entire ledger with a file lock.

This ensures we can't have two separate ledger servers running against
the same directory.

This is mostly an experiment to better understand how to do this on an
SQL ledger for #4039.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-posix-filesystem: Run `scalafmt`.

* ledger-on-posix-filesystem: Use `Resource#use` in `FileSystemLock`.

* resources: Move `Resource#use` to `ResourceOwner#use`.

Makes more sense for the same object to govern both `acquire()` and
`release()`.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-30 14:35:38 +00:00
Andreas Herrmann
87b84772ab
Mark //ledger-service/http-json:tests as flaky (#4282)
Latest failure
https://dev.azure.com/digitalasset/daml/_build/results?buildId=30191&view=logs&j=0da5d1d9-276d-5173-c4c4-9d4d4ed14fdb&t=f58be928-45a1-58e7-c1ad-04869565b3f4&l=6805

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-30 13:22:32 +00:00
Moritz Kiefer
d86adc6830
Create actors in AkkaExecutionSequencerPool as system actors (#4278)
This allows using this from `akka-typed` where you otherwise get an
error that you `cannot create top-level actor from the outside on
ActorSystem with custom user guardian`. See
https://github.com/akka/akka-http/issues/1873#issuecomment-367655223
for the similar issue in `akka-http`. The resolution matches the fix
that `akka-http` https://github.com/akka/akka-http/pull/1878 and
`akka-streams` https://github.com/akka/akka/pull/24460 have used.

I’ve followed the approach taken by `akka-streams` which matches on
the system and only uses a system actor if it gets an
ExtendedActorSystem. For my purposes, it would also be fine to change
the signature to require an `ExtendedActorSystem` but that seems like
a larger change.

changelog_begin
changelog_end
2020-01-30 12:51:17 +01:00
Shayne Fletcher
22fa5edf7e
First cut at developer docs (#4272)
changelog_begin
changelog_end
2020-01-30 11:42:32 +01:00
Richard Kapolnai
325e07e120
Minor clarifications in intro docs about transaction consequences (#4169) 2020-01-30 11:21:38 +01:00
Samir Talwar
fb58dc8756
ledger-on-sql: Make sure the ledger picks up where it left off. (#4266)
* resources: Make it easy to use a resource and then release it.

* ledger-on-sql: Move database transaction methods to `Database`.

* ledger-on-sql: Make sure the ledger picks up where it left off.

This means seeding the dispatcher with the correct head, rather than
starting at 1 every time.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-sql: Move `Index` and `StartIndex` to the package object.

They're used everywhere.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-30 10:16:32 +00:00
Robin Krom
eacacbf44e
fix release script (#4275)
* fix release script

only upload from the linux machine and escape arguments to `publish`,
otherwise they get interpreted by bazel.

CHANGELOG_BEGIN
CHANGELOG_END

* Update release/src/Main.hs

Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update release/src/Main.hs

Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* bracket -> bracket_

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-01-30 10:51:39 +01:00
Stefano Baghino
a1f8aeba88
Clean-up JdbcLedgerDao (#4262)
Removes unused call, fixes access modifiers, removes unnecessary curly braces

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-30 08:58:44 +00:00
Stefano Baghino
abde9d3b8b
Fix contract keys test description (#4277)
Reflects the change happened in 81815838a3

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-30 08:49:50 +00:00
Samir Talwar
e54809cecc
sandbox-perf: Fix broken benchmarks and make sure they stay fixed. (#4265)
The `sandbox-perf` build has been failing for a while with the following
errors:

```
INFO: From Generating benchmark code for //ledger/sandbox-perf:sandbox-perf_codegen:
JMH benchmark generation: JMH Benchmark generator failed
JMH benchmark generation: Benchmark classes should not be final.
   [com.digitalasset.platform.sandbox.perf.LargeTransactionBench]
JMH benchmark generation: The instantiated @State class cannot be abstract.
   [com.digitalasset.platform.sandbox.perf.PerfBenchState]
```

However, these errors are ignored; running the benchmarks runs the
`AcsBench` benchmark and ignores the fact `LargeTransactionBench` and
`SimpleBench` failed to compile.

I've fixed the errors by making sure that classes are not final, and
that `SimpleBench` uses a concrete state class.

This also introduces a cheap patch to the Scala JMH Bazel rules that
makes sure they fail if there are any errors.

I'm not really sure how to patch the Bazel rules properly, but someone
else might have an idea. :-)

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-30 08:25:25 +00:00
Shayne Fletcher
a48b7b039f
Update hlint (#4276)
* Update hlint

changelog_begin
changelog_end

* Update hlint

changelog_begin
changelog_end
2020-01-30 00:06:18 +00:00
Leonid Shlyapnikov
4be77fd889
Temporary work-around, can't reproduce locally: LedgerBegin > LedgerEnd (#4271)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 15:29:17 -05:00
Martin Huschenbett
d6f54612c3
daml-react.ts: Don't use JSX (#4273)
We have only one place where we could use it. The linter is struggling
with it, so let's just avoid it.

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 19:01:00 +00:00
Robin Krom
d9dffc13c4
release 0.13.48 (#4269)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 19:22:41 +01:00
Gary Verhaegen
6233f66ff6
remove debug Windows machine (#4267)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 18:07:53 +01:00
Andreas Herrmann
2e08e23703
Daml.Trigger.Assert for trigger testing API (#4264)
* Daml.Trigger.Assert for trigger testing API

Requires extracting part of Daml.Trigger into Daml.Trigger.Internal to
get access to internal data constructors and functionality

CHANGELOG_BEGIN
- [DAML Trigger - Experimental] Trigger testing functionality is now
available in the module Daml.Trigger.Assert.
CHANGELOG_END

* Set exposed-modules to hide Daml.Trigger.Internal

* API docs for Daml.Trigger.Assert

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-29 16:57:00 +00:00
Robin Krom
ce9cd93519
publish npm packages as public (#4263)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 17:26:46 +01:00
Samir Talwar
4d324f6aaf
Run scalafmt on all files and fix the errors. (#4260)
Unfortunately, downgrading `scalafmt` resulted in files that would be
reformatted to invalid code, because lines would be merged but trailing
commas would be left in.

I've manually run it everywhere and fixed these issues so people don't
have to fix them as they go.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-29 16:15:49 +00:00
daravep
215f751acf
Minor update to daml reference docs (#4243)
Before, we didn't mention in the docs that sum-types need to derive from
(Eq,Show) if they are to be used as arguments of templates or choices.
Now we do.

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 16:30:23 +01:00
Robin Krom
2e9fe6afb6
language: add daml-react package to ts libraries (#4259)
* language: add daml-react package to ts libraries

This adds the library formerly known as `daml-react-hook` into the
monorepo. We renamed it to `@daml/react`.

The tests sadly don't work with bazel right now because the local
imports aren't resolved correctly. Local testing with `yarn run test`
works as usual.

CHANGELOG_BEGIN
CHANGELOG_END

* address moritz comments

* get rid of DAVL mentions

* fix eslint warnings

* Update language-support/ts/daml-react/tsconfig.json

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-01-29 15:37:47 +01:00
Martin Huschenbett
34ac1b0cb8
Release DAML SDK 0.13.47 (#4258)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 15:24:54 +01:00
Stefano Baghino
01f7e5c7b3
Revert "Adopt trailingCommas option in scalafmt (#4214)" (#4257)
This reverts commit 6c094b3f95.

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 11:43:35 +00:00
Samir Talwar
f9de93b192
ledger-on-sql: Don't allow unnamed in-memory databases. (#4255)
* ledger-on-sql: Don't allow unnamed in-memory databases.

It seems that changing the connection pool size at runtime causes weird
issues, predominantly that sometimes all connections are dropped and
therefore the in-memory database is lost after migration. Obviously, it
then stops working.

This works around this problem by simply not allowing unnamed in-memory
databases, guaranteeing that we can share the database between the
various connection pools.

This will probably be necessary anyway when attempting to share the
database between the ledger and the index.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-sql: Make it clear we're generating a new JDBC URL in tests.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-29 10:34:00 +00:00
Miklos
f6939ccac2
Validate submission inputs vs. outputs (#4242)
Check submission inputs vs. outputs right after processing submission.

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 11:14:34 +01:00
Samir Talwar
3ac5b8f9da
ledger-on-whatever: Enable more conformance tests. (#4250)
* ledger-on-{memory,posix-filesystem,sql}: Enable slow conformance tests.

They're now fast enough. :-)

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-{memory,posix-filesystem,sql}: Enable config management tests.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-29 09:30:49 +00:00
Stefano Baghino
87b4917ccf
Run scalafmt to reduce noise on #4217 (#4245)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 10:18:56 +01:00
Stephen Compall
d6a20cba34
Avoid ending at different ledger ends for multi-template-ID queries (#4153)
* make searchInMemory do only one transaction stream

* drop unneeded deletes from memory while doing searchInMemory

* remove unused functions

* rename OffsetBookmark to BeginBookmark

* add AtAbsolute option to GetCreatesAndArchivesSince

* add GetTermination raw command

* propagate GetTermination to ContractsFetch

* remove fetching from searchDbOneTpId_

* fetch multiple contract IDs for DB persistence always to a fixed offset

* bugfix and cleanup; no changelog relevant

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-28 17:50:18 -05:00
Shayne Fletcher
0e53ec0a2b Integrate daml2ts into the assistant (#4252)
changelog_begin
- Generate TypeScript bindings to DAML with `daml codegen ts DAR -o OUTDIR`
changelog_end
2020-01-28 22:03:54 +00:00
associahedron
a8343e62c6 Refactor ModRef out as a type. (#4251)
* Refactor modrefs out as a type.

This is in preparation for moving to a monadic style.

changelog_begin
changelog_end

* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/DataDependencies.hs

Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-01-28 18:17:49 +00:00
Martin Huschenbett
83d720664d
dam2ts: Make template id accessible at the type level (#4248)
We add a third type parameter `I` to the `Template` interface and use it as
the type of the `templateId` field. We add this parameter to `CreateEvent`,
`ArchiveEvent` and `Event` as well. The reason behind this is to allow for
patterns like
```ts
function handleFooOrBar(event: CreateEvent<Foo, Foo.Key, typeof Foo.templateId>
                             | CreateEvent<Bar, Bar.Key, typeof Bar.templateId>) {
    switch (event.templateId) {
        case Foo.templateId: ...
        case Bar.templateId: ...
    }
}
```
and get exhaustiveness checking. This will become particularly handy when
handling upgrades, where `Foo` and `Bar` would be the old and new version
of a template, resp.

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-28 17:49:10 +01:00
associahedron
aeaf3ceab7
Export typeclasses in data-dependencies. (#4247)
* Typeclass declarations

* Add modrefs for typesyns

changelog_begin
changelog_end

* Convert constraints and expose methods using them
2020-01-28 16:44:54 +00:00
Martin Huschenbett
d568c113df
damlc: Allow all build options for damlc ide (#4213)
Currently, the DAML language server use for DAML Studio crashes when the
`build-options` section of `daml.yaml` contains flags like `--package`.
This PR fixes this issue by allowing most of the flags for `damlc build`
for `damlc ide` as well.

CHANGELOG_BEGIN

- [DAML Studio] Allow all command line flags for ``daml build`` for
  DAML Studio as well.

CHANGELOG_END
2020-01-28 17:22:02 +01:00
Samir Talwar
c81cc2ab2a ledger-on-sql: Migrate tables using Flyway. (#4232)
* ledger-on-sql: Pull out a superclass for the various integration tests.

* ledger-on-sql: Use Flyway to migrate tables in an idempotent manner.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-sql: Make it easy to run it with Bazel for experimentation.

* ledger-on-sql: Test that migrations will never change in the future.

* ledger-on-sql: Add a prefix of "ledger_" to the tables.

This is so we don't accidentally conflict with the index when the
schemas are shared.

I am letting myself modify the migrations because the existing
migrations haven't been merged into `master` yet.

* ledger-on-sql: Explain why we change the pool size after migration.

* ledger-api-test-tool: Tests now specify a timeout scale, not a timeout.

Makes it easier to change the default timeout in one place.

* ledger-api-test-tool: Increase timeouts on slow tests.

These tests produce a lot of volume and can make CI flaky.

* ledger-on-sql: Only tear down PostgreSQL in tests.

For other databases, we just create a new file for each test case.

* ledger-on-sql: Reduce the log output in tests.

* ledger-on-sql: Use a separate connection pool for Flyway when possible.

Apparently `setMaximumPoolSize` doesn't really have the desired effect
after the connection pool has already been used. The new test case will
be flaky if we process more than one commit in parallel.

For SQLite, it seems to be OK.

* ledger-on-sql: Use a separate connection pool for Flyway with SQLite.

Except in memory.

* ledger-on-sql: Use a separate PostgreSQL database for each test.

Because performance, innit. Don't have to tear them down.

* ledger-on-sql: Move the `Queries` values inside the `RDBMS` objects.

* ledger-on-sql: Go into even more detail about pool size hijinks.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 16:16:23 +00:00
Samir Talwar
0d5ae27e04 ledger-api-test-tool-on-canton: Upgrade to Canton v0.8.0. (#4244)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 15:10:05 +00:00
Samir Talwar
02b1c8be59 ledger-api-test-tool: Run scalafmt. (#4241)
It affects basically every file, so let's do it all at once.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 13:29:21 +00:00
Samir Talwar
ebfb0c4a97 sandbox: On initialization error, actually print the error and stop. (#4239)
Three problems here:

1. On failure, the Sandbox didn't close down the metrics reporting
thread, meaning the application would hang.
2. If there's a failure:
  a. we didn't print it, and
  b. we didn't exit with the correct status code.

These are now handled.

CHANGELOG_BEGIN
- [Sandbox] On initialization error, report the error correctly and exit
  with a status code of 1. Previously, the program would hang
  indefinitely. (This regression was introduced in v0.13.41.)
CHANGELOG_END

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-28 12:25:29 +00:00
Andreas Herrmann
74984559e8 Test triggers in scenarios (#4233)
* Bazel test for trigger scenario

* daml-triggers: Allow testing trigger rules in scenarios

CHANGELOG_BEGIN
- [DAML Triggers - Experimental] DAML triggers can now be tested in
scenarios. Specifically, a trigger's ``rule`` can be executed in a
scenario and assertions performed on the emitted commands.
CHANGELOG_END

* Allow assertions on create commands

CHANGELOG_BEGIN
* [DAML stdlib] Add `CanAbort` instance for `Either Text`.
CHANGELOG_END

* Add convenience to construct ACS for testRule

* Add assertions for exercise and exerciseByKey

* fix assert message

* Test assertExercise(ByKey)Cmd

* unpackCommands --> flattenCommands

* Add API documentation

* Document that command ids start from "0"

* generalise command assertions to CanAbortm

* export ACSBuilder type

* Better haddocs for trigger command assertions

* explicit let

* ./fmt.sh

* Fix runfiles on Windows

* Add reference to Bazel issue

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-28 11:50:46 +00:00
Moritz Kiefer
54ace2ce06 Initialize package db in damlc test (#4240)
fixes #3436

changelog_begin

- [DAML Compiler] ``damlc test`` now initializes the packagedb automatically which
  means that it will work on projects that declare custom
  ``dependencies`` in ``daml.yaml`` without having to call ``damlc
  init`` first.

changelog_end
2020-01-28 11:22:25 +00:00
associahedron
02b531d254 Preserve class method names during LF conversion. (#4230)
* Preserve class method names during LF conversion.

This PR changes the LF conversion to pick names for dictionary
fields that will be easier to reconstruct as typeclasses
later.

For superclasses this uses "s_1", "s_2", "s_3" and so on,
and for methods this uses "m_foo", "m_bar", "m_baz" and
so on. (Is this enough C++?)

In the future we might want to distinguish between methods
that are mandatory and methods that are optional ... but
I think this should be good enough for now.

This PR also adds a test.

Fixes #4216

changelog_begin
changelog_end

* lint

* Run test in LF 1.7 as well

* Add test for superclass field.
2020-01-28 10:55:02 +00:00
Stefano Baghino
6c094b3f95
Adopt trailingCommas option in scalafmt (#4214)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-28 11:23:47 +01:00
Moritz Kiefer
8a67aff8de Remove position-mapping tests (#4238)
They have been migrated to `ghcide` in
https://github.com/digital-asset/ghcide/pull/388 which is a much more
sensible place for them to live given that the code is in that repo.

changelog_begin
changelog_end
2020-01-28 10:08:15 +00:00
Remy
f73894dc25
Engine: drop useless EContractId Ast node (#4236)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-28 10:41:30 +01:00
Shayne Fletcher
9c0fe75b94
Add daml2ts to the SDK release tarball (#4234)
changelog_begin
- daml2ts now included in the SDK
changelog_end
2020-01-27 13:18:05 -05:00
Shayne Fletcher
a56caa0190
daml2ts : Explain DAML/TypeScript mappings (#4196)
changelog_begin
changelog_end

Co-authored-by: Shayne Fletcher <shayne.fletcher@digitalasset.com>
2020-01-27 12:25:13 -05:00
nickchapman-da
60b0c4a5d2
check and reject cyclic DAML-LF type synonym definitions (#4228)
changelog_begin
changelog_end
2020-01-27 16:49:37 +00:00
Moritz Kiefer
3245a7b587 Fail when a data-dependency fails to compile (#4219)
* Fail when a data-dependency fails to compile

Previously, we emitted errors but just kept on going which sometimes
kinda works but is certainly not the right thing to do.

Now we just crash and abort compilation.

changelog_begin
changelog_end

* why is windows

* undo crap
2020-01-27 15:25:47 +00:00
Moritz Kiefer
40292c2b6d
Avoid including daml-stdlib and daml-prim twice (#4222)
This fixes #4114 and cleans up the situation around
`data-dependencies` and `dependencies` as described in #4218.

There is still more work to be done here mostly around ironing out all
the edge cases and producing useful error messages instead of
silentely doing the wrong thing but I’ll leave that to a separate PR.

To test this, I’ve fixed the packaging tests to no longer deduplicate
package ids (which means they would return the wrong number if
daml-prim ends up in there twice) and I addded a test where we have 3
projects:

- `lib`
- `a` which depends on `lib`
- `b` which depends on `lib` via `dependencies` and 'a' via
  `data-dependencies`.

changelog_begin
changelog_end
2020-01-27 14:56:12 +01:00
associahedron
830c2c65f5 Add missing CurrentSdk prefixes in data-dependencies (#4220)
* Expose scenarios in data-dependencies.

Also add some type signatures.

changelog_begin
changelog_end

* Add missing prefixes in data-dependencies
2020-01-27 12:57:35 +01:00