Commit Graph

5108 Commits

Author SHA1 Message Date
Moritz Kiefer
5936644970
Bump windows cache (#7212)
changelog_begin
changelog_end
2020-08-24 18:19:28 +02:00
Leonid Shlyapnikov
3ecfcc918c
Adding package-max-inbound-message-size (#7147)
* Adding `package-max-inbound-message-size`

this is to allow separate configuration settings for command submission
and package management ledger clients

* Fixing formatting

* Updating docs

changelog_begin

[JSON API] Adding `--package-max-inbound-message-size` command line option.
Optional max inbound message size in bytes used for uploading and downloading package updates. Defaults to the `max-inbound-message-size` setting.

changelog_end

* Addressing code review comments
2020-08-24 14:55:55 +00:00
Gary Verhaegen
f54e193df6
add visibleByKey (#7146)
Adds a Boolean check for the existence of an active contract with a
given key (that is visible with the current set of authorizations), and
explain `lookupByKey` in terms of it to hopefully clarify the relationship
between `lookupByKey` and `fetchByKey`.

Fixes #7143; see that for more information.

CHANGELOG_BEGIN

- [DAML Standard Library] Added a new function `visibleByKey`
  which, given a contract key, returns True if the current context can
  see an active contract with that key.

CHANGELOG_END
2020-08-24 15:04:05 +02:00
Moritz Kiefer
c997f5d145
Fix handling of transient contracts in extractor (#7210)
* Fix handling of transient contracts in extractor

fixes #7201

As described in the issue, the problem was that we completely ignored
the order of events and always processed exercises before creates
which falls apart as soon as they are in the same transaction.

I still think that in the current state doing creates before exercises
would be fine but this seems like the kind of assumption that breaks
very quickly during refactoring (e.g. let’s say you start creating
foreign key references) so instead this PR fixes this properly by
iterating over the transaction tree in pre-order.

This turns out to be a bit more complex than I expected it to be since
we filter the tree on the client side and afterwards we have lost
enough information so that we cannot traverse the tree instead
anymore. Therefore, the “tree” type in extractor now contains a list
of events in pre-order and a comment that it’s not actually a valid
tree or subtransaction.

changelog_begin

- [Extractor] Fix a bug that resulted in transient contracts (created
  and archived within the same transaction) to not be displayed as
  archived. See #7201 for details.

changelog_end

* Update extractor/src/main/scala/com/digitalasset/extractor/ledger/types/TransactionTree.scala

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* Suffix with template name

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2020-08-24 14:43:24 +02:00
Remy
0e9616f784
LF: move the type ordering form interpreter to language package (#7203)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-24 07:18:41 +00:00
Remy
7f60560593
LF: make QualifiedName and Identifier Ordered (#7199)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-21 17:39:51 +02:00
Andreas Herrmann
037cf11710
Add :json command to DAML REPL (#7202)
* REPL test `:json` command

changelog_begin
changelog_end

* Implement :json command in DAML REPL

changelog_begin
- [DAML REPL] You can now convert DAML expressions to JSON in the DAML
  REPL using the meta-command ``:json`` for example ``:json [1, 2, 3]``.
changelog_end

* Extend DAML REPL documentation

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-21 14:53:06 +00:00
Stefano Baghino
ac86a9c5a3
Refactor SQLLedger initialization routine (#7200)
* Refactor SQLLedger initialization routine

Small refactoring to make initialization a bit more readable. Performed
while moving forward with the addition of the participant identifier to
the parameters table (so a few minor details have leaked into this PR).

changelog_begin
changelog_end

* Fix compilation errors

* Address https://github.com/digital-asset/daml/pull/7200#discussion_r474630880

* Fix test, lower test logging noise
2020-08-21 15:29:49 +02:00
Remy
6371f51930
PartialTransaction: replace nested Eithers by a custom ADT (#7191)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-21 10:16:36 +02:00
Stephen Compall
2369a3bbae
use sandbox-style flyway migrations for trigger service (#7092)
* depend on flyway in trigger service; duplicate FlywayMigrations from sandbox

* bring in postgres/h2 detection, other data-source elements needed; use scalalogging

* use DataSource instead of DriverManager for DbTriggerDao, pass to flyway too

- initialization order isn't quite right yet

* hardcode postgres in triggers flyway path

* found an init order that doesn't fail in tests

* better pools for production, and don't forget to close them

* move most init SQL to first migration file

* make logHandler implicit again

- seems to have been an accidental implication of #6334

* move remaining create to V1__Init.sql

* use Prod poolSize

* prove that PostStop signals are never seen (in tests)

* move all the shutdown to Stop message

* remove dead code in DbFlywayMigrations

* always "initialize" database

CHANGELOG_BEGIN
- [Trigger Service] The database format has changed to allow migrations in
  future releases.  Databases are always initialized or migrated to the current
  version on start, so use of ``--init-db`` is no longer required.
  See `issue #7092 <https://github.com/digital-asset/daml/pull/7092>`__.
CHANGELOG_END

* it should always be obvious when there is no escape

* remove logs from experiments

* remove receiveSignal PostStop

* remove unused imports
2020-08-20 18:52:37 +00:00
Gary Verhaegen
6d1adee92f
push script-runner and trigger-runner to Artifactory (#7196)
I have created the corresponding user and repositories on Artifactory,
and tested the `curl` command manually. I'll add the corresponding
credentials to Azure once this is approved.

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-20 19:11:27 +02:00
Andreas Herrmann
9f2367f6cc
IDE distinguish scripts and scenarios (#7197)
* IDE distinguish scripts and scenarios

The code lens above DAML script definitions now reads `Script results`
instead of `Scenario results`.

This also adds DAML Script test cases to the lsp-tests.

The functionality for testing scenario and script code-lenses in the
`Utils` module is identical. However, this adds dedicated definitions
for scripts to clearly distinguish the scenario and script test-cases.

changelog_begin
changelog_end

* Add sum type to distinguish Scenario and Script

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-20 16:36:23 +00:00
Gary Verhaegen
2871ab7a11
add Windows tips to release instructions (#7180)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-20 17:26:21 +02:00
Samir Talwar
30564a7c90
ledger-api-client: Add test cases for LedgerClient. (#7195)
* ledger-api-client: Add integration tests for the simple stuff.

* sandbox-common: Make `SandboxFixtureWithAuth` a mixin.

This makes it useful with `SandboxNextFixture` as well as
`SandboxFixture`.

Also, add types to non-private fields and methods, and make more fields
protected rather than public.

* ledger-api-client: Add tests to make sure the token is passed through.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox-common: Tokens are for auth, not auth.
2020-08-20 15:00:32 +00:00
Remy
ba74146c10
Speedy: shortcup failure in recursive data structure equality (#7083)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-20 16:15:11 +02:00
Stefano Baghino
62fcd16de2
Factor parameters table access out of JdbcLedgerDao (#7194)
* Factor parameters table access out of JdbcLedgerDao

changelog_begin
changelog_end

* Make ParametersTable package private
2020-08-20 15:46:51 +02:00
Remy
17abe2f4cd
DAML for PostgreSQL: prevent dev version of DAML-LF (#7176)
Add a hidden flag to enable dev mode for testing purpose only.

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-20 12:15:38 +02:00
Samir Talwar
5a228354bc
sandbox: Use a different H2 database for every test. (#7184)
* sandbox: Use a different H2 database for every test.

Previously tests would share an index database, but not a ledger
database, if not explicitly overridden. This is _very bad_.

Fortunately all tests did specify a backend explicitly, but it shouldn't
be necessary.

I caught this while writing a new test using `SandboxNextFixture`.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox-common: Simplify SandboxBackend a little.
2020-08-20 10:03:29 +00:00
Andreas Herrmann
3e6ffcad4b
Refactor IdeClient.submit - avoid Future.fromTry (#7179)
The code was using `result: Try` combined with `Future.fromTry` to
signal success or failure. However, the code also used a `try { }
finally { }` block to ensure cleanup. Wrapping the whole block in a
`Future { }` and simply throwing exceptions instead of setting `result =
Failure( )` has the same effect, but avoids the nesting of `try { }` and
`Future.fromTry`.

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-20 09:01:12 +00:00
Moritz Kiefer
255832cefc
Fix failed contract lookups in scenario runner (#7190)
lookupContract catches the exception which results in us continuing
and eventually running into a null-pointer exception.

This is another reason why we should enable the NonUnitStatements wart
but I’ll leave that for a separate PR (last I tried the scenario
service resulted in a ton of false positives).

fixes #7185

changelog_begin
changelog_end
2020-08-20 09:49:03 +02:00
Remy
cc6e8fa901
Engine: submit fail nicely when transaction output cannot be serialized (#7189)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-20 09:20:50 +02:00
Gary Verhaegen
29dfd2d227
fix typo in date docstring (#7187)
As reported by Gyorgy Balazsi on [the forum].

[the forum]: https://discuss.daml.com/t/day-parameter-overflow-for-the-date-function-simple-leap-year-handling/1068/7?u=gary_verhaegen

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-20 09:18:18 +02:00
anthonylusardi-da
3d067b5fa9
Fix some screen widths not showing the content menu collapse/expand icon (#7138)
* Fix some screen widths/zoom levels not showing the content menu collapse/expand icon

* CHANGELOG_BEGIN
[Docs] Fix some screen widths not showing the content menu collapse/expand icon
CHANGELOG_END
2020-08-19 16:37:47 -04:00
azure-pipelines[bot]
6dd542c2be
update compat versions for 1.5.0-snapshot.20200818.5027.0.1b33d374 (#7177)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-08-19 21:42:17 +02:00
Remy
e54c182032
Engine: move check of valide packages inside MutableCompiledPackages (#7186)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-19 19:37:53 +02:00
Gary Verhaegen
94f2a10e70
fix auto release pr message (#7178)
Because the build for the automated release PR is triggered "manually"
by the Azure cron (through the GH API), as far as the build of that PR
is concerned, this is a random commit build, not a PR build, and thus it
doesn't have a PR number.

This works around that by getting the corresponding PR number from the
GH API.

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-19 13:58:56 +02:00
Samir Talwar
c8b10091db
participant-integration-api: Never use a delay of zero. (#7175)
* participant-integration-api: Never use a delay of zero.

If `akka.pattern.after` is passed a delay of zero, it will execute the
body synchronously, potentially leading to a stack overflow error.

This error was observed in tests.

CHANGELOG_BEGIN
CHANGELOG_END

* timer-utils: Add tests for Delayed.Future.

* timer-utils: Add tests for RetryStrategy.

* timer-utils: Remove duplication in RetryStrategy tests.

* timer-utils: Allow for more wiggle room in the RetryStrategy tests.

* timer-utils: Fail after retrying the correct number of times.

* timer-utils: Ensure we don't overflow the stack in RetryStrategy.

* timer-utils: Reject a negative number of retry attempts.
2020-08-19 10:43:31 +00:00
azure-pipelines[bot]
f69006264e
rotate release duty after 1.5.0-snapshot.20200818.5027.0.1b33d374 (#7174)
@garyverhaegen-da is taking care of 1.5.0-snapshot.20200818.5027.0.1b33d374 (#7173), so they get pushed back to the end of the line.

Please do not merge this before #7173.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-08-19 12:36:45 +02:00
Samir Talwar
d0ffc2c8c5
participant-integration-api: Factor out similar behavior in the admin service implementations. (#7171)
* participant-integration-api: Factor out `pollUntilPersisted`.

CHANGELOG_BEGIN
CHANGELOG_END

* participant-integration-api: Flatten `flatMap`s in admin services.

* participant-integration-api: Use the `FutureConverters` implicits.

* participant-integration-api: Inline `waitForEntry` again.

* participant-integration-api: Commas are great.

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* participant-integration-api: Pass the EC to `pollUntilPersisted`.

* participant-integration-api: Extract out more into SynchronousResponse.

* participant-state-index: Stream entries from an optional offset.

Makes them consistent with configuration entries.

* participant-integration-api: Move the ledger end into the strategy.

* participant-integration-api: Use SynchronousResponse everywhere.

* participant-integration-api: Pass inputs to SynchronousResponse#submit.

* participant-integration-api: Add docs to SynchronousResponse.

* participant-integration-api: Add a changelog entry.

CHANGELOG_BEGIN
- [Ledger API] The ConfigurationManagementService will now use the same
  description as other services in case of certain errors. The error
  status codes have not changed.
CHANGELOG_END

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-08-19 10:24:48 +00:00
Andreas Herrmann
a080b47e34
Reset the machine on every submit in IdeClient (#7172)
* Test-case for operation after submitMustFail

Currently this triggers
```
BErrorClient (ClientIOError (GRPCIOBadStatusCode StatusUnknown (StatusDetails {unStatusDetails = ""})))
```

changelog_begin
changelog_end

* Reset the machine after every submit in IdeClient

This ensures that the machine is not stuck in a brocken state after a
`submitMustFail` completed.

changelog_begin
changelog_end

* Machine cleanup in finally block

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-19 10:26:15 +02:00
azure-pipelines[bot]
ed73352843
release 1.5.0-snapshot.20200818.5027.0.1b33d374 (#7173)
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.

@garyverhaegen-da is in charge of this release.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-08-19 10:09:16 +02:00
Remy
1b33d3742a
Revert "Sandbox: add a development mode (#7127)" (#7168)
This reverts commit 87a0ee9383.

CHANGELOG_BEGIN

WARNING remove the changelog entry from 87a0ee93 and starting by :
 "[Sandbox] By default Sandbox rejects the development versions of ..."

CHANGELOG_END
2020-08-18 19:17:56 +02:00
Gary Verhaegen
5b2319e137
multistep macos setup (#5768)
multistep macos setup

This updates the macOS node setup instructions to avoid repeating
identical work and network traffic across all machines through
initialization by building a "daily" image with all the tools and code
we need.

CHANGELOG_BEGIN
CHANGELOG_END

* Fix 3-running-box to remount nix partition

* updated scripts to use multi-step process

* add copyright notices

Co-authored-by: nycnewman <edward@digitalasset.com>
2020-08-18 16:01:02 +02:00
Gary Verhaegen
4f2bd150b9
focus SDK install page (#7131)
- Put Windows first, and add a direct link to the installer.
- Move the manual step to a separate page.
- Link to AdoptOpenJDK as they have by far the simplest download page.

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-18 15:34:28 +02:00
Samir Talwar
aa9e90d7cb
ledger-api-test-tool: Upload DARs, rather than expecting them. (#7163)
* ledger-api-test-tool: Upload DARs, rather than expecting them.

This allows us to avoid knowing the DARs.

CHANGELOG_BEGIN
- [Integration Kit] When running the Ledger API Test Tool, the required
  DAR files are now uploaded to the ledger automatically before running
  the tests. You no longer need to upload these DARs before running the
  test tool.
CHANGELOG_END

* ledger-api-test-cool: Factor out configuration construction.

* ledger-api-test-tool: Upload DARs to all participants.

And don't prime Canton.

* docs: Document that the Ledger API Test Tool uploads DARs by itself now.

* ledger-api-test-tool-on-canton: Remove unused code in the runner.
2020-08-18 13:25:02 +00:00
Moritz Kiefer
820c9ff844
Remove damlLfInstantiate in favor of TypeCon.instantiate (#7166)
fixes #2506

Judging from the issue this was originally introduced to workaround a
bug. I couldn’t actually track down what that bug was but at this
point they are identical so no point keeping this around.

changelog_begin
changelog_end
2020-08-18 12:15:12 +00:00
Samir Talwar
f0e09c54b1
DAML on SQL: Renaming to "DAML for PostgreSQL", Markdown README, and further tweaks. (#7160)
* daml-on-sql: Formatting fixes for the README.

* daml-on-sql: Rename to "DAML on PostgreSQL" within the README.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-on-sql: Format README to make Markdown conversion nicer.

Markdown conversion:

    pandoc --reference-links --reference-location=section -t markdown+backtick_code_blocks-fenced_code_attributes -o ledger/daml-on-sql/README.md ledger/daml-on-sql/README.rst && sed -i 's/\\//g' ledger/daml-on-sql/README.md

* daml-on-sql: It's "DAML for PostgreSQL", not "on PostgreSQL" now.

* daml-on-sql: Generate a Markdown README with a script.

This is easier to parse by human beings than the reStructuredText.

* daml-on-sql: Reflow the README and manipulate whitespace a little.
2020-08-18 11:50:45 +00:00
Samir Talwar
5d1ef35b09
kvutils: Move execution contexts to the submission methods. (#7071)
* kvutils: Move execution contexts to the submission methods.

Previously, the execution contexts were provided at the constructor
level, which meant the submission validator and associated components
would use the resource acquisition execution context. This context is
intended purely for resource acquisition; if we need an execution
context for processing data, we have much more control when we
explicitly create one.

Implementors will still need to avoid passing an execution context
implicitly into the constructors of their own components, instead
favoring an explicit one that is to be used during `commit` or reading
events.

CHANGELOG_BEGIN
- [Integration Kit] kvutils now expects execution contexts to be passed
  in to the various ``SubmissionValidator``, ``LedgerStateAccess``, and
  ``LedgerStateOperations`` methods. This is a source-breaking change.

  Instead of providing an execution context implicitly to your ledger
  implementation, you are encouraged to construct the relevant contexts
  for different operations explicitly. Please refer to the open-source
  implementations as a reference.
CHANGELOG_END

* ledger-on-memory: Don't use the acquisition context for commits.

Instead, use the materializer execution context; it may not be the best
choice but at least it's appropriate.

* kvutils: Standardize method parameter formatting.

* kvutils: Provide old state  APIs as `LedgerStateAccess.v1_4`.
2020-08-18 11:50:33 +00:00
Sofia Faro
e95cd28292
Put sandbox output to stdout in tests. (#7164)
This gives us visibility into issues w/ sandbox in the
tls/deployment/ledger tests. This change only affects test
output.

changelog_begin
changelog_end
2020-08-18 12:35:55 +01:00
Moritz Kiefer
0e833c02a4
Enable NonUnitStatements for repl service (#7161)
changelog_begin
changelog_end
2020-08-18 10:58:00 +02:00
Moritz Kiefer
a8ce8f803b
Enable NonUnitStatements wart for daml-script (#7159)
Didn’t catch too much but at least one completely useless statement.

changelog_begin
changelog_end
2020-08-18 09:42:24 +02:00
moritzkiefer-da
eed3e3c2dc
Require 2 CPUs for repl integration tests (#7158)
We have seen timeouts on port file waits occasionally.

Those timeouts are already very high so reducing load is a more
sensible option than bumping the timeout.

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-08-17 16:49:35 +00:00
Andreas Herrmann
6969aae7d5
Remove duplicated test-case (#7157)
changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-17 15:49:02 +00:00
Samir Talwar
7e646287bd
participant-integration-api: Drop unused participant_id columns. (#6874)
We don't use the `participant_id` column in either the
`configuration_entries` or `party_entries` tables of the index.

This does not change the partcipant state API, so ledger integrators
should not need to make changes.

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-17 17:29:24 +02:00
Moritz Kiefer
d4d0419c64
Use TO_TEXT_CONTRACT_ID in Show instance of ContractId (#7153)
fixes #7114

This PR changes the Show instance of ContractId and flips the switch
on triggers and DAML Script to run in off-ledger mode.

It also adds a test that for DAML Script we actually get back the
correct contract id.

There is a bit of a design decision here in how we want to print
contract ids, so let me list the options I considered. $cid will stand
for the actual cid and all options are wrapped in markdown inline
code.

1. `"$cid"`. Indistinguishable from string. Suggests that there might
be an IsString instance for ContractId.
2. `<$cid>`. Matches the dummy `<contract-id>` but it’s not a dummy so
I don’t think matching that is benefitial.
3. `$cid`. Easy to spot (contract ids start with # and have no
spaces), clearly not a string but might look slightly weird.

changelog_begin

- [DAML Script/DAML Triggers] When using DAML-LF 1.dev, the `Show` instance of `ContractId` will now display the actual contract id instead of a dummy `<contract-id>` value. Note that this only applies to DAML Script and DAML Triggers not to ledger code.

changelog_end
2020-08-17 17:06:24 +02:00
Andreas Lochbihler
74986b9035
Causality and interoperability documentation (#6920)
CHANGELOG_BEGIN
Add documentation for the ordering guarantees on the Ledger API
and for causal consistency with interoperable ledgers
CHANGELOG_END

Co-authored-by: Ognjen Maric <ognjen.maric@digitalasset.com>
Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
2020-08-17 16:15:55 +02:00
Andreas Herrmann
2903987a61
Handle all scenario results in DAML Script IDE client (#7154)
* Handle all scenario results in DAML Script IDE client

changelog_begin
changelog_end

* Address review comment

Consistent error messages

* DAML Script getTime in choice

* Handle `SResultNeedTime` in DAML Script IdeClient

* fmt

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-17 13:46:41 +00:00
Martin Huschenbett
37d74287f1
DAML Engine: Use while loop instead of for loops (#7126)
This PR replaces a few `for (... to/until ...)` loops with `while`
loops which explicitly maintain their counter. In my benchmarks, this
has lead to a 3% performance improvement. Since the affected functions
are the ones used for applying arguments to functions, similar
improvements should be observable regardless of the specific benchmark.

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-17 12:29:33 +00:00
Gary Verhaegen
51e7c88bf5
announce release rotation on Tuesday (#7151)
It's been pointed out to me that some people actually plan their work,
and for them, it would be useful to know at least a day in advance that
they will be doing a release. This PR attempts to accommodate that.

Note: this will run as a new, separate cron. I'll do the Azure setup
after this gets approved.

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-17 13:42:35 +02:00
Gary Verhaegen
a3419f9db8
remove dad-add-soirce-dep.sh (#7152)
This file recently came up in a PR to fix shebang lines, and I realized
it's not been touched since open-sourcing and is not referenced
anywhere in the repo.

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-17 12:59:59 +02:00