Commit Graph

6113 Commits

Author SHA1 Message Date
Gary Verhaegen
532f996f12
ci: clean-up hard drive more often (#8582)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-20 17:22:53 +01:00
Remy
ff2beb6d4a
LF Encoder: fix encoding of LF 1.11 (#8575)
- fix encoding of generic comparisons
- fix encoding of choice observers

CHANGELOG_BEGIN
CHANGELOG_ENd
2021-01-20 17:10:02 +01:00
Moritz Kiefer
f2fb9c9415
Release SDK 1.9.0 (#8572)
changelog_begin
changelog_end
2021-01-20 16:42:38 +01:00
Robin Krom
aa8c1f8fee
damlc: incremental package db initialization (#8541)
* damlc: incremental package db initialization

We keep a hash over all dependencies of a project in the package
database metadata and only recompute the package database if a
dependency changes, i.e. the computed hash changes.

Fixes #4413.
Fixes #8409.

CHANGELOG_BEGIN
CHANGELOG_END

* using Fingerprint

* added tests

* format

* use SdkVersion instead of hardcoded version

* added a reference in tests

* factored out project file template
2021-01-20 15:52:37 +01:00
Moritz Kiefer
a66a3177dd
Bump Maven timeouts and retries (#8578)
* Bump Maven timeouts and retries

Chosen fairly arbitrarily, I don’t have a good idea what sensible
values are but given that we keep running into issues the current ones
are apparently too low.

changelog_begin
changelog_end

* sync up comment with code

changelog_begin
changelog_end

* s/f/0/

changelog_begin
changelog_end
2021-01-20 15:44:28 +01:00
Sofia Faro
c1a736efd1
Add choice observers in the ledger model. (#8576)
* Add choice observers in the ledger model.

To keep the distinction between "observer" and "choice observer"
clear I'm trying to always refer to the former as "contract observer".

The section on divulgence should probably be reworked to favor choice
observers at some point, but I'm not sure how to do that.

changelog_begin
changelog_end

* s/observer/contract observer

* Separate out the choice observer case of informees

* apply suggestion from @oggy-

* specifying choice observers in DAML contract model
2021-01-20 13:16:45 +00:00
mziolekda
b184a6b79a
Bumping the release by one PR (#8577)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-20 13:11:02 +00:00
mziolekda
58ef725aa2
Log workflow id in combination with correlation id to aide troubleshooting (#8543)
* use logging context for additional data

* log workflow id in a cheaper debug message

* revert the logging in the KeyValueParticipantStateWriter
CHANGELOG_BEGIN
CHANGELOG_ENd
2021-01-20 11:41:34 +00:00
azure-pipelines[bot]
73903773f5
release 1.10.0-snapshot.20210119.6103.0.cdcf090b (#8567)
* release 1.9.0-snapshot.20210119.6103.0.cdcf090b

This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.

@nickchapman-da is in charge of this release.

CHANGELOG_BEGIN
CHANGELOG_END

* Bump version, changing prefix from 1.9 to 1.10

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-01-20 11:07:11 +00:00
daravep
2a38aeb0e4
Moved default timeout definition up in callstack (#8551)
The default timeout of 250ms turned out to be problematic for Canton.
Therefore, I'm moving the hard coded 250ms up such that we can override
it from within the Canton code. Other users shouldn't be affected by the
change. Therefore, this is just an internal code change.

Additionally, I've added two new performance targets for the performance
envelope test.

Related to https://github.com/DACH-NY/canton/issues/4996

CHANGELOG_BEGIN
- Made the database connection timeout configurable.
CHANGELOG_END
2021-01-20 09:34:57 +01:00
tudor-da
cdcf090b02
Pass MetricsRegistry to commitStrategySupport from IntegrityChecker (#8508)
* Pass MetricsRegistry to commitStrategySupport from IntegrityChecker

CHANGELOG_BEGIN
CHANGELOG_END

* Address review comments
2021-01-19 20:25:52 +00:00
Gary Verhaegen
01f11109fa
faster disk cleanup check (#8565)
I can't reproduce the `chmod` failure and have no clue how to fix it, so
for now let's just hope it's not too frequent.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-19 19:50:44 +00:00
Moritz Kiefer
fbea4e6ba8
Release 1.9 RC 2 (#8561)
This includes a bugfix to sandbox classic’s default seeding mode and a
kvutils fix.

changelog_begin
changelog_end
2021-01-19 19:32:05 +00:00
Gary Verhaegen
2f01a1c034
reduce ci/cron noise (#8563)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-19 19:12:30 +00:00
Gary Verhaegen
3641751571 ci/unix: better cache size reporting (#8556)
Current reports look like:

```
Disk cache small enough:\n20G/home/vsts/.bazel-cache
```

because `echo` does not convert `\n`. An alternative would be to replace
`echo` with `printf`, but I have had enough issues with
subshells-in-strings lately that I prefer just avoiding them when
possible.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-19 18:29:07 +01:00
Remy
d5cf0ea4ec
sandbox-classic: test the default seeding is not changed (#8560)
regression test for #8555.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-19 17:12:12 +00:00
Remy
e130d8bf03
Sandbox-classic: revert back the change of default contract ID seeding (#8555)
We revert the change of default contract ID seeding that was
introduced in #7945.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-19 17:07:59 +01:00
azure-pipelines[bot]
78f878fc10
update compat versions for 1.8.1 (#8540)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-01-19 16:37:46 +01:00
Sofia Faro
37dcefe530
Update LF spec for exceptions. (#8530)
* Update LF spec for exceptions.

changelog_begin
changelog_end

* typo
2021-01-19 10:14:33 +00:00
azure-pipelines[bot]
1e4e212bcb
update NOTICES file (#8542)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-01-19 08:39:48 +00:00
Remy
441f503a95
LF: Flatten ContractInstance inside NodeCreate (#8526)
The ContractInstance is a concept a bit off. For instance it is not
clear why `agreementText` is more important for a contract that the
signatories, the stackholder or the key. In this PR, we get ride of
the ContractInstance inside LF code, except from the Engine/Ledger API
(com.daml.lf.engine.ResultNeedContract$)

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-18 17:29:32 +01:00
fabiotudone-da
720cb71fb4
Bless snapshot release 1.8.1-snapshot.20210115.5842.0.59f5d407 as release 1.8.1 (#8537)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-18 15:08:19 +01:00
fabiotudone-da
1605f73737
KV fix: add exercise key to inputs (forward-ports #8529) [KVL-797] (#8538)
* Fix InputsAndEffects: add exercise key to inputs (forward-ports #8529)

CHANGELOG_BEGIN
[KV] Fix: add exercise key to submission inputs
CHANGELOG_END

* Fix Scala 2.13 build
2021-01-18 15:05:55 +01:00
fabiotudone-da
61072f948d
Improve release/RELEASE.md (#8535)
* Improve release/RELEASE.md

CHANGELOG_BEGIN
CHANGELOG_END

* Update release/RELEASE.md

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* Address review comments

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-01-18 15:05:38 +01:00
Andreas Herrmann
199c52fd79
Don't always redirect to /login automatically (#8532)
* Don't always redirect to /login automatically

The redirect mode can be configured to never redirect, always redirect,
or redirect based on the request type (redirect for text/html).

In case of no redirect the auth middleware client will reply with 401
Unauthorized with a custom WWW-Authenticate challenge to login on the
auth middleware.

* Make login to redirect configurable on trigger service

By default the trigger service will redirect for HTML requests and not
redirect for JSON requests. The test suite uses automatic redirect as
the OAuth2 test server works without user interaction.

changelog_begin
changelog_end

* Preserve path and query in authMiddlewareUri

This is necessary if the auth middleware lies behind a reverse proxy
with a path prefix or a similar setup.

* Bump default auth middleware login timeout

One minute was to short for a login cycle that requires manual user
input.

* Set token cookie properties

`path = "/"` is required so that the `/login` endpoint can reliably
override the cookie value for other endoints such as `/cb` or
`/v1/triggers`.

* Test redirectToLogin modes

* Redirect on HTML

https://github.com/digital-asset/daml/pull/8532#discussion_r559368335

* Use pass/reject in onRedirectToLogin

8db2bff9af (r559370308)

* default login timeout 5min

https://github.com/digital-asset/daml/pull/8532#discussion_r559535511

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-01-18 14:03:39 +00:00
Robin Krom
c5852e8cd4
fix: project parties are not required (#8524)
* fix: project parties are not required

This fixes a bug in daml assistant. `daml deploy` would fail if there
were no project parties listed in the daml.yaml project config file. Now
project parties are not required anymore.

Fixes #8487.

CHANGELOG_BEGIN
CHANGELOG_END

* added an integration test
2021-01-18 10:47:07 +01:00
fabiotudone-da
1c8212946b
Release 59f5d40754 1.8.1-snapshot.20210115.5842.0.59f5d407 (#8533)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-15 20:25:44 +01:00
Gary Verhaegen
8350224aef
ci: fix linux cleanup for spaces (#8531)
I hate spaces.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-15 17:01:44 +00:00
Robin Krom
55b4a3d433
assistant: move cache to new cache folder. (#8522)
We use the new cache folder specified with `DAML_CACHE` defaulting to
`.cache/daml` as a new cache folder for the assistant to store available
versions etc.

This does not migrate an existing cache folder and just starts a new
one.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-15 17:24:30 +01:00
Gary Verhaegen
36175c0a77
ci/cron: remove hidden.json (#8525)
I discovered yesterday that the `snapshots.json` (and actually also the
`versions.json`) file is no longer purely internal to the docs process,
as it was meant to be, but is now depended upon by the assistant. This
means the renaming from `snapshots.json` to `hidden.json` cannot happen,
and we reverted that yesterday in #8513 (& #8514), though that was done
in a bit of a hurry. This PR aims at cleaning up the resulting mess and
achieve a better long-term end state.

I will be manually removing the `hidden.json` file as soon as this is
merged, so that nothing ends up depending on _that_. There is no
occurrence of `hidden.json` outside this docs cron so hopefully this
works out.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-15 16:04:54 +00:00
azure-pipelines[bot]
20659ade04
update compat versions for 1.9.0-snapshot.20210113.6060.0.9ed787cb (#8495)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-01-15 16:45:51 +01:00
Gary Verhaegen
cf949b6bae
ci: gc linux cache (#8527)
This is the equivalent of #8515 for Linux. There was some concern that
`bazel` would be upset at having that cache removed, so I spent a fair
amount of time trying to break it (on a Linux VM, as for some reason
`bazel` chooses not to use `~/.cache` on macOS). I could not make
`bazel` unhappy by deleting the whole thing. Deleting random files,
however, did end up producing error messages along the lines of:

```
$ bazel build //...
FATAL: corrupt installation: file '/home/vagrant/.cache/bazel/_bazel_vagrant/install/73d06d52dbf3a8e6ed43f5bf5f115eb0/embedded_tools/src/BUILD' is missing or modified.  Please remove '/home/vagrant/.cache/bazel/_bazel_vagrant/install/73d06d52dbf3a8e6ed43f5bf5f115eb0' and try again.
```

which suggest busting the entire thing as a solution, so I think we're
safe here.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-15 16:29:52 +01:00
Remy
f07c94a936
LF: drop useless function getTemplateId from preprocessor. (#8521)
Use instead directly the GenNode's method templateId added in #6752.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-15 13:10:39 +00:00
Gary Verhaegen
756b6ce942
update compiler header (#8520)
Channelling my inner Dan.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-15 12:40:19 +00:00
Robin Krom
32ca8fb831
telemetry: use an extra cache directory for telemetry (#8439)
* telemetry: use an extra cache directory for telemetry

If the sdk's `.daml` directory is not writable, telemetry fails and
kills the IDE. We add a new assistant environment variable "DAML_CACHE".
If set, this directory is used for telemetry cache data.

Fixes #8396.

CHANGELOG_BEGIN
CHANGELOG_END

* try creating directory

* check for already existing machine id file

* set DAML_CACHE to read-only in assistant integration tests
2021-01-15 11:44:18 +01:00
Remy
dc5b22976f
LF: fix Transaction Decoder/Encoder (#8512)
This includes the following changes:

1.  The Decoder was not verifying the value inside a node use the exact
    same version as the node itself.

2.  We use the node version directly in the node encoder (instead of
    used versionedX node methods).

3.  We avoid used of .asJava in the encoder.

4.  We add test for testing the decode verify 1.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-15 11:44:05 +01:00
mzagorski-da
c00c9d13a2
Expand usage of parallel unnesting on participant_contracts and participant_contract_witnesses tables (#8511)
Expand usage of parallel unnesting on participant_contracts and participant_contract_witnesses tables

CHANGELOG_BEGIN
* providing postgresql insert statement based on unnesting for participant_contracts and participant_contract_witnesses tables
* slightly refactored Tables objects to keep compatiblity w H2 database
* provided implicits for arrays types resolution because of performance reasons
CHANGELOG_END

Co-authored-by: tudor-da <tudor.voicu@digitalasset.com>
2021-01-15 10:19:58 +01:00
azure-pipelines[bot]
7e63b0d988
update NOTICES file (#8518)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-01-15 10:18:28 +01:00
Gary Verhaegen
71a1260e56
fix typo in docs (#8502)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-14 19:46:44 +00:00
Gary Verhaegen
96e9dde360
ci: clean-up disk cache if it gets too big (#8515)
Hopefully this works around our recent CI disk space issues, while 80GB
should be large enough that it only happens once per machine per day, so
perf shouldn't be impacted too much.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-14 19:41:07 +00:00
Gary Verhaegen
7f076bcd56
docs cron: update meta files unconditionally (#8514)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-14 20:04:48 +01:00
Moritz Kiefer
dfc48a0010
Update both hidden.json and snapshots.json (#8513)
* Update both hidden.json and snapshots.json

The assistant relies on the latter, our docs cronjob on the former. I
have no idea why we have two but keeping them in sync should be fine.

changelog_begin
changelog_end

* maybe I should test if my code compiles before pushing

changelog_begin
changelog_end
2021-01-14 19:07:35 +01:00
Gary Verhaegen
867041ce61
hide with-dlint option (#8506)
Quoting @cocreature on Slack:

> the fact that --with-dlint is exposed as a public option at all seems
> broken. I don’t think it was ever intended to be one.

CHANGELOG_BEGIN

- [Daml CLI] Three options that were never meant to be public are now
  hidden from the output of various `daml xxx --help` commands. The
  options still have the same effect when present, so this is not (yet)
  a breaking change. Using these options tends to result in very
  unhelpful, when not outright broken, dlint configurations, so we expect
  existing usage to be low to nonexistent. If you do use them please
  share your use-case with us. The options are:
  - `--with-dlint`
  - `--without-dlint`
  - `--allow-overrides`

CHANGELOG_END
2021-01-14 18:56:59 +01:00
Simon Meier
0417952a8f
Sanbox: improve error message when accessing archived contracts (#8500)
CHANGELOG_BEGIN
- [Sandbox] Improve error message when accessing archived contracts.
CHANGELOG_END
2021-01-14 18:04:38 +01:00
Oliver Seeliger
f8784312c8
Disable async commit on h2database (#8505)
This produces a warning in the log that H2database does not support async commit
ever since we pass the flag through the DbDispatcher to the HikariConnection builder.

Disabling the flag when the DbType indicates that the db does not support async commit.

Fixes #8501

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-14 15:34:13 +00:00
Stephen Compall
c0c2a59275
hybrid database-then-gRPC websocket query (#8226)
* group database queries while still providing the matchedQueries part of the event

* SQL query for multiquery websocket request all at once

* fetchAndPersist responds with a single bookmark

* unify connection imports

* convert DB results to domain, reassociate with proper query indices

* reassociate multiple matches with proper query indices

* make overlap more likely in testing overlap

* simpler matchedQueries merging for multi-query case

* integrate DB query with metadata-ful StreamQuery's

* expose daoAndFetch; better insertDeleteStepSource doc

* more efficient query path for contract key streams

* missed LogHandler

* persist resolved template IDs; glue the prefiltered set into the stream

* ticks and phantom removal need the state from the ACS

* compile SQL queries for query language predicates on WS

- wrong matchedQueries order

* harmonize order of ACS-vector and Positives

* misc compilation conversions

* WebsocketServicePostgresInt mixin

* update (C) date

* test websocket queries under postgres

* looking for new way to compile queries with proper matchedQueries offsets

* model that querying without matchedQueries requires only one SQL query

* SQL path for contract key streams

* nondeterminism

* fix 3 fetch tests with SQL syntax

* nondeterminism mk 2

* fix multi-party query tests by dealing with nondeterminism properly

* temp logs to track down the contract duplication

* match new scalafmt from #8437

* remove completed TODOs

* add changelog

CHANGELOG_BEGIN
- [JSON API] If the JDBC query store is enabled, it will be used to optimize
  Websocket queries as well as the previously-supported synchronous queries.
  See `issue #8226 <https://github.com/digital-asset/daml/pull/8226>`__.
CHANGELOG_END

* fix up matchedQueries indices

* complete the fast path for by-id queries

* remove AS c

- suggested by @cocreature; thanks

* remove temporary debugging logs

- suggested by @cocreature; thanks
2021-01-14 10:01:17 -05:00
azure-pipelines[bot]
65b365c2c6
update NOTICES file (#8499)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-01-14 08:45:53 +01:00
Remy
f872f473cb
LF: move package lookup utilities directly inside GenPackage (#8496)
There is no reason why they have are in a special object inside the
engine package.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-13 23:25:39 +01:00
Moritz Kiefer
cdb2c2d0ba
Port the rest of //ledger/... to Scala 2.13 (#8497)
* Port the rest of //ledger/... to Scala 2.13

draw the rest of the fcking owl

Omitted for now are the ledger API test tool which has a dependency
only compatible with 2.12 and the generated code of the Scala
codegen (the codegen compiles and runs with 2.13, the generated code
does not).

changelog_begin
changelog_end

* Less symbols

changelog_begin
changelog_end
2021-01-13 22:23:31 +01:00
Remy
77bec01db3
LF: provide methods to enrich LF values missing type and field names (#8493)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-13 19:02:51 +01:00