Commit Graph

8602 Commits

Author SHA1 Message Date
azure-pipelines[bot]
ef23593ee1
update compat versions for 1.11.3 (#12012)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-12-07 17:45:25 +01:00
Gary Verhaegen
be088c6c68
pipeline: fix pie failure (#12017)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-07 14:45:36 +00:00
Moritz Kiefer
c493c46a94
Test release for split release process (attempt 2) (#12007)
This time with an actual working CI on `main` which might help.

changelog_begin
changelog_end
2021-12-07 13:24:55 +00:00
Moisés Ackerman
8df9a42f29
Interface desugaring cont. (#11964)
* Move toInterfaceContractId and fromInterfaceContractId out of Implements class

* Split Implements class into single-method classes

* Define toInterface outside its class to swap type arguments

This allows users to call 'toInterface @Interface', since the type of the template can usually be inferred

* Move interface classes and functions to DA.Internal.Interface

changelog_begin
changelog_end
2021-12-07 13:56:57 +01:00
Simon Meier
ef4ae931d1
Add Ledger API scala client for user management (#12000)
scala bindings: add client for user management API

CHANGELOG_BEGIN
- [Scala bindings] add methods to call the user management API
CHANGELOG_END
2021-12-07 13:36:05 +01:00
Moritz Kiefer
444b3ac88e
Rerelease 1.11.3 (#12006)
changelog_begin
changelog_end
2021-12-07 13:28:15 +01:00
Moritz Kiefer
c63a2c835f
Drop 1.11.3 release to allow for a retry (#12005)
changelog_begin
changelog_end
2021-12-07 11:07:50 +00:00
Moritz Kiefer
505a37fb01
Fix yaml again … (#12004)
changelog_begin
changelog_end
2021-12-07 10:47:21 +00:00
Stefano Baghino
4099dff791
Promote 1.11.3-snapshot.20211123.6448.0.c679b4ab to release (#11999)
changelog_begin
changelog_end

The snapshot was released with https://github.com/digital-asset/daml/pull/11981
2021-12-07 10:22:03 +00:00
Stefano Baghino
00d3326ebe
Remove source code specific to Scala 2.12 (#12003)
Looks like it has been removed from the build files but not from the source tree.

changelog_begin
changelog_end
2021-12-07 10:08:19 +00:00
Moritz Kiefer
fa88836b10
Fix main build after broken yaml config (#12002)
I love yaml

changelog_begin
changelog_end
2021-12-07 09:42:51 +00:00
Moritz Kiefer
1c4151ba5e
Test release for split release process (#11992)
changelog_begin
changelog_end
2021-12-07 10:00:17 +01:00
azure-pipelines[bot]
47ce5267ab
update NOTICES file (#11995)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-12-07 08:47:08 +01:00
Moritz Kiefer
85e2fa551c
Avoid CPP in ledger API test tool daml code (#11993)
Apparently this works very poorly with damlc_legacy which fails to
find hpp after the latest nixpkgs upgrade. Somewhat confusingly it
only fails to find that on release builds and I don’t understand how
it ever worked but this seems more sensible anyway.

changelog_begin
changelog_end
2021-12-07 07:06:12 +01:00
Robin Krom
299027b4b9
interfaces: doc generation for interfaces (#11971)
* interfaces: doc generation for interfaces

This adds generated documentation for interface choices.

CHANGELOG_BEGIN
CHANGELOG_END

* added issue to todo

* refactor getTemplateDocs/getInterfaceDocs

* refactor template/interface doc extraction
2021-12-07 06:49:42 +01:00
Sofia Faro
9430ee3131
interfaces: Add requires in DefInterface protobuf (#11986)
Part of #11978

changelog_begin
changelog_end
2021-12-07 00:38:59 +00:00
Gary Verhaegen
349d812482
ci: increase hard drive space (not macOS) (#11983)
I've seen quite a few builds failing for lack of disk space recently,
sometimes as early as 2pm CET.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-06 19:41:11 +00:00
Moritz Kiefer
5a019eaaa6
Split release process (PoC) (#11991)
* Split release process (PoC)

This PR adds the necessary infrastructure for the new split release
process. Releases are still triggered via the LATEST file but you can
choose between the old and the new release process by adding
SPLIT-RELEASE at the end of the line.

As a first step this publishes all artifacts we currently publish to
Github releases to our GCP bucket.

changelog_begin
changelog_end

* drop report-start

changelog_begin
changelog_end

* fix gcp bucket

changelog_begin
changelog_end

* review feedback

changelog_begin
changelog_end

* Update azure-pipelines.yml

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* SPLIT-RELEASE ->SPLIT_RELEASE

changelog_begin
changelog_end

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2021-12-06 19:25:00 +01:00
Remy
f228f9575c
LF: replace boolean by ADT in transaction traversal (#11988)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-06 18:19:47 +00:00
Stefano Baghino
8290c12789
Release 1.11.3-snapshot.20211123.6448.0.c679b4ab (#11981)
Changelog
---------

```

- [JSON-API] added metrics to separately track:
    - time taken to update query-store ACS (from ledger)
    - lookup times for the query store

[HTTP/JSON API]
- metrics reporting can now be chosen via the command line option --metrics-reporter (currently hidden), valid options are console, csv://, graphite:// and prometheus://
- metrics reporting interval can also now be chosen via a command line option --metrics-reporting-interval (currently hidden)

- [Ledger HTTP Json Service] Logging output can now be in JSON either via providing the cli option `--log-encoder json` or via setting the env var `LOG_FORMAT_JSON=true`

[HTTP-JSON]
- fixed that json log output could not be enabled via cli options except via usage of env vars

- [Ledger HTTP Json service] Logging can now be configured via the `--log-level` cli argument. Valid values are `error`, `warn`, `info` (default), `debug`, `trace`

- [Ledger HTTP Json Service] Logging now also tells service name if log level was changed.

http-json:
- add contextual id as logging context to distinguish different application runs in logs
- add request id as logging context to distinguish different http requests within an application run
- add for non-static endpoints trace logs which show how long processing it took in ns

- [JSON API] Fix an error where transactions that delete a large
  number of contracts resulted in stackoverflows with the PostgreSQL
  backend and database errors with Oracle.

- [JSON API] While updating the contract table for a query, if the DB appears to be slow,
  JSON API will slow down its own inserts and deletes at some point rather than construct
  ever-larger INSERT and DELETE batch commands.
  See `issue #11589 <https://github.com/digital-asset/daml/pull/11589>`__.
Backport of Hikari connection pool for 1.11.x plus addition of configurable connection pool properties to jdbcConfig string, the properties are listed below
poolSize -- specifies the max pool size for the database connection pool
minIdle -- specifies the min idle connections for database connection pool
connectionTimeout -- long value, specifies the connection timeout for database connection pool
idleTimeout -- long value, specifies the idle timeout for the database connection pool
[Ledger API] Retry the interpretation of a command in case of a race
with other transactions. This fix drastically reduces the likelihood of the error
"Could not find a suitable ledger time after 0 retries".
For every update in the index db log the full context at the INFO level.
[Integration Kit] The state of the participant indexer can now be checked via the GRPC health endpoint
```

Commits
-------

```
c679b4ab23 [1.11.x] Metrics for investigating  json api performance (#11769)
437c3d4639 Bump Windows postgres (#11806)
382168b7f0 Add metrics to the http json service (#9923) (#11768)
b7fc305427 [Backport] Add cli option & system property to enable json only logging for the json api (#11717)
17cacb05f1 [Backport] Add logging command line option to ledger http service (#9581) (#11699)
56531e8610 Backport: Support deletion of a large number of contracts (#11646)
6bf19fce82 Backport: limit contract insertion/deletion batching on backpressure (#11647)
eed4ad32b7 Backport signing tool changes (#11651)
ee7547ed88 [JSON-API] Backport of connection pool for 1.11.x , configurable db options needs forward porting (#11615)
632ba5a7b3 Backport: Avoid collision in execution log postfix (#11644)
cf4f1a1d8f Backport: add tar to dev-env (#10173) (#11636)
b78fa8c4a3 Backport: Restart the submission interpretation in case of a race [DPP-737] (#11578)
5c3220d2db Fix build of release/1.11.x branch (#11582)
cce7c79ec5 Log context of all updates written to the database (#10057)
a9d0473473 Adds Indexer state to GRPC health checks [DPP-434] (#9951) (#9961)
```

changelog_begin
changelog_end
2021-12-06 17:53:12 +01:00
nicu-da
cd8f2c76fc
[kvutils] - Integrate the deduplication offset support into kvutils [kvl-1172] (#11916)
Add support for offset deduplication periods for KV ledgers by converting the offset into a duration using the record time for the completion found of the offset.

changelog_begin
changelog_end
2021-12-06 16:09:59 +00:00
Gary Verhaegen
9fa3c76b64
ci: more reliable compat_pr (#11989)
Potentially running code from the future is a bit scary in terms of
compatibility.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-06 15:13:37 +00:00
Moritz Kiefer
c29f4d333b
Fixed broken merge in //daml-lf (#11985)
changelog_begin
changelog_end
2021-12-06 14:32:37 +00:00
Moritz Kiefer
0285eaaac1
Drop snapshots from compat tests (#11976)
* Drop snapshots from compat tests

We don’t really give strict compatibility guarantees on snapshots and
we actively break them (e.g. you cannot migrate across snapshots). So
this PR stops including snapshots in the compat tests which should
both speed them up a bit and stop irrelevant failures.

changelog_begin
changelog_end

* hlint

changelog_begin
changelog_end
2021-12-06 13:44:30 +01:00
Moritz Kiefer
3922bfaa18
Release 1.18.0 RC 3 (#11977)
changelog_begin
changelog_end
2021-12-06 12:39:04 +00:00
Bernhard Elsner
80768427d0
Add Java and database compatibility docs. (#11979)
* Add Java and database compatibility docs.
CHANGELOG_BEGIN
CHANGELOG_END
* Update the versions we test on to match reality
2021-12-06 12:15:33 +00:00
Moritz Kiefer
e01fcdf143
Add exhaustive tests for evaluation order on creates (#11957)
As mentioned in the comment, I view these tests as a way to guard us
against bricking ledgers on upgrades so they somewhat deliberately
ignore ordering of non-cachable errors relative to each other since
those only change error messages.

changelog_begin
changelog_end
2021-12-06 13:06:46 +01:00
Gerolf Seitz
a27a53978c
Warn against using latest node during release testing. (#11980)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-06 12:48:18 +01:00
nicu-da
b6334f8fde
[kvutils] - Minimize the scope of the ReadService and the WriteService in the Runner [KVL-1172] (#11923)
changelog_begin
changelog_end
2021-12-06 09:47:58 +00:00
nicu-da
383e251e88
[kvutils] - Rename expire_at to prunable_from for clarity [KVL-1199] (#11960)
changelog_begin
changelog_end
2021-12-06 00:02:26 -08:00
nicu-da
28e62820ef
[ledger-api] - Rename the error metadata key for invalid deduplication duration (#11966)
changelog_begin
[ledger-api] - rename metadata key max_deduplication_duration to longest_duration for error concerning invalid deduplication durations
changelog_end
2021-12-05 23:55:47 -08:00
azure-pipelines[bot]
f46f0a0ca5
update NOTICES file (#11975)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-12-05 20:14:35 +01:00
Moisés Ackerman
ad77eed798
Move method interfaceTypeRep to its own class (#11940)
changelog_begin
changelog_end
2021-12-04 15:33:58 +00:00
azure-pipelines[bot]
03793c97ab
update compat versions for 1.17.2-snapshot.20211124.7857.0.b33b94b8 (#11955)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-12-03 15:04:12 +01:00
azure-pipelines[bot]
689d9c3494
update NOTICES file (#11972)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-12-03 08:50:59 +01:00
Moritz Kiefer
cde80df2c4
Mark lsp tests as flaky (#11958)
* Mark lsp tests as flaky

Not quite sure what is going wrong and doesn’t seem worth prioritizing
right now so marking them flaky again (they were marked flaky until a
few weeks ago).

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-12-03 07:42:52 +01:00
Sergey Kisel
020712e2ed
Remove the use of nesting with size approach as suggested by scalacheck. (#11911)
* Remove the use of nesting with `size` and `resize` approach as suggested by scalacheck.

CHANGELOG_BEGIN
CHANGELOG_END

* Biased to shorter names generator for Name
2021-12-03 06:46:45 +01:00
Sergey Kisel
15f509134f
Remove the use of nesting in ValueGen with size approach. (#11941)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-03 06:46:30 +01:00
Robert Autenrieth
5bf96574da
DPP-723 Add global parallelism limit for ACS queries (#11863)
* DPP-723 Add global parallelism limit for ACS queries

changelog_begin
changelog_end

* Use the new limiter

* Remove ThreadPoolBasedConcurrencyLimiter

* Refactor ConcurrencyLimiter

* Reduce test size

* Limit contract fetching instead of id fetching

* Fix CLI options

* Apply reviewer comments
2021-12-02 23:34:52 +01:00
mziolekda
1eabb50ad2
Fix broken migration script that does not cater to empty string interning tables (#11968)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-02 22:41:36 +01:00
Marton Nagy
84aab0b7fe
Fix PreemptableSequence merge function (#11951)
By switching to transform from onComplete, to force execution before the future is completed.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-02 20:48:26 +01:00
Andreas Herrmann
c97f923d8c
Use more deterministic distdir path in haskell_cabal_* (#11963)
* Update rules_haskell

changelog_begin
changelog_end

* Use mostly deterministic distdir path

changelog_begin
changelog_end

Use a path for distdir in the `haskell_cabal_*` rules that is based on
the package name and deterministic unless there is a collision with an
already existing file or directory.

The distdir path enters flags that are passed to GHC and thereby enters
the 'flag hash' field of the generated interface files. If the distdir
path is indeterministic, then the interface file will also be
indeterministic.

* Pin com_google_protobuf to 3.17.3

The latest version of rules_haskell pins protoc to 3.19.1. However, this
generates code that also requires a more recent version of the Java
proto libraries. This change pins the protoc version at the same version
as the protobuf-java library, see https://github.com/protocolbuffers/protobuf/issues/9236.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-12-02 19:25:26 +00:00
Moritz Kiefer
53a74bc108
Include module names in LF type errors (#11962)
After staring at a type mismatch between `Nested` and `Nested` this
seemed like a sensible change. We could also include the package
ids. I don’t have strong feelings about this but this seems like a
clear uncontroversial improvement.

changelog_begin
changelog_end
2021-12-02 16:59:02 +00:00
Simon Meier
c19e3f5c46
Add additional rights and clarifications to user management API (#11908)
* user management: document optional primary party

CHANGELOG_BEGIN
CHANGELOG_END

Breaks-protobuf: true
2021-12-02 16:39:31 +01:00
Remy
b02ed7765a
LF: Introduce configurable limits on produced transactions (#11948)
This is part of #11691

This PR allows to limits:
- the number of signatories,
- the number of observers,
- the number of controllers,
- the number of inputContracts,

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-02 16:31:09 +01:00
Moritz Kiefer
4571a48c48
Add skeleton for trace-based LF evaluation tests (#11949)
* Add skeleton for trace-based LF evaluation tests

The recent issues around wrong ordering of contract id typechecks got
me thinking about this again so I wanted to hack up a PR of how we can
test evaluation order sensibly directly at the LF level.

This is only testing a single create but I first wanted to see if we
agree that this is a sensible approach using a simple example.

If we agree that this is a sensible approach, I’d suggest to extend
those tests later in separate PRs to make them exhaustive.

changelog_begin
changelog_end

* cleanup

changelog_begin
changelog_end
2021-12-02 11:38:59 +00:00
Gerolf Seitz
be7e377d79
Release 1.17.2-snapshot.20211124.7857.0.b33b94b8 (#11953)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-02 10:22:48 +00:00
mziolekda
96f0f4f63f
Introduce better TLS version tests (#11950)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-02 11:00:39 +01:00
Robin Krom
20d1b95fe2
docs: support for interface implementations (#11643)
* docs: support for interface implementations

This produces documentation for interface implementations of templates.

CHANGELOG_BEGIN
CHANGELOG_END

* Update compiler/damlc/daml-doc/src/DA/Daml/Doc/Render/Output.hs

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

* add check for modulename for implementations

CHANGELOG_BEGIN
CHANGELOG_END

* added golden tests to daml-test-files

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
2021-12-02 09:51:41 +00:00
Moritz Kiefer
cf110a4bbf
Only run ledger API test tool compat tests between stable releases (#11954)
We break things way too frequently between releases so the exclusions
are a huge mess with little benefit.

changelog_begin
changelog_end
2021-12-02 09:34:20 +00:00