Commit Graph

6642 Commits

Author SHA1 Message Date
Remy
70379d4e85
LF: Experiment Hook for 1.dev (#9051)
This PR adds a hook in the compiler and the engine to allow easy prototyping of new features.

In particular by using this hook, one can add a new feature in 1.dev with modifying:

    compiler
    type checkers (both Haskell and Scala)
    archive Protobuf
    archive decoder

In addition of development speed, this also adds a bit of confidence, as the peaces enumerated above are not touched when adding the feature: The feature can be added by modifying only 1.dev Daml standard library and Speedy.

aa7991f8 shows a use case of this hook.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-11 21:36:49 +01:00
Moritz Kiefer
ba98e6d5f7
Release another snapshot (#9106)
This time including the fix from
e2730d322d which may or may not be
sufficient.

changelog_begin
changelog_end
2021-03-11 20:47:52 +01:00
Moritz Kiefer
8e25227cbb
Fix heartbeating in the JSON API when starting from offset (#9104)
We filter heartbeat ticks until we get the first step message. This is
when starting from the ACS but it is incorrect when starting from an
existing offset where this results in us not emitting heartbeats until
we get the first step message. This PR fixes this by passing along the
initial offset and adds a test for this.

changelog_begin

- [Json Api] Fix a bug where heartbeating on websocket connections did not start until the first transaction was received when resuming from a previous ledger offset. See https://github.com/digital-asset/daml/issues/9102

changelog_end

fixes #9102
2021-03-11 19:37:57 +01:00
Gary Verhaegen
e2730d322d
ci/windows: add missing mkdir (#9103)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-11 18:16:19 +00:00
azure-pipelines[bot]
ad151df676
rotate release duty after 1.11.0-snapshot.20210309.6463.0.f7abca91 (#9062)
@S11001001 is taking care of 1.11.0-snapshot.20210309.6463.0.f7abca91 (#9061), so they get pushed back to the end of the line.

Please do not merge this before #9061.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-11 13:14:27 -05:00
Moritz Kiefer
02f3612fed
Release new snapshot (#9098)
This is mainly to test the revamped publishing process.

changelog_begin
changelog_end
2021-03-11 17:48:02 +01:00
Matthias Schmalz
1c254a9a68
Enhanced docker-compose Graphite/Grafana (#9067)
CHANGELOG_BEGIN
Enhanced docker-compose project for deploying Graphite/Grafana:
- Fixed incorrect aggregation of ".count" metrics by DropWizard.
- Changed Graphite retention schedule to 10s granularity during 7 days, 1m granularity during 30 days, 10m granularity during 1800 days.
- Preserve Grafana configuration during restarts.
- Added script to manually purge all data/configuration.
- Mount Graphite configuration directories on host.
- Enhanced documentation.
- Tidied up directory structure.
CHANGELOG_END
2021-03-11 17:34:43 +01:00
Stefano Baghino
e33479af4c
Apply new Daml logo to Navigator (#9101)
Also removes mark.png, which was unused.

changelog_begin
[Navigator] New logo
changelog_end
2021-03-11 16:04:55 +00:00
Moritz Kiefer
ecd3110f48
Fix compat job (#9099)
* Fix compat job

changelog_begin
changelog_end

* Cleanup head_sdk

changelog_begin
changelog_end

* turns out removing everything also removes files that shouldn’t be removed

changelog_begin
changelog_end

* gnah

changelog_begin
changelog_end

* maybe this was a bad idea

changelog_begin
changelog_end

* trynottocry

changelog_begin
changelog_end
2021-03-11 15:13:49 +00:00
Moritz Kiefer
5cb857d902
Release EE SDK tarballs and installer (#9086)
* Release EE SDK tarballs and installer

As before, no way of testing this. I’ll do a snapshot afterwards.

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* Rename EE artifacts

changelog_begin
changelog_end
2021-03-11 14:49:48 +01:00
Stefano Baghino
63623dc9a4
Refactor TableSpec (#9088)
changelog_begin
changelog_end
2021-03-11 13:27:30 +00:00
Moritz Kiefer
d1031ba1e9
Fix typo in azure-pipelines.yml (#9087)
changelog_begin
changelog_end
2021-03-11 13:27:18 +00:00
Remy
6e65b9b038
LF: move value normalizer to transaction module. (#9072)
Like that people can use it outside test.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-11 13:30:53 +01:00
Gerolf Seitz
695ad072ac
Fix double push in CommandTracker (#9059)
The command tracker detects timeouts outside the regular pull/push
mechanism of the custom akka graph stage. Basically the timeout
detection jumps the line when emitting outputs on `resultOut`. If it
then processes a regular completion, it tries to push to `resultOut`
even though it hasn't been pulled again in the meantime. Using `emit`
instead of `push` when a completion arrives makes akka take care of
handling the signaling properly.

Fixes DPP-285

CHANGELOG_BEGIN
[Integration Kit, Sandbox, Daml on SQL, Scala Ledger Bindings]:
Fixed a bug that caused the CommandTracker used by the synchronous
CommandService to crash in case of a lot of timeouts.
CHANGELOG_END
2021-03-11 13:19:29 +01:00
Sofia Faro
e538f2d676
Add DA.Exception and enable LF primitives. (#9082)
- Add `DA.Exception` module that re-exports `DA.Internal.Exception`
while hiding the `DamlException` class.

- Uncommented out the primitive calls in `DA.Internal.Exception`, so
the standard library is actually calling the underlying LF primitives.

- Expanded the `ExceptionDesugared` test to actually throw and catch
an exception. :-)

changelog_begin
changelog_end
2021-03-11 11:42:16 +00:00
Andreas Herrmann
2be1a964f8
Avoid submitTree on simple exercise commands (#9080)
* Factor out traverseEventInTree

* Generalize encodeSubmitCreated to simple exercise

Where simple exercise events are those that cause one contract creation
and return its contract id.

* Use submit on simple exercise commands

* Define type to track simple events

changelog_begin
changelog_end

* encode SimpleEvent

* SimpleEvents from tree

* Test SimpleEvent.fromTree

* Test encodeSubmitSimpleEvents

* Match Kind.Empty explicitly

* Use ListBuffer

* Factor out treeEventCreatedCids

* foldMap

* Scala 2.12 vs 2.13

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-11 11:41:04 +00:00
Robin Krom
d412de68f5
cheat-sheet: update (#9079)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-11 12:01:16 +01:00
Moritz Kiefer
979e12fa68
Move artifact publishing out of yaml files (#9071)
* Move artifact publishing out of yaml files

The current publishing process pretty much hardcodes the set of
artifacts we publish in the yaml config. This is a problem because we
always release from `main` so the yaml files are always
identical. However, we will add new artifacts over time and this
starts falling apart. This PR changes this such that the process
described in the yaml files is very generic and just uploads and
downloads everything in a directory whereas the details are handled in
bash scripts that will come from the respective release branch and are
therefore version-dependent.

As usual for these type of changes, I don’t have a great way to test
this. I did do some due diligence to test that at least the artifacts
are published correctly and I can download them but I can’t test the
actual publishing.

changelog_begin
changelog_end

* Update ci/copy-unix-release-artifacts.sh

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Update ci/copy-windows-release-artifacts.sh

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Update ci/publish-artifactory.sh

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2021-03-11 11:44:02 +01:00
Stefano Baghino
0e4af7418d
Add runner for non-repudiation (#9076)
* Add runner for non-repudiation

Closes #8633

changelog_begin
changelog_end

* Fix existing tests

* Add test for non-repudiation-app option parser

* Remove unnecessary dependencies from non-repudiation testing

* Fix Scala 2.13 build errors

* Fix help message for --api-address and --api-port
2021-03-11 10:57:50 +01:00
azure-pipelines[bot]
4784cfa8b5
update NOTICES file (#9078)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-11 10:14:07 +01:00
Gary Verhaegen
f0ca332983
fix release notification (#9075)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-10 17:12:14 +00:00
Remy
8a35528514
LF: Make 1.12 default compiler output (#9058)
CHANGELOG_BEGIN
- [Compiler] The compiler produces LF 1.12 by default. LF 1.12 only reduces transaction size.
CHANGELOG_END
2021-03-10 16:44:29 +00:00
azure-pipelines[bot]
a1edf97256
update compat versions for 1.11.0 (#9066)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-10 16:03:25 +01:00
azure-pipelines[bot]
5aac471bf5
release 1.12.0-snapshot.20210309.6463.0.f7abca91 (#9061)
* release 1.11.0-snapshot.20210309.6463.0.f7abca91

This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@S11001001 is in charge of this release.

Commit log:
```
f7abca919f fix readme in create-daml-app: add npm install (#9060)
e04bd91eda Move Daml Profiler to EE version of sandbox/sandbox-classic (#9054)
7859bc13e1 inline help: scripts JSON API clarification (#9057)
642385a692 Upgrade elliptic dependency to address vulnerabilities (#9055)
f2b9f118cf Check that only the command signed payload goes through (#9053)
5197374cab Build SDK EE tarball (#9049)
1bec2116ba Add tests for client bindings (#9036)
27fd9327f6 Disclose transactions in the flat transactions stream only to the stakeholders - test case [DPP-276] (#9028)
45b33757b2 Introduce a helper to limit signing to commands (#9050)
0c4d8ac19c Speedy: rollback within exercise (#9040)
caa023b72e ci/cron/check: remove dade-assist calls (#9048)
85f0965ab9 LF: fix archive decoder for GREATER_DATE (#9047)
121534c54d ci/cron/check: low-hanging perf improvement (#9042)
fb0bd82740 update NOTICES file (#9045)
4fd42a6772 reduce noise on daily tests (#9039)
5d88c08832 Bump ghcide (#9041)
41d62ea5c0 send CI failures on separate channel (#9038)
e26d006f6f slack: reduce @here mentions (#9037)
ae110361f0 Add more shared infrastructure for non-repudiation testing (#9035)
75140744aa Speedy: building transactions with rollback (#8983)
98410e7c7f update compat versions for 1.11.0-snapshot.20210304.6422.0.d3d5042a (#9032)
0f1d2a9c47 Upgrade akka to 2.6.13 (#9026)
b1a59d0c70 Release SDK 1.11.0 RC 2 (#9031)
41e049339b Fix --extract option in ledger api test tool (#9029)
0281b442b8 -Werror all the things (#9027)
2c08586d33 Expose Daml stacktraces for Daml Script errors (#9025)
c7ee410fa3 Use DA.Map in triggers if available (#9023)
d347934db3 Make non-repudiation PostgreSQL back-end certificate adding idempotent (#9024)
eaf7fb6f63 Add non-repudiation API (#8982)
5a3c95251c Document latency of Windows ad-hoc machine software installation (#9022)
3758b57a87 pkg deps: Move all dalfs to package dependency directory (#9002)
f70d3391ef update compat versions for 1.10.2 (#9016)
e5421288d9 Improve errors on duplicate record field names (#9010)
65fbcfe8e9 Move execution of commands out of Runner (#9009)
2688ad6f0d autorelease: improved PR message (#9008)
185e91646d Release 1.10.2 (#9006)
4ed3f1a6de Cut a snapshot release for the 1.8.0 special patch (#8998)
ba1456fd5a rotate release duty after 1.11.0-snapshot.20210302.6414.0.72870630 (#8992)
259bf682c4 compat: fix killing stale sandbox(es) (#9003)
32dc8b007b Include stack traces for all ScriptF commands (#8999)
1525957c7a release 1.11.0-snapshot.20210302.6414.0.72870630 (#8991)
926949e503 Use single party submit/submitTree where appropriate (#8995)
```
Changelog:
```

- [Daml Compiler] Fix a bug where passing `--ghc-option=-Werror` also
  produced errors for warnings produced by `-Wmissing-signatures` even
  if the user did not explicitly enable this.

- [Daml Script] When running Daml Script on the command line you will
  now see a Daml stacktrace on failures to interact with the ledger
  which makes it significantly easier to track down which of the calls
  fails. By default, you will only get the callsite of functions like
  `submit`. To extend the stack trace, add `HasCallStack` constraints
  to functions and those will also be included.

- [Triggers] The trigger library now uses `DA.Map` instead of the
  deprecated `DA.Next.Map` if the targeted Daml-LF version supports it.
  This is a breaking change: Code that interfaced with the triggers
  library using `DA.Next.Map`, e.g. with
  `Daml.Trigger.getCommandsInFlight` or `Daml.Trigger.Assert.testRule`,
  will need to be changed to use `DA.Map` instead.
```

CHANGELOG_BEGIN
CHANGELOG_END

* 1.12 not 1.11

changelog_begin
changelog_end

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-03-10 14:55:32 +00:00
Sofia Faro
e6ea8fa9b3
Start adding exception syntax support. (#8960)
* Start adding exception syntax support.

changelog_begin
changelog_end

* .

* ..

* . .

* ....

* .___.

* ..__..

* ._._._.

* update stack snapshot

* recompile ghc-lib

* update stackage

* add CmmParse patch

* incorporate CmmParse.y patch into ghc branch

* Add a first test for exception syntax desugaring.

* Add a test for multiple message declarations

* update ghc patch

* Update duplicate message test with location

* add brackets test

* Add final patch SHA

* update stack-snapshot and unpin for unix

* Update stackage pin on windows
2021-03-10 12:07:53 +00:00
Gary Verhaegen
32e8c66d8d
fix typo in release notification (#9069)
So that on non-automated releases we get nothing instead of an empty
diamond.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-10 11:38:12 +00:00
Gary Verhaegen
4bf05edf2b
tell people to speak up about snapshot blockers (#9068)
Suggested by @stefanobaghino-da; thanks!

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-10 11:18:57 +00:00
azure-pipelines[bot]
c200eb8244
update NOTICES file (#9063)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-10 10:15:37 +01:00
Moritz Kiefer
b408b02723
Release SDK 1.11.0 (#9064)
changelog_begin
changelog_end
2021-03-10 09:14:25 +00:00
Richard Kapolnai
f7abca919f
fix readme in create-daml-app: add npm install (#9060)
* fix readme in create-daml-app: add npm install

* changelog_begin
changelog_end
2021-03-09 22:02:55 +01:00
Moritz Kiefer
e04bd91eda
Move Daml Profiler to EE version of sandbox/sandbox-classic (#9054)
* Move Daml Profiler to EE version of sandbox/sandbox-classic

This splits Sandbox targets into EE/CE targets and exposes the option
in the EE version. The option still exists in the CE option for now
until we have released EE artifacts to not break users that might know
about it without an alternative.

There is also a small test that makes sure that this actually works
since classpaths are dumb and it didn’t work at first.

changelog_begin
changelog_end

* Fix publish target

changelog_begin
changelog_end

* Publish transitive dep

changelog_begin
changelog_end

* I hate bash

changelog_begin
changelog_end
2021-03-09 19:35:14 +01:00
Gary Verhaegen
7859bc13e1
inline help: scripts JSON API clarification (#9057)
* inline help: scripts JSON API clarification

While writing a forum response and trying to use `daml script` against
the JSON API, I was a bit confused by the existing help texts.

CHANGELOG_BEGIN
CHANGELOG_END

* thanks, scalafmt, that's very useful feedback
2021-03-09 16:01:22 +00:00
Moritz Kiefer
642385a692
Upgrade elliptic dependency to address vulnerabilities (#9055)
* Upgrade elliptic dependency to address vulnerabilities

This PR bumps our already existing resolution on elliptic to address a
security issue.

I don’t believe in keeping deps at arbitrarily outdated versions so
this PR fully regenerates the yarn.lock files.

changelog_begin
changelog_end

* Apparently upgrading Navigator is bad

changelog_begin
changelog_end
2021-03-09 14:44:10 +01:00
Stefano Baghino
f2b9f118cf
Check that only the command signed payload goes through (#9053)
Left over from https://github.com/digital-asset/daml/pull/9036

Addresses https://github.com/digital-asset/daml/pull/9036#pullrequestreview-606492151

changelog_begin
changelog_end
2021-03-08 21:34:10 +01:00
Moritz Kiefer
5197374cab
Build SDK EE tarball (#9049)
* Build SDK EE tarball

This sets up the infrastructure to build an SDK EE tarball and allows
for swapping out all files included in the tarball depending on the
edition. As an example, this includes the JSON API with (partial)
Oracle support in the EE tarball.

This PR does not yet address publishing this artifact to Artifactory.
I’ll tackle that in a separate PR.

changelog_begin
changelog_end

* Build in temp dir because Windows is stupid

changelog_begin
changelog_end

* directories are bad

changelog_begin
changelog_end

* Navigator resources are actually needed

changelog_begin
changelog_end
2021-03-08 17:48:00 +01:00
Stefano Baghino
1bec2116ba
Add tests for client bindings (#9036)
* Add tests for client bindings

changelog_begin
changelog_end

Closes #8636

* Address https://github.com/digital-asset/daml/pull/9036#discussion_r588332093
2021-03-08 16:32:23 +00:00
Kamil Bożek
27fd9327f6
Disclose transactions in the flat transactions stream only to the stakeholders - test case [DPP-276] (#9028)
* Test case draft

* Proper test for disclosing events in the flat transactions stream only to stakeholders

* Improved assertion for TXNotDiscloseCreateToNonSignatory test

*
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-08 16:28:54 +01:00
Stefano Baghino
45b33757b2
Introduce a helper to limit signing to commands (#9050)
This allows the signing interceptor to be generic for now (which
is useful to test), while allowing the users to use a simple
static method to only sign commands directed to the command and
command submittion services.

This is intended to address https://github.com/digital-asset/daml/pull/9036#discussion_r588332093

https://github.com/digital-asset/daml/pull/9036 should be rebased
on the main branch after this has been merged.

changelog_begin
changelog_end
2021-03-08 16:26:02 +01:00
nickchapman-da
0c4d8ac19c
Speedy: rollback within exercise (#9040)
* WIP

At a baseline test in RollbackTest for an exercise when no exception thrown.

changelog_begin
changelog_end

* call abortExercises if exception throwing unwinds stack through KCloseExercise Kont frames
2021-03-08 15:11:48 +00:00
Gary Verhaegen
caa023b72e
ci/cron/check: remove dade-assist calls (#9048)
* ci/cron/check: remove dade-assist calls

We can only run this in a context where the dev-env is already set up
anyway, as that's how we get Bazel to build the script in the first
place.

CHANGELOG_BEGIN
CHANGELOG_END

* remove skip_java logic
2021-03-08 13:55:19 +00:00
Remy
85f0965ab9
LF: fix archive decoder for GREATER_DATE (#9047)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-08 13:42:02 +00:00
Gary Verhaegen
121534c54d
ci/cron/check: low-hanging perf improvement (#9042)
Two quick improvements I made while waiting on #9039:
- Avoid loading Java. When looking at the logs flow by this seemed to be
  taking a huge amount of time.
- Isolate the gcloud config files, which allows for running gcloud
  downloads in parallel.

Together these reduce the `check_releases` runtime from about 5 hours to
about 2. There's much more (and smarter) work needed on this, but this
was really easy to do.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-08 07:56:33 +00:00
azure-pipelines[bot]
fb0bd82740
update NOTICES file (#9045)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-08 08:47:32 +01:00
Gary Verhaegen
4fd42a6772
reduce noise on daily tests (#9039)
Getting a separate message per test was fine when there was one, but
this kind of got our of hand at this point.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-05 22:50:23 +00:00
Moritz Kiefer
5d88c08832
Bump ghcide (#9041)
* Bump ghcide

Includes https://github.com/digital-asset/daml-ghcide/pull/13 meaning
we can now remove the hacks for missing signatures

changelog_begin

- [Daml Compiler] Fix a bug where passing `--ghc-option=-Werror` also
  produced errors for warnings produced by `-Wmissing-signatures` even
  if the user did not explicitly enable this.

changelog_end

* Bump to merged commit

changelog_begin
changelog_end
2021-03-05 18:17:28 +01:00
Gary Verhaegen
41d62ea5c0
send CI failures on separate channel (#9038)
This will send CI failures to `#ci-failures-daml`.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-05 14:10:30 +00:00
Gary Verhaegen
e26d006f6f
slack: reduce @here mentions (#9037)
The people who care about these alerts monitor the channel closely
enough anyway, and having frequent automated @here bells ringing makes
it harder for individuals to highlight important messages.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-05 13:55:58 +00:00
Stefano Baghino
ae110361f0
Add more shared infrastructure for non-repudiation testing (#9035)
More dummy participant components, shared across the non-repudiation sub-tree.

These will also be used in client bindings tests.

changelog_begin
changelog_end
2021-03-05 14:48:03 +01:00
nickchapman-da
75140744aa
Speedy: building transactions with rollback (#8983)
* WIP

add LF examples to test try-catch & throw

add (integer) info to template for created contracts

call {begin,end,abort,rollback}Try during speedy on-ledger execution

add test to exercise abortTry

changelog_begin
changelog_end

* share boilerplate in test setup

* test value of created contracts, not just the count

* naming

* avoid Scenarios in tests. Just working directly with Update expressions

* use machine.withOnLedger, so crash if we are not
2021-03-05 12:33:22 +00:00
azure-pipelines[bot]
98410e7c7f
update compat versions for 1.11.0-snapshot.20210304.6422.0.d3d5042a (#9032)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-05 11:33:57 +01:00