Commit Graph

4769 Commits

Author SHA1 Message Date
Stephen Compall
134dbf0f01
check NodeFetch value_version on decode (#6786)
- decoding the template ID is not valid if the value_version is unknown

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-20 11:23:05 +02:00
Gerolf Seitz
e199bfac20
Ledger API: Only set the offset in the last ACS response (#6782)
* Ledger API: Only set the offset in the last ACS response

Fixes #6757.

CHANGELOG_BEGIN
[Sandbox][DAML Integration Kit]: Bug Fix: The ActiveContractService now only sets
the offset in the last response again instead of in every response
element.
CHANGELOG_END
2020-07-20 09:22:37 +02:00
Robert Autenrieth
2159de08ef
Remove index transformation (#6789)
The index is not part of the public API,
and we want to always enable metrics.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-20 09:20:59 +02:00
dependabot[bot]
68f46db36e
Bump lodash from 4.17.15 to 4.17.19 in /compiler/daml-extension (#6778)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>

changelog_begin
changelog_end

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-07-17 19:16:20 +00:00
Robert Autenrieth
1074736316
Add time model documentation (#6705)
* Update existing docs, removing references to old time model
* Add detailed time model docs
* Rename ledger effective time to ledger time

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-17 17:54:26 +02:00
Robert Autenrieth
7ce9748066
Split sandbox code into separate packages (#6695)
* Move public code into daml-integration-api

CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
2020-07-17 17:06:06 +02:00
Gary Verhaegen
f173ea966a
remove clean --expunge (#6783)
All nodes have been restarted following #6770, let's get our Windows
nodes back to some semblance of performance.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-17 15:00:10 +00:00
Moritz Kiefer
b12a19181e
Add a --timeout option to daml ledger commands (#6774)
* Add a --timeout option to `daml ledger` commands

At least in the compatibility tests, we see timeouts fairly regularly
in the tests for fetch-dar. If we see those timeouts in the tests, I
see no reason why our users should never see that especially on
actually distributed ledgers and hardcoded timeouts are a bad idea
anyway so now they are configurable.

I’ve also changed the timeout for the fetch tests (those are the only
once I’ve seen timeout) to 120s.

changelog_begin

- [DAML Assistant] The `daml ledger` commands now accept a `--timeout`
  option which can be used to change the timeout for API requests from
  the default of 30s.

changelog_end

* Generate parties as uuids to avoid collisions

changelog_begin
changelog_end

* Match on Deadline Exceeded as well

changelog_begin
changelog_end

* Update compatibility/bazel_tools/test_utils/DA/Test/Util.hs

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Add some docs

changelog_begin
changelog_end

* Better version checks

changelog_begin
changelog_end

* fix imports

changelog_begin
changelog_end

* This is not how you exception

changelog_begin
changelog_end

* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-07-17 16:35:55 +02:00
fabiotudone-da
ebf312873c
Add CachingDamlLedgerStateReaderWithFingerprints factory method (#6769)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-17 16:07:54 +02:00
dependabot[bot]
2904a5d236
Bump lodash from 4.17.15 to 4.17.19 in /language-support/ts/packages (#6779)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>

changelog_begin
changelog_end

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-07-17 13:54:13 +00:00
Stefano Baghino
31b8e3c661
Remove documentation to use Navigator using old assistant (#6773)
This references the old assistant and the deprecated configuration
files. The positioning for Navigator right now (correct me if I'm wrong)
is that it's a development tool, so I don't think it makes a lot of
sense publicizing it otherwise.

Closes #3237

changelog_begin
changelog_end
2020-07-17 12:24:33 +00:00
dependabot[bot]
30c91a0d07
Bump lodash from 4.17.15 to 4.17.19 in /navigator/frontend (#6767)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>

changelog_begin
changelog_end

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-07-17 12:08:29 +00:00
dependabot[bot]
a8957d19e6
Bump lodash from 4.17.15 to 4.17.19 (#6766)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>

changelog_begin
changelog_end

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-07-17 12:02:51 +00:00
Moritz Kiefer
96216557ab
Undo lodash hack (#6771)
lodash released a non-broken version now and while dependabot isn’t
smart enough to realize this one has a vulnerability it still seems
like a good idea to bump it.

changelog_begin
changelog_end
2020-07-17 12:01:23 +00:00
Moritz Kiefer
147a2700c0
Bump Windows cache (#6770)
To “fix” the “output was not created” errors.

changelog_begin
changelog_end
2020-07-17 12:41:10 +02:00
Gerolf Seitz
35b50992f0
Allow running StandaloneApiServer in read-only mode. (#6721)
Also fixes #5635, removing usage of `ReadService` from the `StandaloneApiServer`.

The configuration stream of the Ledger API LedgerConfigurationService is now properly backed by the configuration entries instead of just serving the initial configuration.

CHANGELOG_BEGIN
[DAML Integration Kit]: ``StandaloneApiServer`` can now be run in a read-only mode.
  - The type of the constructor parameter ``writeService`` of ``StandaloneApiServer`` changed to ``Option[WriteService]``. Passing ``None`` will not start any of the admin services, the command service, and the command submission service.
  - The constructor parameter ``readService`` of ``StandaloneApiServer`` has been removed.
  - A new constructor parameter ``ledgerId`` has been added to ``StandaloneApiServer``. It is used to verify that that ``StandaloneApiServer`` is run against an index storage for the same ledgerId. Initialization is aborted if this is not the case.
[DAML Integration Kit]: The ``LedgerConfigurationService`` now properly streams configuration changes.
CHANGELOG_END
2020-07-17 09:57:00 +02:00
Remy
c32355b9fd
DAML-LF: compute the newest version of LF used by a package (#6747)
We add a method to CompiledPackages that returns the latest language version that package uses directly or indirectly through is dependencies.

This will be used in the context of #5164

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-17 09:40:21 +02:00
Leonid Shlyapnikov
c203cd3600
Fixing a flaky test (#6765)
* fixing flaky test, WIP

* introducing `storeSync`

to guarantee that transactions get processed in the specified order

removing unnecessary assertion that hid the fact that the order of offsets
read from DB was invalid

* cleaning up

changelog_begin
changelog_end

* addressing code review comments, thanks @S11001001!

* suspend it

* cleaning up a bit more
2020-07-16 19:27:27 -04:00
Gary Verhaegen
92c4aac0f9
remove warnings from skeletn template (#6762)
If a user were to follow @cocreature's [suggestions][0] for compiler
warnings, the current skeleton template would generate two warnings.
That seems like it may give a bad impression, so I believe we should fix
it, even though the warnings are not enabled by default.

[0]: https://discuss.daml.com/t/making-the-most-out-of-daml-compiler-warnings/739

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-16 18:36:30 +02:00
Andreas Herrmann
4be01e4df6
Update ghcide 0.1.0 --> 0.2.0 (#6745)
* Update rules_haskell hie-bios support

* Decouple Haskell ghcide and DAML ghcide

Creates a separate `stack_snapshot` to pull in `ghcide` for the Haskell
IDE use case independent of the `ghcide` for DAML. This allows to update
these two `ghcide` instances independently. As DAML uses `ghcide` the
library updates can be involved if the API experienced breaking changes.
At the same time we may wish to update `ghcide` for Haskell earlier to
make use of new features and stay compatible with rules_haskell's ghcide
support.

* Fix Haddock warnings reported by ghcide

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-16 16:30:09 +00:00
Leonid Shlyapnikov
7220f805ab
ignoring a flaky test (#6759)
* ignoring a flaky test

* mark ignored test with a github issue

changelog_begin
changelog_end
2020-07-16 15:49:56 +00:00
Remy
8d74754450
kv tools: fix typos in kvutils tools README (#6758)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-16 16:59:34 +02:00
Rohan Jacob-Rao
8668bb78f9
Release testing: Fix line number in quickstart template code (#6750)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-16 10:08:31 -04:00
azure-pipelines[bot]
f9b3957e81
update compat versions for 1.3.0 (#6754)
* update compat versions for 1.3.0

CHANGELOG_BEGIN
CHANGELOG_END)

* fix sandbox-migration for new LedgerIdRequirement

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-07-16 16:03:03 +02:00
Leonid Shlyapnikov
c39bd0764b
Additional stream pagination integration tests (#6751)
* Additional test cases for #6698

tested the failure on the previous implementation of `com.daml.platform.store.dao.events.EventsRange`

changelog_begin
changelog_end

* Addressing code review comments

thanks @S11001001!

* EventsRange.isEmpty test case

* decrease the number of runs

* thanks @stefanobaghino-da

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* thanks @stefanobaghino-da

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* thanks stefanobaghino-da

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* thanks @stefanobaghino-da

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* updating comments

* cleaning up

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-07-16 13:46:55 +00:00
Gary Verhaegen
34ca4f7219
run tests on auto-compat pr (#6755)
While closely following the 1.3 release through our pipeline to check
that #6709 worked as expected, I realized that the automatically-created
PR does not start the normal tests either, presumably because it's been
opened by a bot. The bot doe shave write access to the repo (obviously,
as it can create the PR in the first place), but somehow that doesn't
seem to count as a PR with write access for Azure.

So this PR adds the normal test run too, so we don't need to manually
say `/azp run` on the PR.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-16 11:36:22 +00:00
Gary Verhaegen
277e652b55
remove dade-base-profile (#6743)
It does not seem to be used anymore, and has been deprecated since Jul
2018.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-16 11:17:42 +00:00
Moritz Kiefer
ebacb34caa
Release SDK 1.3.0 (#6753)
changelog_begin
changelog_end
2020-07-16 12:07:34 +02:00
Stefano Baghino
a4b666a7ee
Document Sandbox metrics (#6662)
* Document Sandbox metrics

Closes #5573

changelog_begin
changelog_end

* Selected a list of relevant metrics to document

* Provided a description of selected namespaces and metrics

* Alphabetical ordering

* Fix nested list

* Visual fixes, removing cache metrics

* No need to mock Metrics

* Update docs/source/tools/sandbox.rst

Co-authored-by: Peter Garmaz <40190826+pgarmaz-da@users.noreply.github.com>

* Adapts to recent change in metrics CLI option format

* Update docs/source/tools/sandbox.rst

Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>

* Address https://github.com/digital-asset/daml/pull/6662#discussion_r455163515

* Document the jvm metrics namespace

* Document HikariCP metrics

* Fix title underline

Co-authored-by: Peter Garmaz <40190826+pgarmaz-da@users.noreply.github.com>
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
2020-07-16 09:31:49 +00:00
Remy
bdcfb6c114
DAML-LF: add templateId method to GenNode (#6752)
* DAML-LF: add templateId abstract method to GenNode

* changelog

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-16 10:17:04 +02:00
Remy
15da222939
DAML-LF: move some type definitions out of Transaction object (#6739)
SubmittedTransaction and CommittedTransaction are moved from com.daml.lf.transaction.Transaction to
com.daml.lf.transaction

This helps intelliJ type inference.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-16 09:13:30 +02:00
Moritz Kiefer
91302d0f4a
Fix merge conflict in KVUtils test adapter (#6748)
NodeId from the Transaction object got deprecated.

changelog_begin
changelog_end
2020-07-15 19:30:01 +00:00
Remy
471863570a
Replay benchmark: adapt dar to ledger export (#6661)
Try to match transactions from the ledger export to a different non original dar.
This is useful to benchmark a different compilation of the same original daml source.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-15 21:10:35 +02:00
Moritz Kiefer
674c37cf5a
Stop publishing a tarball with the protobuf files to Maven (#6742)
This was never intentional, nobody even knew that this was possible
and we have an alternative, documented way of getting this via github
releases.

To avoid introducing this issue again, I’ve removed non-jar artifact
types from the Maven upload script.

fixes #448

changelog_begin
changelog_end
2020-07-15 18:48:49 +02:00
Rohan Jacob-Rao
43bf8ab7c9
Release 1.4.0 snapshot (#6741)
changelog_begin
changelog_end
2020-07-15 12:00:54 -04:00
Moritz Kiefer
882027f9d7
Stabilize DAML REPL (#6740)
changelog_begin

- [DAML REPL] DAML REPL has moved out of early access status and is
  now a stable feature of the SDK.

changelog_end
2020-07-15 14:46:30 +00:00
Moritz Kiefer
d6e5862645
Add platform-version field to daml.yaml (#6736)
* Add `platform-version` field to `daml.yaml`

This PR adds the `platform-version` field to `daml.yaml`. Based on the
approach agreed upon in #6558, the logic for this all sits in
`daml-helper` and there are no changes to the assistant.

The details of how the logic work are in a comment so I’m not going to
repeat them here but the commands that are affected are:

- `daml sandbox`
- `daml sandbox-classic`
- `daml json-api`
- `daml start` (but only for sandbox and the JSON API, not for
  Navigator or anything else)

For tests, I’ve added a test to the compat workspace that installs two
SDKs simultaneously and tries out various combinations and verifies
that we get the correct version. Open to other ideas for testing this
but that seemed like the most sensible option that actually tests what
we run.

changelog_begin

- [DAML Assistant] You can now specify the version of Sandbox and the
  JSON API independently of your SDK version by setting
  ``platform-version`` in your ``daml.yaml``. This is useful if you
  are deploying to a ledger that is running components from a
  different SDK version. See
  https://docs.daml.com/tools/assistant.html#project-config-file-daml-yaml
  for details.

changelog_end

* Run platform-version tests

changelog_begin
changelog_end

* Fix tag globbing

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* Try to fix env vars

changelog_begin
changelog_end

* Remove hardcoded references to 1.2.0

changelog_begin
changelog_end

* Rephrase doc comment

changelog_begin
changelog_end

* get things to compile

changelog_begin
changelog_end

* maybe fix things for realz

changelog_begin
changelog_end

* Remove debugging output

changelog_begin
changelog_end

* Get angry at windows

changelog_begin
changelog_end

* why is windows

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2020-07-15 16:30:01 +02:00
Miklos
f7bbfc2a1b
[kvutils] Added initial set of metrics for pre-execution (#6738)
* Added metrics for decoding and total time of pre-execution.
CHANGELOG_BEGIN
CHANGELOG_END

* Code tidying.

* Added timer for tracking time spent with generating write sets.
2020-07-15 14:27:14 +00:00
Leonid Shlyapnikov
9abfff0388
Fixing typo in the JSON API error response (#6735)
changelog_begin
changelog_end
2020-07-15 09:23:40 -04:00
Miklos
8e04b2add9
[kvutils] Do not expose fingerprints to Committers (#6737) 2020-07-15 14:59:44 +02:00
Miklos
761787ce4b
[kvutils] Pre-executing submission validator (#6711)
* Removed engine constrcutor parameter.

* Basic abstractions for pre-executing submission validator.

* Fix build.

* Shared byteToLogEntryId with BatchedSubmissionValidator.

* Added some more metrics.

* Removed involved participants from PreExecutionResult.
CHANGELOG_BEGIN
CHANGELOG_END

* Reformatted/added headers.

* Fix build.

* Code tidying.

* Moved PreExecutionOutput into a separate file.

* Moved type definition into companion object.

* Added some tests for PreExecutingSubmissionValidator.

* Update ledger/participant-state/kvutils/src/main/scala/com/daml/ledger/validator/preexecution/LedgerStateReaderWithFingerprints.scala

Co-authored-by: Simon Meier <meiersi-da@users.noreply.github.com>

* Removed left-in code.

* Added some assertions for .zip operations.

* Test result for unexpected message.

* Test setting min/max record time.

* Reuse code for generating a key-value pair for log entries.

* Tests for CachingDamlLedgerStateReaderWithFingerprints.

* Store empty values as well with their fingerprints.

* Moved weight function into companion object.

* Test that the results come back in right order.

* Code tidying.

* Basic test case for LogAppenderPreExecutingCommitStrategy. Added missing headers.

* Code tidying.

* Fix build.

* Fix build.

* Revert "Store empty values as well with their fingerprints."

This reverts commit ca68ea967f.

# Conflicts:
#	ledger/participant-state/kvutils/src/test/suite/scala/com/daml/ledger/validator/caching/CachingDamlLedgerStateReaderWithFingerprintsSpec.scala

* Added some docs.

* Added test case for ensuring that we don't cache None values.

* Fix build.

Co-authored-by: Simon Meier <meiersi-da@users.noreply.github.com>
2020-07-15 08:55:19 +00:00
Moritz Kiefer
da995886bb
Fi setvar call in compat update PR (#6734)
The revert that removed the use of lib.sh broke this again.

changelog_begin
changelog_end
2020-07-15 10:42:16 +02:00
Moritz Kiefer
10beda69d3
Release RC 2 for SDK 1.3.0 (#6731)
This includes (only) the bugfix for the ledger offset stuff in the
ledger API server.

changelog_begin
changelog_end
2020-07-15 00:34:32 +02:00
Moritz Kiefer
52b9eabbcc
Revert "refactor ci jobs: add setvar to ci/lib.sh (#6708)" (#6732)
This reverts commit 61e9df3eaf.

This interacts very badly with the fact that we check out old commits
for releases. While we could fix it for this particular issue, I don’t
think this buys us enough to make this worth doing and it makes it
easy to introduce issues in the future if we modify lib.sh

changelog_begin
changelog_end
2020-07-14 23:53:49 +02:00
Moritz Kiefer
4ac0a47c81
Fix typing of nested decoders in daml2js (#6729)
I accidentally broke this when I added the types (very sorry about
that). Unfortunately while the tests should actually have caught this,
they didn’t because they are also broken. The `&&` results in bash
assuming the exit code is handled and `set -e` doesn’t kick in so
despite an error things proceeded just fine.

The change turns
```
  Add2: (<a_a1Ag>(a_a1Ag: damlTypes.Serializable<a_a1Ag>) => damlTypes.Serializable<Expr2.Add2>);
```
into
```
  Add2: (<a_a1Ag>(a_a1Ag: damlTypes.Serializable<a_a1Ag>) => damlTypes.Serializable<Expr2.Add2<a_a1Ag>>);
```

which is obviuosly what we want and also fixes the test. I did verify
that the tests fail afterwards without the change to daml2js.

changelog_begin
changelog_end
2020-07-14 19:22:55 +00:00
Leonid Shlyapnikov
918bae2ed4
Fix for event_offset to event_sequential_id conversion (#6712)
* adding a test case for the offset order returning expected transactions,

moving the exiting test case under flat transactions behavior group

* Fixing event_offset to event_sequential_id conversion

handling the conversion on an empty ledger,
adding a bit more logging,
skipping DB query on an empty range

CHANGELOG_BEGIN
CHANGELOG_END

* addressing code review comments

simplifying the query
returning SQL that forces the event_offset index usage

* addressing code review comments

simplifying the query
returning SQL that forces the event_offset index usage

* simplifying it a bit more
2020-07-14 21:11:51 +02:00
Rohan Jacob-Rao
619209d0b1
Trigger service: Document steps to replicate auth flow from the command line (#6725)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-14 15:05:41 -04:00
Moritz Kiefer
38e1637a6a
Bump timeout of repl tests (#6728)
We’ve added a bunch more tests which now result in us hitting the
timeout sadly.

changelog_begin
changelog_end
2020-07-14 20:37:59 +02:00
Gary Verhaegen
8176fb0c8d
fix the the typo (#6723)
```
s/the the /the /
```

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-14 16:55:33 +00:00
Brian Healey
520089778d
default Engine to StableConfig as that is what you want if you are not in dev mode (#6679)
CHANGELOG_BEGIN
Default new Engine constructor to Engine.StableConfig so it does not
need to be overridden unless you specifically want to run in
Engine.DevConfig mode
CHANGELOG_END

Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
2020-07-14 09:46:55 -04:00