Commit Graph

545 Commits

Author SHA1 Message Date
Stefano Baghino
9af85e56e9
Introduce DAML-LF value caching for transaction service (#6052)
* Introduce DAML-LF value caching for transaction service

Allows to keep the DAML-LF values in the most recently indexed events in memory,
so that they don't have to be deserialized on the client from their Protobuf encoding.

Closes #6044

CHANGELOG_BEGIN
[Sandbox] The --max-lf-value-translation-cache-entries option allows to set a
number of events for which DAML-LF values are cached. Could reduce latency in
serving transactions for consumers that are reasonably fast.
CHANGELOG_END

* Add missing dependency

* Address https://github.com/digital-asset/daml/pull/6052#discussion_r428076003

* Update ledger/sandbox/src/main/scala/com/digitalasset/platform/sandboxnext/Runner.scala

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

* Address https://github.com/digital-asset/daml/pull/6052#discussion_r428071324

* Address https://github.com/digital-asset/daml/pull/6052#discussion_r428076905

* Address https://github.com/digital-asset/daml/pull/6052#discussion_r428081294

* Fix fatal warnings

* //ledger/caching has to be used whenever sandbox is run

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-05-26 08:33:53 +00:00
Remy
9e456a1016
DAML-LF: Kill RelativeContractId (#5991)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-25 22:30:45 +02:00
Shayne Fletcher
17f6c29958
Remove --pure-lockfile to yarn (#6056)
changelog_begin
changelog_end
2020-05-20 13:41:46 -04:00
Shayne Fletcher
38f23dd08b
Write stdout log for yarn errors (#6050)
changelog_begin
changelog_end
2020-05-20 10:41:16 -04:00
Robin Krom
0f75971f52
language: send live event when reconnecting (#5982)
not sending a live event after reconnecting means that the 'loading'
indicater is never updated to 'false'. we could also not set it to
'true' on the 'close' event, but that can lead to a rather unintuitive
behaviour where data is stale and then suddenly updated without any
indication.

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-19 16:57:16 +02:00
Moritz Kiefer
84dbce9eba
Fix typescript tests (#5934)
The new version of eslint has broken stuff, see the linked issue.

changelog_begin
changelog_end
2020-05-11 19:37:53 +00:00
Samir Talwar
ab1c114c74
scala/bindings: Remove the unnecessary @silent annotation. (#5904)
This means we can remove the dependency on `silencer-lib`, which means
that users don't have to upgrade to Scala v2.12.11.

CHANGELOG_BEGIN
- [Scala Bindings]: We no longer require users to upgrade to Scala
  v2.12.11.
CHANGELOG_END
2020-05-08 09:39:02 +00:00
Moritz Kiefer
c21768c616
Make Bearer prefix optional in Haskell ledger bindings (#5858)
* Make Bearer prefix optional in Haskell ledger bindings

This matches the change to the JVM bindings in 1.0.0

changelog_begin

- [DAML Assistant] The ``daml ledger`` commands no longer require the
``Bearer `` prefix in the access token file. This matches the behavior
 of DAML Script and other SDK tools.

changelog_end

* Fix tasty crash

changelog_begin
changelog_end

* Update compatibility/bazel_tools/daml_ledger/Main.hs

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-05-06 12:09:34 +02:00
Oliver Seeliger
4c0ffdf4df
Upgrade scala compiler silencer to 1.6.0 (#5822)
* Upgrade scala compiler silencer to 1.6.0

CHANGELOG_BEGIN
CHANGELOG_END

* Adapt build bazel file to new targets

* Switch to silencer plugin scala 2.12.11 per Samir's feedback

rather than 2.12.8

* Add missed bazel files

* Review feedback from Leo
2020-05-04 21:30:46 +00:00
Leonid Shlyapnikov
51d96a7d2e
Upgrading to scalac/runtime v2.12.11, we have been on v2.12.6 for too long (#5807)
https://github.com/scala/scala/releases/tag/v2.12.11

Fixing scalac deprecation and unused variable warnings

`/:` is deprecated, using foldLeft
`:\` is deprecated, using foldRight

Removing unused deps reported by bazel buildozer + scalafmt

changelog_begin
Upgraded Scala compiler and Scala runtime to v2.12.11.
See https://github.com/scala/scala/releases/tag/v2.12.11
changelog_end
2020-05-01 11:37:07 -04:00
Stephen Compall
067f3c987d
require https reverse proxy (#5660)
* new --leak-passwords-firesheep-style option; functions to check forwarded protocol

* enforce https reverse-proxy in all JWT-accepting endpoints

* make HttpService.start take config record

* test that X-Forwarded-Proto or Forwarded is enforced

* use new start signature in daml-script tests

* use insecure http mode for ts codegen tests

* note on regex

* use insecure option in daml assistant integration tests

* log allowNonHttps setting

* add non-https option to more places in daml-assistant tests

* add non-https option to getting started guide

* rename --leak-passwords-firesheep-style to --allow-insecure-tokens

- per suggestion by @garyverhaegen-da, @hurryabit

CHANGELOG_BEGIN
- [JSON API] By default, checks that connections are made through a reverse-proxy
  providing HTTPS, ensuring that JWT tokens don't leak.  To disable this check,
  such as for development, pass ``--allow-insecure-tokens``.
  See `issue #5572 <https://github.com/digital-asset/daml/issues/5572>`_.
CHANGELOG_END

* daml start includes --allow-insecure-tokens by default

- as indicated by @cocreature
2020-04-28 14:33:38 -04:00
Leonid Rozenberg
95f10f1afb
Fix typo of extra d (#5762)
changelog_begin
changelog_end
2020-04-28 19:21:40 +02:00
Robin Krom
7e070d6d16
language:docs: fix link in README of generated ts. (#5745)
While it's replaced already in the shipped docs together with the npm
package, it wasn't fixed on docs.daml.com.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-28 16:10:13 +02:00
Robert Autenrieth
1128833ab5
Write a config on startup (#5591)
* Write a config on startup

CHANGELOG_BEGIN
CHANGELOG_END

* Rename things

* Make the initial ledger config configurable

* Demote log level for unsupported config writes

* Update ledger/sandbox/src/main/scala/com/digitalasset/platform/apiserver/services/admin/ApiConfigManagementService.scala

Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>

* Update ledger/sandbox/src/main/scala/com/digitalasset/platform/configuration/LedgerConfiguration.scala

Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>

* Move code around

* Delay config submissions on kvutils ledgers

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2020-04-21 20:35:42 +00:00
Moritz Kiefer
7c031f25f7
Upgrade rules_nodejs to version 1.6.0 (#5539)
* Upgrade rules_nodejs to version 1.6.0

closes #5367

This includes the fixes for the issues in jest that we’ve been seeing.

changelog_begin
changelog_end

* Fix eslint rules

* A bit of progress

* Try to add LinkablePackageInfo (doesn’t seem to work yet)

* Add rootDirs

* revert da_ts_library

* da_ts_library: add LinkablePackageInfo info

* Remove react hook workaround

Since rules_nodejs 1.6.0 this fails with the following error:
```
  ● Test suite failed to run

    Configuration error:

    Could not locate module react mapped as:
    /.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.

    Please check your configuration for these entries:
    {
      "moduleNameMapper": {
        "/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
      },
      "resolver": null
    }

      49 | // like a promis without being one.
      50 | /* eslint-disable @typescript-eslint/no-floating-promises */
    > 51 | var react_1 = __importStar(require("react"));
         |                            ^
      52 | var react_hooks_1 = require("@testing-library/react-hooks");
      53 | var index_1 = __importStar(require("./index"));
      54 | var events_1 = require("events");

      at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
      at Object.<anonymous> (index.test.js:51:28)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```

* rootDirs is not needed for tsc

This is only required for ts_project

* Update yarn Bazel packages

* docs/theme add missing dependencies

* Remove unused attribute module_root

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-17 12:03:33 +00:00
Shayne Fletcher
05f8b4b8ad
suppress type synonym for Key when it's undefined (#5587)
changelog_begin
changelog_end
2020-04-16 13:39:08 -04:00
Shayne Fletcher
843c6d5d0f
Add ledger associated type synonyms (#5571)
changelog_begin
changelog_end
2020-04-16 14:12:45 +00:00
Shayne Fletcher
7b52823b60
stop hard coding ports (#5553)
changelog_begin
changelog_end
2020-04-14 14:15:57 -04:00
Shayne Fletcher
6cd5280867
add support for daml2js to write daml-ledger code (#5550)
changelog_begin
changelog_end
2020-04-14 16:14:49 +00:00
Robin Krom
6f017cbd39
language: reconnect websocket on closed connection (#5416)
This change adds a simple retry logic to the ledger websocket methods.

If the connection is closed, and the connection was previously healthy,
meaning it was live and running for more than 30 seconds, we try to
reconnect to the stream.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-14 10:32:36 +00:00
Rohan Jacob-Rao
7fafe6a08a
GSG: test messaging feature and remove copied code snippets (#5507)
* Diff with messaging feature and some noise manually removed

* Bazel target to use patch file in other build targets

* Patch file as data dep for integration tests

* Attempt to patch and test messaging feature in create-daml-app test

changelog_begin
changelog_end

* Use exports_files instead of filegroup

* Remove file existence checks that don't make sense

* Add patch to dev_env and reference it from integration tests

* Include patch on windows for later

* Set up yarn env again after codegen

* Restore file check

* Fix typo in comment on util function

* Add Tasty steps to make process explicit

* Use messaging patch for code snippets in GSG

* Use messaging code from template instead of copy

* Remove copied message code

* Refactor script to copy template code with messaging patch

* No need to retry second yarn install (only local deps should be updated)
2020-04-11 19:53:44 +00:00
Martin Huschenbett
06632267e9
Haskell: Add assertFileExists to DA.Test.Util (#5520)
* Haskell: Add assertFileExists to DA.Test.Util

This PR adds a helper function `assertFileExists` that captures the
`doesFileExist ... >>= assertBool ...` pattern that is very common in
our Haskell test suites. It also adds the inverse
`assertFileDoesNotExist` function. Both functions are now used where
appropriate.

CHANGELOG_BEGIN
CHANGELOG_END

* Add directory dependency to test-util lib

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-09 15:29:31 +00:00
Martin Huschenbett
a469d9471b
daml2js: Write all dependencies into package.json (#5490)
* daml2js: Write all dependencies into package.json

The dependencies of the generated packages on `@daml/types` and
`@mojotech/json-type-validation` somehow got lost either during a
refactoring or when changing how we depend on other generated packages.

The tests simply running `daml2ts` did not catch this because we write
a temporary `package.json` one level higher up in the directory tree
in order to `yarn install` the dependencies only once.

The tests using the generated packages also did not catch this since
they depend on `@daml/types`, which in turn depends on
`@mojotech/json-type-validation`, themselves and hence had both packages
in scope.

In other words, this is very tricky to test. We try to alleviate this
problem by sharing the code for generating the dependencies between the
function writing the actual `package.json` files and the one writing
the temporary `package.json`. This makes sure that these dependencies
do not get out of sync again.

CHANGELOG_BEGIN
CHANGELOG_END

* Adjust tests

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-08 11:44:05 +00:00
Gerolf Seitz
cadaa59ad8
Fix redirects, java-bindings javadoc, and live-preview.sh (#5446)
* Fix redirects, java-bindings javadoc, and live-preview.sh

- javadoc_library now supports sources from filegroups as well
- //language-support/java:javadoc now generates javadoc for ledger-api, java bindings, rxjava bindings
- live-preview.sh refers to the correct javadoc target //language-support/java:javadoc
- removed leading / from redirects.map

* Only generate daml-lf javadocs if not on windows

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-07 09:53:26 +02:00
Martin Huschenbett
28627a665f
Refactor daml-assitant ITs regarding yarn (#5453)
There are multiple tests that setup an environment for yarn to find
our TypeScript libraries. This PR unifies this setup by providing a
single function to set it up.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-06 18:28:39 +02:00
Leonid Shlyapnikov
26c476d3cd
Commenting out local maven repository config (#5452)
changelog_begin
changelog_end
2020-04-06 11:55:13 -04:00
Robin Krom
a413c1cec9
Replace file globs in ts package BUILD files (#5424)
* language: remove file globs from ts library BUILD files

We replace them with toplevel `sources` definition, since sadly
filegroups don't work here.

CHANGELOG_BEGIN
CHANGELOG_END

* update app-arch section regarding daml2js

* remove files field in tsconfigs
2020-04-06 13:07:27 +02:00
Remy
943919585e
Java Codegen: fix template code generation (#5443)
* Java Codegen: fix template code generation

* changelog

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-06 08:56:51 +00:00
Leonid Shlyapnikov
0842866f25
Scala codegen examples: removing bintray repository, relying on daml.yaml and -Dda.sdk.version (#5436)
* Removing bintray repository, relying on daml.yaml and -Dda.sdk.version

to resolve DA SDK version.

changelog_begin
changelog_end

* Fixing scalafmt

* cleanup

* Adding local maven repo, you can run examples against local dev SDK 0.0.0
2020-04-06 06:33:30 +00:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Martin Huschenbett
a5b41245b0
dam2js: Write better package.json files (#5417)
If we don't provide a license field, `yarn install` issues a warning.
Since sharing the code generated by `daml2js` with others doesn't
make sense in the current state, I've chosen the most conservative
license, namely "UNLICENSED", such that our users don't accidentally
license their code to others.

In the long run, we should make this field configurable.

This PR also improves the description of the generated packages and
makes the temporary `package.json` used by `daml2js` private.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-03 11:58:31 +00:00
Martin Huschenbett
2a10109ed2
daml2js: Remove the -p flag (#5402)
The `-p` flag was used to provide the path to the `package.json` which
constitutes the root of a yarn workspace. Since we don't use yarn
workspaces anymore, this flag is now useless and the PR removes the
last traces of it.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-03 08:24:28 +00:00
Martin Huschenbett
31c78d57d8
Rename daml2ts to daml2js (#5394)
This is pretty much a search and replace over the whole repo, including
file names.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-03 08:58:23 +02:00
Moritz Kiefer
2b036f596d
Fix yarn invocation in daml2ts on Windws (#5391)
The details are in a source code comment but roughly it boils down to
`proc` vs `shell` looking things up in PATH differently.

changelog_begin
changelog_end
2020-04-02 21:33:57 +00:00
Robin Krom
100375e777
Update ts packages docs (#5375)
* language: docs: fix links in README's and update docs of useQuery

This fixes the broken links to the generated documentation and updates
the docs on `useQuery`, `useStreamQuery`.

CHANGELOG_BEGIN
CHANGELOG_END

* added two examples for useQuery/useStreamQuery without a query parameter
2020-04-02 15:36:08 +00:00
Robin Krom
d4073b94a0
Signal live streaming with 'live' event (#5329)
* language: daml-ledger: emit live events on streams

We handle the 'offset' field of the JSON API correctly and inform
downstream consumers with a 'live' event once we know that the stream is
live updating.

CHANGELOG_BEGIN
CHANGELOG_END

* Update language-support/ts/daml-ledger/index.ts

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

* Fix tests

CHANGELOG_BEGIN
CHANGELOG_END

* Run build-and-lint-test only on Linux

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-04-02 15:31:26 +00:00
Martin Huschenbett
c627c7e74c
@daml/react: Fix name of loading flag in README (#5370)
The actual name of the flag is `loading`. For some reason the README
claims it's `isLoading`. This PR fixes the README.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-02 12:04:55 +00:00
nickchapman-da
ddc11a7063
Refactor deployment tests: (#5342)
- Move deployment tests (deployTest, fetchTest) out of integration-tests.
- Use DA.Test.Sandbox where appropriate.
- Split out code for useful test patterns: i.e. calling commands quietly, getFreePort.

changelog_begin
changelog_end
2020-04-02 10:17:21 +01:00
Samir Talwar
4b9b2e8e61
Reduce the default maximum heap size for all Scala processes. (#5350)
* bazel_tools: Set `unused_dependency_checker_mode` in one place.

* bazel_tools: Set the default max heap size for Scala processes to 2GB.

And the default initial max heap size to 512MB.

CHANGELOG_BEGIN
CHANGELOG_END

* bazel_tools: Set the `scalac` heap size to 2GB and stack size to 2MB.

* bazel_tools: Delete `da_scala_macro_library`, as it's unused.

* bazel_tools: Revert the description of `da_scala_library_suite`.

Misread it.
2020-04-01 19:14:20 +00:00
Shayne Fletcher
6daf37c4da
Make 'callProcessSilent' a test library function (#5323)
changelog_begin
changelog_end
2020-04-01 11:42:00 +00:00
Martin Huschenbett
921d0efd40
TS libs: Fail linting on warnings (#5332)
Currently, we happily accept warnings of the linter as if nothing was
wrong. This PR makes the linter fail on warnings. It also moves the
linter configs into their own files to unclutter the `package.json`
files we ship to our users a bit.

Ane we fix all the warnings we now get from the linter.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-01 11:41:35 +00:00
Gerolf Seitz
329320bad9
Organize maven coordinates (#5272)
* Use com.daml as groupId for all artifacts

CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END

* Add 2 additional maven related checks to the release binary

1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId

* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
2020-04-01 11:41:18 +02:00
Gerolf Seitz
9b5c5e23b9
Make wall clock time the default time mode for sandbox (#5301)
CHANGELOG_BEGIN
[Sandbox] Wall Clock Time mode (``--wall-clock-time``) is now the
default.
CHANGELOG_END
2020-04-01 08:53:16 +02:00
Moritz Kiefer
b43b8d8672
Fix build-and-lint tests (#5320)
Hardcoding the version breaks as soon as we make a release PR where
the version is not 0.0.0.

changelog_begin
changelog_end
2020-03-31 20:20:56 +02:00
Shayne Fletcher
6e94197544
Upgrade daml2ts test to a more recent davl (#5316)
changelog_begin
changelog_end
2020-03-31 18:10:46 +00:00
Samir Talwar
52dbcf5d95
postgresql-testing: "Lock" ports when starting PostgreSQL. (#5310)
* http-json: Ask for a free port by specifying port 0.

This will avoid race conditions.

* bindings-akka-testing: Delete RandomPorts; it's unused.

* ports: Fix the Bazel test glob.

* ports: Move FreePort to postgresql-testing and add a test case.

* postgresql-testing: Make `FreePort.find()` return a `Port`.

* postgresql-testing: Lock free ports until the server starts.

This uses a `FileLock`, which should work well on all our
supported operating systems as long as everyone agrees to use it.

CHANGELOG_BEGIN
CHANGELOG_END

* postgresql-testing: Try to find a free port 10 times, then give up.

* postgresql-testing: Use a shared directory for the port lock.

* postgresql-testing: Try an alternative way of getting `%LOCALAPPDATA%`.
2020-03-31 17:13:02 +00:00
Robin Krom
8e9a8388ad
language:daml-ledger: tests for streaming methods (#5180)
This adds tests mocking a websocket server to test the
`streamFetchByKey` and `streamQuery` methods of the ledger class.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-31 16:03:19 +00:00
Shayne Fletcher
5cd3311a90
Mark build-and-lint-test flaky (#5309)
changelog_begin
changelog_end
2020-03-31 14:07:17 +00:00
Martin Huschenbett
bb1c5ed039
TypeScript libs: Enforce return type annotations (#5287)
For some reason the lint that catches missing return type annotations
was turned off. It's turned on now.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-31 09:28:23 +00:00
Andreas Herrmann
89a9f5c7d2
tarball reproducibility (#5258)
* integration-tests reproducibility

* package-app reproducibility

* Make remaining tar czf reproducible

* package-app

CHANGELOG_BEGIN
CHANGELOG_END

* Reproducibility of remaing tar invocation

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-31 10:09:52 +02:00