Commit Graph

3122 Commits

Author SHA1 Message Date
Moritz Kiefer
7124479647
Add subtract to the daml-stdlib (#4523)
changelog_begin

- [DAML Standard Library] Add a ``subtract`` function which is useful
  as a replacement for sections of ``(-)``, e.g., ``subtract 1`` is
  equivalent to ``\x -> x - 1``.

changelog_end
2020-02-14 13:55:47 +00:00
Samir Talwar
a589f4af0b
sandbox: Move more resource acquisition into the owner. (#4501)
* sandbox: Move more resource acquisition into the `owner`.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Reimplement SandboxClientResource as a resources.Resource.

* codegen: Use resources in TestUtil.

* sandbox: Manage PostgreSQL in tests with ResourceOwners.
2020-02-14 13:52:45 +00:00
Gary Verhaegen
433f484188
stop producing/publishing the Bazel execution logs (#4527)
They can weigh close to 1GB, and the internal Azure networks are
unreliable enough that this adds up significant amounts of time to our
already slow CI pipeline (usually around a minute, but I've seen at
least one case where it took almost 28 minutes).

Given that we pretty much never look a them, 🔥.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-14 14:34:01 +01:00
Moritz Kiefer
b386f07588
Mark Shake tests as flaky (#4526)
changelog_begin
changelog_end
2020-02-14 13:31:50 +00:00
Gary Verhaegen
d96b225dad
remove jfrog credentials after use (#4522)
Not critically important, but it seems like good hygiene.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-14 13:54:28 +01:00
Gary Verhaegen
c8e6486c79
pin Terraform plugin versions (#4519)
We're currently depending on a floating "latest", which is often a bad
idea. Today my machine decided to upgrade the google plugin,w hich is no
specifying some new fields for the GCS objects, and therefore `terraform
plan` doe snot look clean anymore, even though there has been no change
to the terraform files (nor to the infrastructure).

This PR aims to make our Terraform setup more reproducible by pinning
Terraform plugin versions. It's also a way to track the application of
the "new" Terraform setup, as it is technically a standard change
(though hopefully a very safe one).

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-14 13:52:27 +01:00
Moritz Kiefer
1fce415359
Only extract main dalf from a dependency (#4515)
This is a spin off from my fixes for making data-deps work with
typeclasses cross-SDK.

We only have the interface files for the main dalf so it doesn’t
really make sense to extract the other dalfs. The current behavior of
extracting all dalfs results in them being picked up by
`GeneratePackageMap` even if GHC doesn’t know about them which causes
issues in other placse.

I’ve adapted the collision check to check for transitive dependencies
when creating the project db.

changelog_begin
changelog_end
2020-02-14 13:33:48 +01:00
Moritz Kiefer
514b8596a4
Clarify SDK version in DAML studio multi-package docs (#4518)
changelog_begin
changelog_end
2020-02-14 12:11:03 +00:00
Moritz Kiefer
6b9c0407c7
Fix error messages on incorrect --package flags (#4516)
The string needs to include the full flag instead of only including
--package. Otherwise you get rather unhelpful error messages.

changelog_begin
changelog_end
2020-02-14 10:36:29 +01:00
Gerolf Seitz
0b718b2657
Remove empy lines in proto docs to fix rendering on docs.daml.com (#4498)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-14 10:19:02 +01:00
Remy
8394f4ba7d
[DAML-LF] Version new contractId in Proto values (#4460)
* version new contractId

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-13 20:45:46 +01:00
Miklos
a440a3fabd
Simplified API fixes (#4505)
Removed LedgerStateAccess.participantId and renamed LedgerReader.retrieveLedgerId => ledgerId.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-13 19:17:07 +01:00
Martin Huschenbett
c958aecf5e
@daml/ledger: Add workaround for websocket proxying issues (#4503)
Unfortunately, the development server of `create-react-app` does not
proxy websockets properly. Thus, we need to connect to the JSON API
directly when using it. This now be done by setting the
`REACT_APP_JSON_API_PORT` environment variable in `.env.development`.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-13 16:23:09 +00:00
Remy
4137ff510b
[Engine] Adding a specific type for ParticipantId (#4504)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-13 17:15:34 +01:00
Samir Talwar
2c4807f3bf
release: Sort artifacts.yaml. (#4502)
I verified that there are no semantic changes, only cosmetic ones, by
running the following before and after, and checking the diff.

```
< release/artifacts.yaml ruby -rjson -ryaml -e 'puts JSON::generate YAML::load $stdin' | jq '. | sort_by(.target)' > ~/Desktop/artifacts.{old,new}.json
```

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-13 13:52:48 +00:00
Moritz Kiefer
71ddad381e
Improve docs around state-driven apps (#4496)
changelog_begin
changelog_end
2020-02-13 11:18:18 +01:00
Moritz Kiefer
8015119c81
Fix release notes for 0.13.52 (#4497)
changelog_begin
changelog_end
2020-02-13 09:11:23 +00:00
nickchapman-da
d084dba531
Subscribe to CI notifications (#4494)
changelog_begin
changelog_end
2020-02-12 17:36:03 +00:00
nickchapman-da
626a89eead
prepare release 0.13.52 (#4491)
changelog_begin
changelog_end
2020-02-12 15:01:25 +00:00
Miklos
34eec2399a
Index doesn't know about ReadService (#4475)
Made JdbcIndex not depend on ReadService.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-12 15:49:12 +01:00
Moritz Kiefer
73f60a44cb
Wrap -> kind in parentheses in data-dependencies (#4492)
Previously we translated (* -> *) -> * -> * to * -> * -> * -> *
which breaks things like MonadTrans.

changelog_begin
changelog_end
2020-02-12 14:41:14 +00:00
Andreas Herrmann
a44e00937b
Add allocatePartyWithHint(On) to DAML Script (#4489)
* allocatePartyWithHint(On)

CHANGELOG_BEGIN
- [DAML Script - Experimental] The participant argument in ``allocatePartyOn`` is wrapped in ``ParticipantName`` to avoid confusion with the ``displayName`` argument.
- [DAML Script - Experimental] Add ``allocatePartyWithHint`` and ``allocatePartyWithHintOn`` that allow to specify the ``partyIdHint`` to the backing participant. See https://github.com/digital-asset/daml/issues/4472.
CHANGELOG_END

* test-cases for allocatePartyWithHint(On)

* DAML formatting

* Supply "" party id hint instead of None

Addressing review comment
https://github.com/digital-asset/daml/pull/4489#discussion_r378245989

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-02-12 14:17:32 +00:00
Moritz Kiefer
8b512d9212
Handle multiple constraints on an instance in data-dependencies (#4490)
* Handle multiple constraints on an instance in data-dependencies

Previously we translated `instance (Foo a, Foo b) => Foo (a, b) where`
into `instance Foo a => Foo b => Foo (a, b)` which is a syntax
error. We already did this correctly for `HasField` so this PR mostly
just shuffles things around to always use the non-HasField specific
parts.

changelog_begin
changelog_end

* Prefix fields of DFunHead wtih dfh
2020-02-12 13:54:43 +00:00
Moritz Kiefer
1721e12f64
Wrap function types in parentheses for data-dependencies (#4488)
This fixes a bug where `data X = X (a -> b)` was reconstructed as
`data X = X a -> b` which is a syntax error.

changelog_begin
changelog_end
2020-02-12 14:02:02 +01:00
Moritz Kiefer
56db27e50e
Use the new token format in daml-helper (#4486)
We were still using the old token format without the
https://daml.com/ledger-api key. This resulted in a warning on every
call to `daml start` which looks rather ugly. I’ve changed it to
construct the token using the JWT library instead of hardcoding it
which should make it easier to modify in the future

changelog_begin
changelog_end
2020-02-12 12:53:42 +01:00
Martin Huschenbett
b59a871dbd
@daml/ledger: Add Ledger.streamFetchByKey method (#4487)
The `Ledger.streamFetchByKey` method exposes the `/v1/stream/fetch`
endpoint of the JSON API.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-12 11:09:48 +00:00
Martin Huschenbett
9602145e8b
@daml/ledger: Add Ledger.streamQuery method (#4478)
The `Ledger.streamQuery` method exposes the `/v1/stream/query` endpoint
of the JSON API.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-12 11:40:14 +01:00
Moritz Kiefer
e7b8cdba89
Require explicit time mode for DAML script and DAML trigger (#4484)
This should provide a better migration path for people that still rely
on static time by forcing them to make this explicit. Given that both
DAML script and DAML triggers are still experimental, I’m not marking
this as a breaking change

changelog_begin

- [DAML Script - Experimental] The time mode must now always be
  specified explicitly. Use ``--static-time`` to recover the previous
  default time mode.

- [DAML Triggers - Experimental] The time mode must now always be
  specified explicitly. Use ``--static-time`` to recover the previous
  default time mode.

changelog_end
2020-02-12 11:16:25 +01:00
Moritz Kiefer
724367017d
Fix sandbox error message when both -w and -s are provided (#4482)
changelog_begin
changelog_end
2020-02-12 09:55:52 +00:00
Moritz Kiefer
4e3cad411b
Extend LSP multi-package tests (#4481)
This adds a test for running the LSP server directly in the project
directory to make sure that we do not need to build first in that case
and that no --package flags are necessary.

changelog_begin
changelog_end
2020-02-12 10:21:01 +01:00
fabiotudone-da
6e1fd55e45
Publish com.daml.ledger:participant-state-kvutils-app (#4480)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-12 09:45:52 +01:00
Moritz Kiefer
36e188cac8
Translate unsupported kinds to a special Erased type (#4474)
* Translate unsupported kinds to a special Erased type

This should simplify `data-dependencies` and avoid issues like #4470
since we can match on the type instead of having to guess which types
can and which cannot be translated back to DAML.

changelog_begin
changelog_end
2020-02-12 09:03:19 +01:00
Shayne Fletcher
a334e210c7
Try to get package name from manifest (if not provided) (#4476)
Also, mark `build-and-lint` "exclusive".

changelog_begin
changelog_end
2020-02-11 16:02:18 -05:00
Moritz Kiefer
943b43066f
Switch to HashSet/HashMap for NormalizedUri/NormalizedFilePath (#4468)
* Switch to HashSet/HashMap for NormalizedUri/NormalizedFilePath

This matches the change to ghcide in
https://github.com/digital-asset/ghcide/pull/420. Now that we have
optimized Hashable instances it makes sense to use them as much as
possible.

changelog_begin
changelog_end

* debug windows crap

* Fix a bug in vr scenario notifications

* Revert "debug windows crap"

This reverts commit f58fdb92c1.

* attempt to fix windows

who are we kidding, this is not going to work
2020-02-11 15:02:11 +00:00
Moritz Kiefer
730da9e332
Handle references to Any in data-dependencies (#4469)
changelog_begin
changelog_end
2020-02-11 14:54:03 +01:00
Gary Verhaegen
0a251b3fa5
switch CI nodes to permanent (#4455)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-11 02:07:42 +01:00
Stephen Compall
d9deb950ef
expand documentation on WebSockets subprotocols to a more-visible section (#4463)
* expand documentation on WebSockets subprotocols to a more-visible section

CHANGELOG_BEGIN
CHANGELOG_END

* combine reference in websocket section to auth section

* resection

* subprotocols sometimes called protocols

- suggested by @hurryabit; thanks
2020-02-10 20:05:03 +00:00
Leonid Shlyapnikov
53b95ecb20
Expose total functions behind the endpoint mappings for DABL to re-use (#4462)
* Extract partial functions for DABL to re-use

CHANGELOG_BEGIN
CHANGELOG_END

* Actually DABL needs total functions,

they will map them to different endpoints
2020-02-10 15:00:03 -05:00
Gerolf Seitz
85db1033d7
Fix memory leak when using the reset service (#4447)
* sandbox: Don't hold on to old resources when resetting.

Now there's one hell of a memory leak.

CHANGELOG_BEGIN
- [Sandbox] Fixed a memory leak when using the ResetService; not
  everything was cleaned up correctly.
CHANGELOG_END

* sandbox: Split out SandboxClientResource from SandboxServerResource.

Gonna replace SandboxServerResource with a ResourceOwner acquisition.

* sandbox: Don't capture the API server in the SandboxServer resource.

When we reset, this is stored forever, leading to a memory leak.

Tested by rewriting the SandboxServerResource to use
`SandboxServer.owner`.

* sandbox: Revert the test client resource to calling `shutdownNow()`.

* sandbox: Make sure the fixture is recreated properly on each test run.

* sandbox: Make `SandboxState` a non-case class.

The `toString()` was unnecessarily heavy.

* sandbox: Futures, futures everywhere.

Avoid a race condition where the server is stopped before it starts by
storing a `Future[SandboxState]` rather than the `SandboxState` itself.

This doesn't trigger the same memory leak as storing a
`Resource[SandboxState]` because we don't capture the object itself in
the `flatMap` in the same way with `Future`.

* sandbox: Remove an unused parameter left in for debugging.

* sandbox: Replace `@VisibleForTesting` with a comment.

* sandbox: Add more comments to the weird logic in SandboxServer.

* sandbox: Get rid of the `Port` type alias; it was confusing.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-10 16:47:39 +00:00
Miklos
c7183979fd
Added submission validator (#4437)
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-10 17:07:16 +01:00
Remy
e02e5d61bc
Engine: fork method to resolve contractIds (#4411)
* engine: fork method to resolve relative contractId

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-10 16:52:06 +01:00
Jussi Mäki
19063a7c4e
Test tool flags: participant-shuffle and no-wait-for-parties (#4450)
* Shuffle participants used in tests

* Add no-wait-for-parties flag to the test tool

This flag is needed for privacy-preserving ledgers that do not expose
party allocations to non-submitting participants.

* Add shuffle-participants flag to test-tool

CHANGELOG_BEGIN
CHANGELOG_END

* Address code review. Make --no-wait-for-parties hidden.
2020-02-10 13:10:26 +00:00
Gary Verhaegen
77c8c7ce77
add redirect for getting started guide (#4451)
We've created a cool video that happens to have the wrong link in it
(missin l in the extension), and changing videos is more expensive than
changing text files.

Also, the current URL is a tad long, so I've added a shorter one.

Note: as this is part of the docs bundle (actually generating an HTML
redirect page), this won't be live until next version is published.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-10 13:30:27 +01:00
Moritz Kiefer
1dc40ca239
Support multiple-packages in damlc ide (#4445)
* Support multiple-packages in `damlc ide`

changelog_begin

- [DAML Studio] You can now open DAML Studio in the root of a
  multi-package project instead of opening it separately for each
  package. Take a look at the documentation for details on how to set
  this up.

changelog_end

There are a few caveats here:

1. You need a ``daml.yaml`` in the root of your project directory. I
think this is somewhat sensible but we should add a warning to VSCode
if you open it in a directory that does not have a ``daml.yaml`` (in a
separate PR).

2. Changes are not picked up accross dependencies. This is a larger
undertaking and given the current setup simply impossible (we don’t
know that the source files of one package belong to the DAR referenced
in the ``dependencies`` field of the other package. We can make this a
bit better by at least detecting that the ``.dar`` has changed but
let’s do that separately.

3. Since ``daml init`` runs once on startup, it will run in the root
directory instead of initializing the package db of the individual
packages. This is fixable but will conflict with #4391 so let’s
address this separately.

I’ve added docs to the daml studio section that explain the caveats.

* Use the proper sdk version in lsp-tests
2020-02-10 12:20:56 +00:00
Moritz Kiefer
095cf9732a
Remove dependency on ghcide’s fakeDynFlags (#4456)
`fakeDynFlags` has been removed from ghcide. We still need it since we
set up our environment in weird ways and want it to be platform
independent so this PR just copies the definition into `daml`.

changelog_begin
changelog_end
2020-02-10 12:30:37 +01:00
Rohan Jacob-Rao
cdcd4002f6
Fix laoding typo (#4454)
Not sure why the typechecker didn't catch this one.
I'm assuming this was changing an unused `laoding` field and the real
`loading` field was never used (or inaccurate) for `fetchByKeyResults`.

changelog_begin
changelog_end
2020-02-08 14:12:21 +01:00
Leonid Shlyapnikov
a194ce2a34
Wrap streaming API events in JSON object (#4453)
* Wrap streaming API events in json object

CHANGELOG_BEGIN

[JSON API - Experimental] wrap Streaming API events in JSON object:
   ``{ "events": [ E1, E2, ... ] }``
   See: 4384

CHANGELOG_END

* cleanup
2020-02-07 17:17:42 -05:00
Gary Verhaegen
1681922f90
ci: temp machines for scheduled killing experiment (#4386)
* ci: temp machines for scheduled killing experiment

Based on our discussions last week, I am exploring ways to move us to
permanent machines instead of preemptible ones. This should drastically
reduce the number of "cancelled" jobs.

The end goal is to have:

1. An instance group (per OS) that defines the actual CI nodes; this
would be pretty much the same as the existing ones, but with
`preemptible` set to false.
2. A separate machine that, on a cron (say at 4AM UTC), destroys all the
CI nodes.

The hope is that the group managers, which are set to maintain 10 nodes,
will then recreate the "missing" nodes using their normal starting
procedure.

However, there are a lot of unknowns I would like to explore, and I need
a playground for that. This is where this PR comes in. As it stands, it
creates one "killer" machine and a temporary group manager. I will use
these to experiment with the GCP API in various ways without interfering
with the real CI nodes.

This experimentation will likely require multiple `terraform apply` with
multiple different versions of the associated files, as well as
connecting to the machines and running various commands directly from
them. I will ensure all of that only affects the new machines created as
part of this PR, and therefore believe we do not need to go through a
separate round of approval for each change.

Once I have finished experimenting, I will create a new PR to clean up
the temporary resources created with this one and hopefully set up a
more permanent solution.

CHANGELOG_BEGIN
CHANGELOG_END

* add missing zone for killer instance

* add compute scope to killer

* authorize Terraform to shutdown killer to update it

* change in plans: use a service account instead

* .

* add compute.instances.list permission

* add compute.instances.delete permission

* add cron script

* obligatory round of extra escaping

* fix PATH issue & crontab format

* smaller machine & less frequent reboots
2020-02-07 21:04:03 +01:00
Leonid Shlyapnikov
98ab189e3e
Encode tuples as records in the JSON API docs, add a new test (#4452)
tuples are formatted as records in the response, so this way input format
matches the output

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-07 13:46:36 -05:00
Moritz Kiefer
8590ff8274
Document all arguments of the trigger rule (#4448)
Previously we were missing the time.

changelog_begin
changelog_end
2020-02-07 18:20:04 +01:00