Commit Graph

3410 Commits

Author SHA1 Message Date
Gary Verhaegen
dd1b44eccc
add grep to dev-env (#4903)
Because it's not in there and we're all surprised to learn that.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-10 10:34:01 +01:00
Gerolf Seitz
72cafaef70
Update release testing to include running daml script (#4915)
Also the line numbers are updated.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-10 08:49:47 +01:00
Gary Verhaegen
390bcf2ae3
retry release (#4914)
Retrying #4902 with #4912 and #4913.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-09 20:46:22 +01:00
Gary Verhaegen
8f6be82660
retry releases (#4913)
This PR changes the release version format for snapshot releases to
allow for an optional "build number", i.e. "how many times have we
screwed up this release before this attempt?".

Adding this to the version string should be fine because:

- It is a number, so GHC will be happy.
- It is dot-separated and (manually) incrementing, so all three formats
  will sort it correctly.
- The SemVer->GHC conversion only looks at the beginning and ending of
  the snapshot string, and this changes the middle.

This is necessary because sometimes we screw up releases (e.g. #4902),
but also because sometimes releases screw themselves up by somehow
corrupting the Windows cache, and so far changing the version number has
been the only way out of that. So far that has meant changing the target
commit, but that's a very poor reason to choose a target commit.

We should not have to include additional code in a release just because
our release process is flaky.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-09 19:43:29 +01:00
Gary Verhaegen
41643315ac
hopefully fix Azure pipelines github tag release (#4912)
Maybe. If I'm reading
[that](https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/github-release?view=azure-devops)
right.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-09 19:39:08 +01:00
Gerolf Seitz
72d5d6e02e
Set LATEST version for snapshot release. (#4902)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-09 17:16:52 +01:00
Moritz Kiefer
58c101c3ab
Mark lsp tests as flaky on Windows (#4906)
:sadpanda:

changelog_begin
changelog_end
2020-03-09 16:53:36 +01:00
Samir Talwar
51f596daa7
Sandbox Next: Implicit party allocation. (#4894)
* ledger-api-test-tool: Fix warnings flagged by IntelliJ IDEA.

* ledger-api-test-tool: Open-world mode.

In open-world mode, parties aren't allocated; their names are just
reserved for the test case, so that no other test will accidentally use
the same party name.

This is so we can test ledgers which dynamically allocate parties, such
as Sandbox.

* sandbox: Run conformance tests in "open-world" mode.

This means that the tests don't explicitly allocate parties (except for
a few), instead relying on Sandbox's implicit party allocation feature.

This is not enabled for Sandbox Next yet.

* sandbox-next: Implicit party allocation.

This is added to the command submission service.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox-next: Don't implicitly allocate pre-existing parties.

* ledger-api-test-tool: Move pre-allocation into ParticipantTestContext.

* ledger-api-test-tool: We can reserve parties or wait for them. Not both.

Make illegal states unrepresentable as early as possible.

* sandbox: Name ApiSubmissionService's private methods a little better.

* sandbox: Move ApiSubmissionService's conditional logic into methods.

* sandbox: Document why we set `implicitPartyAllocation` to `false`.

* sandbox: Document why `implicitPartyAllocation` is dangerous.
2020-03-09 15:49:11 +00:00
Moritz Kiefer
7b1f193012
Error out on dependencies on newer LF versions (#4901)
This doesn’t really make sense since the main point of targetting an older
LF version is because your server does not support the newer LF
version but including dependencies in newer LF versions makes that
completely useless. In the current state, this also produces a bunch
of errors that look very confusing and while we might be able to fix
them, I don’t think it’s worth doing.

changelog_begin
changelog_end

fixes #4596
2020-03-09 14:59:34 +00:00
Robert Autenrieth
6f8c3ad832
Add command deduplication docs (#4693)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-09 15:33:45 +01:00
associahedron
e67fef1d27
Add extra test on record constructor check. (#4898)
Based on @hurryabit's comments from last PR.

changelog_begin
changelog_end
2020-03-09 14:31:47 +00:00
Stefano Baghino
1908de10e4
Rework ValueSerializer (#4897)
* Rework ValueSerializer

- Handle errors directly in the convenience method (it's the only way in which it's used)
- Don't drop the root cause when a serialization error occurs
- Remove outdated comment

CHANGELOG_BEGIN
CHANGELOG_END

* Add alternative with error context for deserialize method

* Make error context evaluated lazily

* Rename inner helper to avoid name clash
2020-03-09 15:17:06 +01:00
Shayne Fletcher
d2c819b4cd
Daml2ts config constants (#4887)
* Config constants

changelog_begin
changelog_end

* Lift constants to top of file for easy maintenance

changelog_begin
changelog_end
2020-03-09 09:36:04 -04:00
Samir Talwar
9ec45eb12d
sandbox: Error when a party already exists. (#4896)
CHANGELOG_BEGIN
- [Sandbox] Respond with an error when trying to create a party that
  already exists.
CHANGELOG_END
2020-03-09 13:18:04 +00:00
Samir Talwar
3e7d016875
sandbox: Normalize language around listing all known parties. (#4895)
This renames methods backing the `ListKnownParties` request to
from `parties`, `getParties` or `listParties` to `listKnownParties`.

CHANGELOG_BEGIN
- [Ledger API Server] Renamed two metrics:
  ``daml.index.parties`` was renamed to ``daml.index.list_known_parties``
  ``daml.index.db.get_parties`` was renamed to ``daml.index.db.list_known_parties``
CHANGELOG_END
2020-03-09 13:15:33 +00:00
Remy
047495b2af
[DAML-LF] define an ordering on values (#4808)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-09 13:51:22 +01:00
Samir Talwar
0f4fb9a8f5
Ledger API Server: Add the GetParties endpoint. (#4888)
* sandbox: Add a database test for storing and retrieving parties.

* sandbox: Add database queries for selecting one or many parties.

* ledger-api-test-tool: Add a test for `ListKnownParties`.

* sandbox: Add an endpoint to retrieve a single party's details.

CHANGELOG_BEGIN
- [Ledger API] Added an endpoint to retrieve a single party's details at
  ``com.digitalasset.ledger.api.v1.admin.PartyManagementService.GetParty``.
  Please consult the ledger API reference documentation for more
  information.
CHANGELOG_END

* sandbox: Add an endpoint to retrieve a multiple parties' details.

CHANGELOG_BEGIN
- [Ledger API] Added an endpoint to retrieve multiple parties's details at
  ``com.digitalasset.ledger.api.v1.admin.PartyManagementService.GetParties``.
  Please consult the ledger API reference documentation for more
  information.
CHANGELOG_END

* sandbox: Getting a single party is a special case of multiple parties.

So let's use that code path and stop duplicating work.

* sandbox: Remove `GetParty`, as it's subsumed by `GetParties`.

"Subsumed" is a great word.
2020-03-09 12:09:56 +00:00
Gary Verhaegen
8e217d219e
amend release process notes (#4871)
Based on running through the new process twice now, I wanted to try and
clarify some of the steps.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-09 12:50:18 +01:00
Gerolf Seitz
cf74e74d05
Fix calculation of transaction tree projections (#4885)
Events in transaction trees should only reference other events
that:
1) are either create or exercise events
2) the requesting parties are a witness of

This applies to recalculated root nodes as well as
the child event ids referenced in exercise nodes.

CHANGELOG_BEGIN
[Sandbox]: fixed projection of transaction trees.
CHANGELOG_END
2020-03-09 12:37:14 +01:00
associahedron
943832bc16
Add a warning for data X = Y {...}. (#4892)
* Add a warning for data X = Y {..}.

Part of #4718.

changelog_begin

- [DAML Compiler] The DAML compiler now emits a warning when you declare a single-constructor record type where the constructor name does not match the type name, such as ``data X = Y {...}``. This kind of type declaration can cause problems with cross-SDK upgrades because the record constructor name cannot be recorded in the DAML-LF representation. In the future, this warning will be upgraded to an error.

changelog_end

* s/Ctor/Constructor/g
2020-03-09 11:28:16 +00:00
Robin Krom
0e046d9eca
docs: new recommend path architecture documentation (#4795)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-09 12:15:37 +01:00
Stefano Baghino
e3c9f363bd
Avoid some unnecessary work when encoding a transaction (#4890)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-09 11:02:25 +00:00
Moritz Kiefer
d4f96d9d16
Move upgrading docs out of experimental section (#4889)
I’ve added a note that you have to target LF 1.8 for this to work.

fixes #4767

changelog_begin
changelog_end
2020-03-09 10:38:58 +01:00
Gerolf Seitz
2b76d1962c
Deprecate ledger initialization with scenarios (#4864)
* Deprecate ledger initialization with scenarios

CHANGELOG_BEGIN
[Sandbox] Initializing the sandbox with scenarios is now deprecated in
favor of using DAML Script. The scenario parameter will be removed in
the near future. A warning is logged on startup.

The DAML SDK templates and quickstart guide are using DAML Script.

See the DAML Script migration guide for more information:
https://docs.daml.com/daml-script/index.html#using-daml-script-for-ledger-initialization
CHANGELOG_END
2020-03-09 09:30:18 +01:00
Moritz Kiefer
1feffdbdb1
Improve error messages when we cannot find packages in the script runner (#4884)
The logic for detecting these needs to be improved but for now this at
least gives a useful error message instead of some internal stacktrace.

changelog_begin
changelog_end
2020-03-06 18:42:57 +01:00
Moritz Kiefer
d381603475
Detect whether sandbox has started using --port-file (#4880)
On MacOS using Java 13 (at least that was the only combination where
I’ve managed to reproduce this). Sandbox spits out errors of the
following form in ``daml start``

```
java.net.SocketException: Connection reset
```

This appears to be caused by the TCP requests that we send to detect
whether sandbox has started. Switching to using the ``--port-file``
option makes this issue go away.

fixes #4670

changelog_begin

- [DAML Assistant] In certain configurations ``daml start`` produced
  a (harmless) ``Connection reset`` log message. This has now been
  fixed.

changelog_end
2020-03-06 16:18:43 +00:00
associahedron
02ae2a29d9
data-dependencies: Call buildHiddenRefMap once per package. (#4881)
* Call buildHiddenRefMap once per pkg.

Instead of calling it once per module.

changelog_begin
changelog_end

* Remove special cases
2020-03-06 16:17:57 +00:00
Samir Talwar
a0f250a727
sandbox: Avoid converting to byte arrays before deserializing. (#4865)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-06 15:40:26 +00:00
Samir Talwar
c6954c086b
sandbox-next: Error if a scenario is provided. (#4875)
* sandbox-next: Pull runner configuration into the constructor.

No need to do it on `acquire()` if it's pure.

* sandbox-next: Error if a scenario is provided.

Sandbox-Next doesn't support scenarios, instead favoring DAML Script.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-06 14:51:42 +00:00
Samir Talwar
b4a529eda9
sandbox: Format Cli.scala. (#4876)
Not sure why formatting was turned off at one point, but it seems to be
OK now.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-06 14:43:04 +00:00
Samir Talwar
82c40f2ecb
kvutils: Throw meaningful errors when dealing with a Left. (#4835)
* kvutils: On error opening an envelope, throw the correct message.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-sql: On error when querying state, throw the correct error.

* kvutils|ledger-on-sql: Remove unnecessary curly braces around `throw`.
2020-03-06 13:53:34 +00:00
Moritz Kiefer
d8afe7a4cc
Support specifying sandbox/navigator/json-api/script options in daml.yaml (#4877)
changelog_begin

- [DAML Assistant] You can now specify options for
  Sandbox/Navigator/the HTTP JSON API in ``daml.yaml`` via
  ``sandbox-options``/``navigator-options``/``json-api-options``. These
  options will be picked up by running ``daml start``. This is
  particularly useful for specifying ``--wall-clock-time`` and for
  specifying a fixed ledger ID during development.

changelog_end

fixes #2993
2020-03-06 14:52:45 +01:00
Moritz Kiefer
c7eb3f6b6c
Wrap Script in StateT to make evaluation order a bit less important (#4869)
* Wrap Script in StateT to make evaluation order a bit less important

This PR wraps the Script newtype in `StateT` which means that
evaluation won’t do much so `debug` behaves a bit more sensibly and
you don’t end up evaluating a script that only consists of `pure` and
`>>=` if you do not execute it.

fixes #4821

changelog_begin

- [DAML Script] Fix an issue where ``debug`` messages were output
  before the script was executed.

changelog_end

* Inline StateT and improve error messages
2020-03-06 13:42:11 +01:00
Shayne Fletcher
60fa929336
Eliminate duplication of package name calculation (#4874)
changelog_begin
changelog_end
2020-03-06 07:24:50 -05:00
Moritz Kiefer
ad55159989
Address review comments on script migration docs (#4870)
changelog_begin
changelog_end
2020-03-06 12:39:10 +01:00
Gary Verhaegen
2eedd291ec
bind repl to localhost (#4863)
Currently the repl server is bound to 0.0.0.0, which is not great for
security and makes running the tests a bit disruptive on macOS.

This binds it to 127.0.0.1 instead.

CHANGELOG_BEGIN
- [DAML Repl - Experimental] The REPL server will now bind to 127.0.0.1
  instead of 0.0.0.0.
CHANGELOG_END
2020-03-06 11:40:55 +01:00
Moritz Kiefer
52bf9b2a5c
Fix git tag of releases (#4862)
Previously, we tagged the commit that made the release instead of the
commit we are building the release off.

changelog_begin
changelog_end
2020-03-06 10:54:07 +01:00
Shayne Fletcher
9858a38d6b
version -> sdkVersionOrErr (#4861)
changelog_begin
changelog_end
2020-03-06 10:33:52 +01:00
fabiotudone-da
9635e14d56
Restore LedgerFactory inheritors' ability to use Config for setups (#4860)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-06 08:25:27 +00:00
Shayne Fletcher
7fe94be065
Drop special casing 'daml-types' (#4859)
changelog_begin
changelog_end
2020-03-05 18:02:49 -05:00
Shayne Fletcher
5171ed5313
Shift comment and metadata write calls around (#4858)
* Shift comment and metadata write calls around

Also remove licence line from package.json files

changelog_begin
changelog_end

* Broken Windows cache. Unbreak it. NOT FOR MERGING TO MASTER.

* Restore Windows cache; safe now to merge.
2020-03-05 17:47:23 -05:00
Moritz Kiefer
fa968699c8
Unexperimentalize DAML script (#4856)
The only remaining thing that I wanted to get in before doing that was
overloading ``submit`` which has now happened. I’m sure we can come up
with tons of improvements but I don’t expect breaking changes to the
current state.

changelog_begin

- [DAML Script] DAML Script is no longer experimental.

changelog_end

fixes #4615
2020-03-05 20:40:15 +01:00
Shayne Fletcher
e1a164ef8b
Introduce a parameter pack for function 'daml2ts' (#4852)
changelog_begin
changelog_end
2020-03-05 13:54:39 -05:00
mergify[bot]
6b851229b8
Refactor extraction of events from transaction (#4781)
* Refactor extraction of events from transaction

Closes #1909

CHANGELOG_BEGIN
CHANGELOG_END

* Remove unnecessary filtering

* Fix disclosure rule for flat transaction

* Refactor and split collection and filtering

* Replace transaction filtration with blinding info

* Move transient contract remover in transaction conversion

* Remove dangling file

* Simplify transient contract filtering

* Further refinements

* Simplify transaction tree event extraction

* Move newRoots up the file for consistency and readability

* Remove collect from GenTransaction, replace with custom iterator

* Address https://github.com/digital-asset/daml/pull/4781#discussion_r388167562

* Switch to a strict collect method

* Replaced direct access to map with contains
2020-03-05 18:06:05 +00:00
mergify[bot]
1007fd58e6
Track erased types in data-dependencies. (#4833)
* Track erased types in data-dependencies.

This PR introduces a tracker for Erased types, DA.Generics, and any type, typeclass, or typeclass instance that dependens on them transitively. This is designed to detect and cut out any such definition that will cause problems.

Apologies for the large PR. Originally this was just a small extension to the tracking of old-style typeclasses, but each bugfix uncovered a new bug, and the only way to get it to completely work was to do it all at once. That's why the tracker is so extensive -- the only thing it doesn't track is regular functions, because they will never introduce a cyclic reference to an erased type that gets exposed in the type system (until we implement dependent DAML).

Right now the tracker runs once per data-dependency module, but in a future PR I will make it run once per data-dependency package to reduce the repetitive. This was all tested on a very large DAR, and it runs fine, but it should be much faster once it runs once per package.

changelog_begin
changelog_end

* lint

* Address comments

* Add a great test case
2020-03-05 17:54:55 +00:00
mergify[bot]
c1d601586a
ledger-on-sql: Ensure log entries are returned in the correct order. (#4854)
I have no idea how this worked previously. I think we got lucky.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 17:18:08 +00:00
mergify[bot]
c6d9000369
Added --damlc option to daml-sdk-head (#4853)
changelog_begin
changelog_end
2020-03-05 16:38:56 +00:00
mergify[bot]
60013a1535
Remove DA.Upgrade module (#4839)
Given that Generic instances are not supported cross-SDK this module
only causes confusion and I’d rather remove it.

changelog_begin
changelog_end
2020-03-05 16:30:54 +00:00
Gary Verhaegen
950d8c3501
remove perf tests from CI (#4851)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 17:29:28 +01:00
Gary Verhaegen
75c7d48d5b
bind localhost for json-api and scenarios (#4576)
Both were previously binding 0.0.0.0, which is inherently insecure. More
importantly to me, that meant running `bazel test //...` essentially
rendered my computer unusable for however long it took (which is
_long_), as it kept popping up focus-stealing dialogs about whether or
not I wanted to trust "java" to open an incoming network connection.

The ScenarioService does not seem to have an existing setup for CLI args
and my Scala-fu is not good enough to add one, so I just changed the
hard-coded path.

The JSON API already had an option, just with the wrong default. This is
technically a breaking change, but I'm hoping to pass it under the
"experimental" flag we still have on the JSON API.

CHANGELOG_BEGIN
- [JSON API - Experimental] As a security improvement, the JSON API
server will now bind on ``127.0.0.1`` by default. Previous behaviour was
to bind on ``0.0.0.0``; you can get that behaviour back by passing in
the (existing) flag ``--address 0.0.0.0``.

- [DAML SDK] The Scenario Service will now bind on ``127.0.0.1``. Previous
behaviour was to bind on ``0.0.0.0``.

CHANGELOG_END
2020-03-05 17:10:49 +01:00