Commit Graph

3526 Commits

Author SHA1 Message Date
Moritz Kiefer
ad55159989
Address review comments on script migration docs (#4870)
changelog_begin
changelog_end
2020-03-06 12:39:10 +01:00
Gary Verhaegen
2eedd291ec
bind repl to localhost (#4863)
Currently the repl server is bound to 0.0.0.0, which is not great for
security and makes running the tests a bit disruptive on macOS.

This binds it to 127.0.0.1 instead.

CHANGELOG_BEGIN
- [DAML Repl - Experimental] The REPL server will now bind to 127.0.0.1
  instead of 0.0.0.0.
CHANGELOG_END
2020-03-06 11:40:55 +01:00
Moritz Kiefer
52bf9b2a5c
Fix git tag of releases (#4862)
Previously, we tagged the commit that made the release instead of the
commit we are building the release off.

changelog_begin
changelog_end
2020-03-06 10:54:07 +01:00
Shayne Fletcher
9858a38d6b
version -> sdkVersionOrErr (#4861)
changelog_begin
changelog_end
2020-03-06 10:33:52 +01:00
fabiotudone-da
9635e14d56
Restore LedgerFactory inheritors' ability to use Config for setups (#4860)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-06 08:25:27 +00:00
Shayne Fletcher
7fe94be065
Drop special casing 'daml-types' (#4859)
changelog_begin
changelog_end
2020-03-05 18:02:49 -05:00
Shayne Fletcher
5171ed5313
Shift comment and metadata write calls around (#4858)
* Shift comment and metadata write calls around

Also remove licence line from package.json files

changelog_begin
changelog_end

* Broken Windows cache. Unbreak it. NOT FOR MERGING TO MASTER.

* Restore Windows cache; safe now to merge.
2020-03-05 17:47:23 -05:00
Moritz Kiefer
fa968699c8
Unexperimentalize DAML script (#4856)
The only remaining thing that I wanted to get in before doing that was
overloading ``submit`` which has now happened. I’m sure we can come up
with tons of improvements but I don’t expect breaking changes to the
current state.

changelog_begin

- [DAML Script] DAML Script is no longer experimental.

changelog_end

fixes #4615
2020-03-05 20:40:15 +01:00
Shayne Fletcher
e1a164ef8b
Introduce a parameter pack for function 'daml2ts' (#4852)
changelog_begin
changelog_end
2020-03-05 13:54:39 -05:00
mergify[bot]
6b851229b8
Refactor extraction of events from transaction (#4781)
* Refactor extraction of events from transaction

Closes #1909

CHANGELOG_BEGIN
CHANGELOG_END

* Remove unnecessary filtering

* Fix disclosure rule for flat transaction

* Refactor and split collection and filtering

* Replace transaction filtration with blinding info

* Move transient contract remover in transaction conversion

* Remove dangling file

* Simplify transient contract filtering

* Further refinements

* Simplify transaction tree event extraction

* Move newRoots up the file for consistency and readability

* Remove collect from GenTransaction, replace with custom iterator

* Address https://github.com/digital-asset/daml/pull/4781#discussion_r388167562

* Switch to a strict collect method

* Replaced direct access to map with contains
2020-03-05 18:06:05 +00:00
mergify[bot]
1007fd58e6
Track erased types in data-dependencies. (#4833)
* Track erased types in data-dependencies.

This PR introduces a tracker for Erased types, DA.Generics, and any type, typeclass, or typeclass instance that dependens on them transitively. This is designed to detect and cut out any such definition that will cause problems.

Apologies for the large PR. Originally this was just a small extension to the tracking of old-style typeclasses, but each bugfix uncovered a new bug, and the only way to get it to completely work was to do it all at once. That's why the tracker is so extensive -- the only thing it doesn't track is regular functions, because they will never introduce a cyclic reference to an erased type that gets exposed in the type system (until we implement dependent DAML).

Right now the tracker runs once per data-dependency module, but in a future PR I will make it run once per data-dependency package to reduce the repetitive. This was all tested on a very large DAR, and it runs fine, but it should be much faster once it runs once per package.

changelog_begin
changelog_end

* lint

* Address comments

* Add a great test case
2020-03-05 17:54:55 +00:00
mergify[bot]
c1d601586a
ledger-on-sql: Ensure log entries are returned in the correct order. (#4854)
I have no idea how this worked previously. I think we got lucky.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 17:18:08 +00:00
mergify[bot]
c6d9000369
Added --damlc option to daml-sdk-head (#4853)
changelog_begin
changelog_end
2020-03-05 16:38:56 +00:00
mergify[bot]
60013a1535
Remove DA.Upgrade module (#4839)
Given that Generic instances are not supported cross-SDK this module
only causes confusion and I’d rather remove it.

changelog_begin
changelog_end
2020-03-05 16:30:54 +00:00
Gary Verhaegen
950d8c3501
remove perf tests from CI (#4851)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 17:29:28 +01:00
Gary Verhaegen
75c7d48d5b
bind localhost for json-api and scenarios (#4576)
Both were previously binding 0.0.0.0, which is inherently insecure. More
importantly to me, that meant running `bazel test //...` essentially
rendered my computer unusable for however long it took (which is
_long_), as it kept popping up focus-stealing dialogs about whether or
not I wanted to trust "java" to open an incoming network connection.

The ScenarioService does not seem to have an existing setup for CLI args
and my Scala-fu is not good enough to add one, so I just changed the
hard-coded path.

The JSON API already had an option, just with the wrong default. This is
technically a breaking change, but I'm hoping to pass it under the
"experimental" flag we still have on the JSON API.

CHANGELOG_BEGIN
- [JSON API - Experimental] As a security improvement, the JSON API
server will now bind on ``127.0.0.1`` by default. Previous behaviour was
to bind on ``0.0.0.0``; you can get that behaviour back by passing in
the (existing) flag ``--address 0.0.0.0``.

- [DAML SDK] The Scenario Service will now bind on ``127.0.0.1``. Previous
behaviour was to bind on ``0.0.0.0``.

CHANGELOG_END
2020-03-05 17:10:49 +01:00
mergify[bot]
6defabe39e
Document migration from scenarios to DAML script (#4834)
* Document migration from scenarios to DAML script

changelog_begin
changelog_end

* Add a more general section on ledger initialization

* fix tests
2020-03-05 16:09:21 +00:00
Shayne Fletcher
ed75b64fef
'writeRootPackageJson' -> 'setupWorkspace' (#4847)
Rename this function in empathy with
https://github.com/digital-asset/daml/pull/4829 and compute dependency
pairs not triples as per https://github.com/digital-asset/daml/pull/4799#discussion_r387883681

changelog_begin
changelog_end
2020-03-05 11:00:53 -05:00
Andreas Herrmann
7a2bf268fe
ld -s --> ld -unexported_symbols_list=* on macOS (#4838)
On MacOS the `-s` linker flag (`--strip-all`) is obsolete. Instead, one
should pass `-unexported_symbols_list=*` to strip all symbols.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-03-05 16:56:01 +01:00
mergify[bot]
86362ee0bf
fix typo in daml-script docs (#4840)
Redundant.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 15:12:53 +00:00
mergify[bot]
35ee0b3057
daml2ts: Always use SDK version for @daml/types (#4829)
Currently, the version of `@daml/types` used by the generated TS is
configurable. This is not very helpful since `daml2ts` and `@daml/types`
are developed in lock step and still moving quite fast.

This PR changes `daml2ts` to always use the SDK version for as the
version of `@daml/types`. This also requires us to fix the test
for DAVL which used a hard coded version of `@daml/types` so far.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 15:12:06 +00:00
mergify[bot]
e785bb7402
Sandbox: Re-use actor systems between the StandaloneApiServer and StandaloneIndexerServer. (#4827)
* sandbox: Re-use the root actor system in the StandaloneIndexerServer.

* kvutils/app: Don't use the ActorSystem execution context randomly.

Instead, make `Runner` a proper ResourceOwner, with an `acquire` method.

* sandbox: Re-use the root actor system in the StandaloneApiServer.

CHANGELOG_BEGIN
CHANGELOG_END

* resources: Remove the now-unused `ResourceOwner.sequence` functions.

They weren't well-thought-out anyway; they acquire resources
sequentially, rather than in parallel.
2020-03-05 14:50:18 +00:00
Moritz Kiefer
0a6be2b341
Overload submit so that it doesn’t collide with DAML script (#4831)
This introduces a `HasSubmit` typeclass (following the naming scheme
of `HasCreate`, …) and instances for `Scenario` and `Script`. This
avoids the need to hide `submit` in every single DAML script.

changelog_begin

- [DAML Standard Library] ``submit`` and ``submitMustFail`` are now
  overloaded so that they can be used in both scenarios and DAML script.

changelog_end
2020-03-05 15:43:35 +01:00
Shayne Fletcher
d33b507344
Replace over-enthusiastic uses of 'maybe' with case expressions (#4837)
changelog_begin
changelog_end
2020-03-05 09:43:13 -05:00
mergify[bot]
46a218a22f
sandbox-next: Disable ResetService tests. (#4836)
They're too flaky. We'll bring them back once we work out the
performance issues.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 14:06:02 +00:00
mergify[bot]
1842015a46
daml2ts: Factor out assertFileExists in tests (#4830)
* daml2ts: Factor out assertFileExists in tests

The pattern `assertBool "..." =<< doesFileExist ...` appears so many
times that it definitely deserves its own function.

CHANGELOG_BEGIN
CHANGELOG_END

* Remove assertions for existence of DAR files

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 13:55:30 +00:00
Moritz Kiefer
42dd6349e9
Cache unmangleIdentifier (#4822)
* Cache unmangleIdentifier

Previously we unmangled once per reference to an identifier rather
than doing the unmangling once per entry in the string interning
table. This PR fixes this which brings a pretty decent performance
improvement. On my testcase the time for converting for converting
from the low-level proto AST to the high-level Haskell AST goes down
from 13.5 to 7.5s on a certain DALF that we know very well. Max
residency also goes down from 2GB to 1.5GB (although that number is
somewhat unreliable ime) and allocations drop by 8%.

changelog_begin
changelog_end

* Move error message to unmangleIdentifier
2020-03-05 13:35:28 +01:00
Shayne Fletcher
3b6bd630cc
Rearrange to avoid 'stripPrefix'; cut down on 'T.pack'/'T.unpack' (#4818)
* Rearrange to avoid 'stripPrefix'; cut down on 'T.pack'/'T.unpack'

changelog_begin
changelog_end

* Remove ticks as suggested in https://github.com/digital-asset/daml/pull/4799#discussion_r387886292

* Fix indentation (as per https://github.com/digital-asset/daml/pull/4799#discussion_r387892680)
2020-03-05 07:28:47 -05:00
mergify[bot]
95c1e5edaa
Allow LedgerFactory to provide a full-blown ReadWriterService (#4792)
* Allow `LedgerFactory` to provide a full-blown `ReadWriterService` rather than a `LedgerReaderWriter` (needed by at least vDAML)

CHANGELOG_BEGIN
CHANGELOG_END

* Address review points

* Address Samir's review point in `SqlLedgerFactory`

* Finish addressing Samir's review point in `SqlLedgerFactory`

* Split reader and writer owners in `LedgerFactory` as suggested by Gerolf

* Remove unneeded `val` from `KeyvalueParticipantState[Reader|Writer]` constructor params

* Remove unneeded type parameter from `app.Runner`

* Leave `LedgerFactory` a full ledger builder but split hierarchy upwards and clarify responsibilities

* Rename `SimpleLedgerFactory` to `KeyValueLedgerFactory`
2020-03-05 12:02:23 +00:00
Gary Verhaegen
3410000fc7
change msys2 mirror (#4825)
The `repo.msys2.org` server is currently broken and has been for about 7
hours at least from what I can find out online. This PR changes to the
next mirror in the list that seems to work for me locally; list taken
from [the GitHub
repo](https://github.com/msys2/MSYS2-packages/blob/master/pacman-mirrors/mirrorlist.msys).

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 11:56:32 +01:00
Gary Verhaegen
80652bd51f
report-std-change: handle GitHub errors (#4814)
There have been a few GitHub glitches last week that resulted in a few
commits on master not being associated with a PR (though they really
were created from merging a PR, and the correct PR number is in their
title).

This makes the report script crash on not finding the PR, so this PR
fixes that. And a comment.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 10:06:58 +01:00
mergify[bot]
9bb80e552c
daml2ts: Keep workspace package.json intact (#4820)
Keep all fields in the workspace `package.json` given to `daml2ts`
instead of only `private` and `workspaces`.

This also allows for removing the `sed` hack in the `build-and-lint`
test and use the `resolution` field of the workspace `package.json`
to point to our local versions of `@daml/types` and `@daml/ledger`.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-05 00:45:11 +00:00
Shayne Fletcher
0a0786d596
Set test size to large (#4817)
changelog_begin
changelog_end
2020-03-04 15:59:23 -05:00
mergify[bot]
cb1395e923
Remove damlc migrate (#4816)
* Remove damlc migrate

``damlc migrate`` hasn’t worked for quite a while and we emitted a
warning for months so given that we don’t have plans to make it work
again in the near future, I think it does more harm than good to keep
it around.

changelog_begin

- [DAML Compiler] After being deprecated for a while the ``damlc
  migrate`` command has now been removed. See
  https://docs.daml.com/upgrade/ for up to date documentation
  on model upgrades.

changelog_end

fixes #3704 (by removing the tests 😇)

* yeah the windows cache is once again broken \o/

* Revert "yeah the windows cache is once again broken \o/"

This reverts commit 38d7877aa4.
2020-03-04 20:36:48 +00:00
Leonid Shlyapnikov
f9057bea19
handle dual LiveBegin case (#4815)
* make append of two liveness markers proper with respect to offsets

- never happens in a real stream, but satisfying some random tests
- test randomly harder

CHANGELOG_BEGIN
CHANGELOG_END

* move Monoid and Semigroup instances for Steps to main code

- suggested by @leo-da; thanks

* stray empty InsertDeleteStep
2020-03-04 14:59:26 -05:00
Shayne Fletcher
48595f774d
daml2ts : Generate a package of each DALF (#4799)
* daml2ts : generate one package per dalf
2020-03-04 14:51:40 -05:00
Moritz Kiefer
c866ac5132
Speed up unmangleIdentifier (#4810)
* Speed up unmangleIdentifier

On my (admittedly not super scientific) benchmark, this brings the
time used to convert from the low-level proto Haskell AST to the
high-level AST from 20s down to 16s on a certain DALF that we are all
too familiar with.

changelog_begin
changelog_end

* address review comments
2020-03-04 19:48:03 +01:00
mergify[bot]
a40190d695
minor cleanup, using Option.exists instead of cata. (#4812)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-04 18:10:04 +00:00
mergify[bot]
c001b4f22c
test that ContractStreamStep#append preserves data sensibly (#4809)
* test that ContractStreamStep#append preserves data sensibly

CHANGELOG_BEGIN
CHANGELOG_END

* (C) header

* test more general associativity
2020-03-04 17:59:02 +00:00
Moritz Kiefer
c6e3bf5399
Only unmangle once in decodeValueName (#4813)
The `decodeValueName` code is rather confusing. It calls things
unmangled that are mangled and the other way around.

Furthermore, it unmangles twice, once in `decodeNameString` and once
directly in `decodeValueName`. The code claims that this is a
compatiblity hack but unless someone can explain to me what exactly is
failing here or CI fails, I would prefer to just kill this.

changelog_begin
changelog_end
2020-03-04 17:29:55 +00:00
Gary Verhaegen
c2240df083
change standard change report time (#4804)
In the current setup, the report generation starts at 4. However, the
daily killing of all machines also starts at 4, giving the report little
chance of ever finishing.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-04 18:22:32 +01:00
Moritz Kiefer
a928402050
Make debug a bit more lazy (#4807)
* Make `debug` a bit more lazy

Previously `debug x >>= f` would print the debug statement when it got
evaluated rather than when the monadic execution got executed. This is
rather confusing. Now we explicitly make it lazy by wrapping the trace
statement in a lambda passed to >>=.

changelog_begin

- [DAML Standard Library] Fix a bug where ``debug`` printed the trace
  statement before the action got executed. Note that this means that
  ``debug`` now has a slightly more restrictive type. You can use
  ``trace`` directly if this causes problems.

changelog_end

* Update compiler/damlc/tests/src/DA/Test/ShakeIdeClient.hs

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-03-04 17:12:24 +00:00
Moritz Kiefer
569a8a2192
Address upgrade doc review comments (#4811)
This PR addresses the comments from @hurryabit on #4750.

changelog_begin
changelog_end
2020-03-04 18:09:45 +01:00
nickchapman-da
a64610925c
Extend the documentation for DAML upgrade to shows project setup, build and deploy steps. (#4794)
changelog_begin
changelog_end
2020-03-04 15:23:14 +00:00
Samir Talwar
85998f8f06
Sandbox-Next: Add the ResetService. (#4802)
* sandbox-next: Make the Runner a real ResourceOwner.

* sandbox: Don't construct the ResetService twice.

* sandbox: Inline and simplify methods in StandaloneApiServer.

* resources: Define a `ResettableResource`, which can be `reset()`.

`reset()` releases the resource, performs an optional reset operation,
and then re-acquires it, binding it to the same variable.

* resources: Pass the resource value into the reset operation.

* sandbox: Fix warnings in `TestCommands`.

* sandbox-next: Add the ResetService.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Make sure the SandboxResetService resets asynchronously.

It was being too clever and negating its own asynchronous behavior.

* sandbox-next: Forbid no seeding.

This double negative is really hard to phrase well.

* sandbox-next: Implement ResetService for a persistent ledger.

* sandbox: Delete the comment heading StandaloneIndexerServer.

It's no longer meaningful.

* sandbox-next: No need to wrap the SandboxResetService in an owner.

* sandbox-next: Bump the ResetService test timeouts.

It looks like it's definitely slower than on Sandbox Classic™. Gonna
look into this as part of future work.

* Revert to previous asynchronous reset behavior

Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-03-04 14:40:35 +00:00
nickchapman-da
3239a810eb
Snapshot release (#4806)
changelog_begin
changelog_end
2020-03-04 14:11:13 +00:00
Robert Autenrieth
98095cf80d
Clean up command deduplication (#4801)
* Always return error on duplicate submissions

* Remove unnecessary submission information

Now that duplicate submissions always return an error,
we don't need to store the original submission result.

CHANGELOG_BEGIN
CHANGELOG_END

* Rename ttl to deduplicationTime/deduplicateUntil

* Store absolute deduplicateUntil in domain commands

* Fix my own initials

* Remove CommandDeduplicationEntry

Instead, use CommandDeduplicationResult everywhere,
removing the extra layer.
2020-03-04 14:08:08 +01:00
Gary Verhaegen
6a1c75cfd4
fix Windows signing (maybe) (#4753)
During the latest attempt at making a snapshot release (#4749),
everything seemingly went well except for the step of signing the
Windows installer.

Based on a very obscure erorr message (Access Denied) and a bit of
Google search, my current hypothesis is that the signing fails because
the artifact produced by Bazel is read-only. This was not an issue in
the previous setup because we were getting the installer from an Azure
internal download between different jobs. We are now getting the binary
directly from Bazel.

This also adds a `set -e` to the relevant Bash snippet, because ideally
they should always have one.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-04 13:18:49 +01:00
Moritz Kiefer
351091a76e
Only log message about duplicate package uploads in debug level (#4803)
It is basically impossible to not hit this all the time if you upload
more than one package so issuing a warning is a bit confusing.

changelog_begin

- [Sandbox] The warning about duplicate package uploads is no longer
  emitted by default. You can enable them by passing
  ``--log-level=debug``.

changelog_end
2020-03-04 10:36:20 +00:00
Remy
267671c340
[engine] track usage of getTime (#4790)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-04 10:17:55 +01:00