Commit Graph

481 Commits

Author SHA1 Message Date
Martin Huschenbett
d82d47a275
daml2ts: Write module A.B.C into file src/A/B/C/module.ts (#5209)
Instead of writing the TypeScript for DAML module `A.B.C` into the file
`src/A/B/C.ts` we now write it into `src/A/B/C/module.ts`. This is in
preparation for also writing a file `src/A/B/C/index.ts`, which
re-exports `src/A/B/C/module` but also `src/A/B/C/D` for each `D`.

We also make sure to use the correct path separator on Windows.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-26 16:38:12 +01:00
Shayne Fletcher
24cbeddd35
Follow up to earlier PR (#5201)
changelog_begin
changelog_end
2020-03-26 05:16:16 -04:00
Martin Huschenbett
c9e2f7929b
daml2ts: Import external dependencies via their index module (#5203)
Instead of reaching into to the internals of external dependencies,
we no import these package via their root `index.ts` file. This gives
us the possibility of the changing the internal structure without
having to think about this kind of import as long as we keep the
interface the same. This is exactly what I plan to do in the next PR
which will fix the `A` vs `A.B` bug.

More precisely, instead of writing two imports like
```typescript
import * as pkgXYZ_A_B from '@daml.js/foo-0.0.1/lib/A/B';
import * as pkgXYZ_C from '@daml.js/foo-0.0.1/lib/C';
```
we only write one import
```typescript
import pkgXYZ from '@daml.js/foo-0.0.1';
```
and replace use sites of `pkgXYZ_A_B` and `pkgXYZ_C` with `pkgXYZ.A.B`
and `pkgXYZ.C`, resp.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 22:52:43 +00:00
Martin Huschenbett
95f5d6822b
daml2ts: Emphasize differences between internal and external imports (#5197)
This prepares a followup change in which I want to import external
dependecies entirely through their `index.ts` file. This will become easier
once the differences between internal and external imports are more
apparent.

Importing external dependencies through their `index.ts` files is part
of a restructuring the file layout in order to solve the bug where we
cannot have modules `A` and `A.B` at the same time.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 21:11:15 +00:00
Martin Huschenbett
5460379d37
daml2ts: Write package id directly into index.ts (#5195)
There's no more reason to have the package id in a separate file
`packageId.ts`. This was different before we had an `index.ts`.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 19:24:43 +00:00
Martin Huschenbett
b1913e6d38
TS libs: Remove deprecated methods/hooks (#5168)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-25 20:05:27 +01:00
Shayne Fletcher
9351b7a7b9
Build js packages at code-generation time (#5171)
changelog_begin
changelog_end
2020-03-25 14:13:37 -04:00
Robert Autenrieth
a3ddde3a9d
Remove MRT and LET (#5172)
Contributes to #4194.
Closes #4231.
Closes #5022.
CHANGELOG_BEGIN
- [Ledger API] The protobuf fields ledger_effective_time and maximum_record_time have been removed from
  command submission. These fields were previously deprecated following the introduction
  of a new ledger time model. See issue `#4194 <https://github.com/digital-asset/daml/issues/4194>`__.
[Java Bindings] removed the usage of ledgerEffectiveTime and
maximumRecordTime, and instead added minLedgerTimeAbsolute and
minLedgerTimeRelative in CommandSubmissionClient and CommandClient
CHANGELOG_END
2020-03-25 16:50:27 +01:00
Samir Talwar
bfe27d23db
sandbox: Capture timing metrics for API server calls. (#5145)
* sandbox: Capture timing metrics for API server calls.

`timer` is a superset of `meter`, so this doesn't lose any existing
behavior; just adds new behavior.

CHANGELOG_BEGIN
- [Ledger API Server] Added timing metrics for all GRPC endpoints.
CHANGELOG_END

* sandbox: Rename SandboxClientResource to GrpcClientResource.

* sample-service: Clean up warnings.

* sandbox: Add tests for MetricsInterceptor.

* sandbox: Split the API metrics interceptor from the naming.

* sandbox: Use `MetricRegistry.name` instead of string interpolation.

* rs-grpc-akka: Restrict the test library to the DAML workspace.

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-03-25 09:56:37 +00:00
Robert Autenrieth
3f597aae16
New ledger time (#5100)
* Tighten result type

Command execution can't result in a sequencer error

* New helper method for extracting used contracts

* New error clause

* Add a DAO query for the maximum time of contracts

* Implement algorithm for finding ledger time

CHANGELOG_BEGIN
CHANGELOG_END

* fixup ledgerTimeHelper

* Use new ledger time algorithm

* Mark LET/MRT as deprecated

CHANGELOG_BEGIN
- [Ledger API] DAML ledgers have switched to a new ledger time model.
  The ledger_effective_time and maximum_record_time fields of command submission are deprecated,
  the ledger time of transactions is instead set automatically by the ledger API server.
  Ledger time is no longer strictly monotonically increasing, but only follows causal monotonicity:
  ledger time of transactions is greater than or equal to the ledger time of any used contract.
  See `#4345 <https://github.com/digital-asset/daml/issues/4345>`__.
CHANGELOG_END

* Add ledger time skew check

* Remove command updater

LET/MRT are now deprecated, this class is now useless

* Remove old time model validator

* Switch to new time model check: kvutils

* Switch to new time model check: in-memory ledger

* Switch to new time model check: SqlLedger

* Use initial ledger config

* Ignore user provided LET

* Use TimeProvider in submission services

* Use deduplication_time in daml-script runner

- Also remove unnecessary command completion output of CommandTracker.
- Remove usage of maximum record time in CommandTracker.

* Use arbitrary default value for deduplication time

* Use built-in Instant ordering

* Remove obsolete test

* Remove obsolete test: CommandStaticTimeIT

* Refactor test: TransactionMRTCompliance

* Disable test: CommandTrackerFlow timeout

* thread maxDeduplicationTime through to CommandTracker

* Improve test

* Refactor command client configuration

* Deduplication time should always use UTC

* Add missing method in TimedIndexService after rebase

* Put more details into the deduplication error response.

* Use system time for command dedup submittedAt.

* Use explicit UTC time source in command validator

* Revert CommandTracker[Flow] to previous completion-recovering-behavior

* Adapt scala client command config to new config params

Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-03-25 09:28:56 +01:00
Martin Huschenbett
f8e383daf6
daml2ts: Consolidate ESLint exception in generated code (#5167)
Currently, we write exceptions for using namespaces and all exceptions in
`index.ts` at the place where they occur. This is pretty noisy.

After this PR, we write all exceptions at the beginning of the file.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 22:19:00 +00:00
Stefano Baghino
181df8f3ab
Act on Option2Iterable wart (#5154)
Some Option2Iterable ignore annotations are not needed, others were needed for unused methods.

In a few occasions we were ignoring the warning for the very purpose for which is was there,
i.e. avoiding an implicit conversion. I'm all for not verifying this rule if we agree we
don't need it.

For ProcessFailedException it was a bit gratuitous, I changed the way in which the exception
message is built.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 11:25:05 +00:00
Moritz Kiefer
2b47f6994d
Switch default DAML-LF target to 1.8 (#5127)
changelog_begin

- [DAML Compiler] The default output DAML-LF target version is now
  1.8. You can target 1.7 by specifying ``--target=1.7`` in the
  ``build-options`` field in your ``daml.yaml``.

changelog_end

Fix daml2ts tests
2020-03-24 11:02:13 +01:00
Shayne Fletcher
a5faa1cdbc
Use local paths in package dependencies (#5137)
changelog_begin
changelog_end
2020-03-23 13:54:56 -04:00
Shayne Fletcher
bf22d8db59
Fix package name (#5120)
changelog_begin
changelog_end
2020-03-22 12:01:56 -04:00
Robin Krom
7a9648c714
language: react: tests for useReload. (#5095)
This adds tests to see that the `useReload` hook of @daml/react works.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 22:25:28 +01:00
Shayne Fletcher
78d34dd9ca
Experiment with 'eitherDecode' (#5098)
changelog_begin
changelog_end
2020-03-19 20:07:19 +00:00
Martin Huschenbett
8bd2172569
daml2ts: Remove "itself" from progress output (#5088)
I've received some feedback that the "itself" in
```
Generating <some hash> as itself
```
is confusing. Also, the first noun in each line said what is being
_processed_ not what is being _generated_. I've fixed that too.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 18:05:54 +01:00
Robin Krom
b7f36d5e10
streamQuery tests (#5059)
* language:daml-react: tests for useStreamQuery/useStreamFetchByKey

This adds tests for `useStreamQuery` and `useStreamFetchByKey` of the
`@daml/react` library.

CHANGELOG_BEGIN
CHANGELOG_END

* renamed hooks.test.ts -> index.test.ts
2020-03-19 16:42:34 +01:00
Robert Autenrieth
60a845ec72
Add new protobuf fields for new ledger time model (#5036)
* Add new protobuf fields for new ledger time model

Fixes #4344.

CHANGELOG_BEGIN
CHANGELOG_END

* Adapt haskell bindings
2020-03-19 13:53:49 +01:00
Martin Huschenbett
d9f2eed632
daml2ts: Simplify generated build config (#5085)
We don't use TypeScript's project references. Hence there's no point in
passing `--build` to `tsc`. Let's stop doing that hence.

By default, the TypeScript compiler type checks the `.d.ts` of all
dependencies of a project. This is painfully slow. For the packages
generated by `daml2ts` it also doesn't make a lot of sense since we control
all dependencies except for `@mojotech/json-type-validation`, which is
written in TypeScript itself and hence has very sane typings anyway.

This default behaviour can be disabled by setting `skipLibCheck` to `true`.
Doing that decreases the compilation time of _every_ single package generated
by `daml2ts` for the DAVL project from ~10s to ~2s. Let's get that time back!

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 12:21:05 +00:00
Shayne Fletcher
9fbd3da302
Use type application for 'decode' (#5081)
changelog_begin
changelog_end
2020-03-19 06:53:04 -04:00
Remy
c097266edc
java-binding: fix protobuf testing generator (#5082)
* exclude com.daml.ledger.javaapi.data.ValueSpec test

* Revert "exclude com.daml.ledger.javaapi.data.ValueSpec test"

This reverts commit 9b737bc452.

* java-binding: fix testing Generators

* revert minor change

* changelog

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-19 09:59:36 +00:00
nickchapman-da
d81caa9568
rework daml_compile bazel rule (#5070)
Avoiding `damlc compile/package` commands (which we would like to deprecate), and replace with plain `damlc build` together with a post dar->dalf extraction step in the couple of places where we actually want the .dalf for testing.

changelog_begin
changelog_end
2020-03-18 23:06:55 +00:00
Samir Talwar
cbeeb5aafc
sandbox: Fail to start if a time mode is not explicitly specified. (#5033)
* sandbox: Fail to start if a time mode is not explicitly specified.

CHANGELOG_BEGIN
- [Sandbox] Sandbox is switching from Static Time mode to Wall Clock
  Time mode as the default. To ensure that our users know about this,
  for one version, there will be no default time mode. Instead, users
  will have to explicitly select their preferred time mode by means of
  the `--static-time` or `--wall-clock-time` switches. In the next
  release, Wall Clock Time will become the default, and users who are
  happy with the defaults will no longer need to specify the time mode.
CHANGELOG_END

* daml-script|triggers: Specify time mode when testing against Sandbox.

* daml-assistant: Default the Sandbox to wall clock time.

CHANGELOG_BEGIN
- [DAML Assistant] Initializing a new DAML project adds a switch to
  ``daml.yaml`` to ensure Sandbox can continue to start with ``daml
  start``::

      sandbox-options:
        - --wall-clock-time
CHANGELOG_END

* docs: Update the DAML Script and Triggers docs to use Wall Clock time.

It's now what Sandbox will use by default when using `daml init`.

* docs: Change the Quickstart to run Sandbox in wall clock time.

This explains why the contract IDs may vary.

It also updates the manual release testing script to match.
2020-03-18 08:25:03 +00:00
Gerolf Seitz
cf032a02cf
Introduce Stable Offsets (#4953)
A "stable offset" in the context of the Participant Server is the offset
that was provided by the ledger backend (be it kvutils, corda, daml on sql).

The Participant Server does not keep a participant-local offset anymore.
In a single domain/kvutil setup, this makes offsets stable across participants,
since all participants will see the same offset for the same transaction.

The following changes were needed to achieve this:
- The participant server always uses the offset provided by the backend
  AS IS (no more +1 magic).
- Offsets provided to the Ledger API in requests must be treated as
  startExclusive and endInclusive (previously beginInclusive and
  endExclusive).

CHANGELOG_BEGIN
[Ledger API]: Offsets have been redefined. Instead of being represented
by a number or a structured string, an offset is now an opaque string
that can be compared lexicographically.
[DAML Integration Kit]: The bounds for ``Dispatcher`` are now
startExclusive and endInclusive.
CHANGELOG_END

---------
ledger api:
ledger_offset.proto
  Changed definition of offsets, since they can now be compared
  lexicographically.

---------
participant-state-api:
Offset:
  Changed from Array[Long] to ByteString. Ledgers need to make sure that the
  offsets produced are strictly monotonically increasing according to
  lexicographical order.

---------
akka-streams:
Dispatcher, DispatcherImpl, SubSource:
  Changed interval handling to exclusive/inclusive.

---------
ledger-on-memory:
InMemoryLedgerReaderWriter, InMemoryState:
  Changed interval handling to exclusive/inclusive.

---------
ledger-on-sql:
CommonQueries, SqlLedgerReaderWriter:
  Change interval in query and boundary handling.

---------
kvutils:
KeyValueParticipantStateReader, KVOffset:
  Convenience functions for kvutils to add or remove sub-indexes for
  offsets.
  KV ledger implementations can use KVOffset to construct a structured offset.

---------
Participant Server:
JdbcLedgerDao:
  Use Offset instead of Long.
  Fetch offsets directly as Offset from the database with proper anorm
  integration.
  Change interval handling to exclusive/inclusive.

CommandCompletionsReader, CommandCompletionsTable:
  Change interval handling to exclusive/inclusive.

BaseLedger:
  Use Offset instead of Long.
  Change interval handling to exclusive/inclusive.

Conversions:
  Anorm integration for using Offset in queries and result parsers.

JdbcIndexer:
  Remove references to "extenalLedgerEnd" and participant-local Long
  offset (headRef).

---------
sandbox:
In general:
  Use the Offset type everywhere instead of Long.

SQL migrations:
  Change all offset columns to bytea or BINARY.

LedgerBackedIndexService:
  Proper bounds checking has been pushed down to Dispatcher, which
  allowed simplifying the acceptedTransactions implementation.

InMemoryLedger, LedgerEntries:
  Change interval handling to exclusive/inclusive.
  Transaction lookup by ID is now O(n) because transaction IDs are not
  necessarily the same as the offset.

SqlLedger:
  Remove external offset references.
2020-03-18 08:43:41 +01:00
Shayne Fletcher
e71cba698e
daml2ts : Simplify docs, use mapped types for enums. (#5044)
* Simplify docs, use mapped types for enums.

changelog_begin
changelog_end

* remove serializale check

* add 'keys' property to enums

* Simplify docs just a little bit more
2020-03-17 16:27:46 +00:00
Robin Krom
e537b496ee
Deprecate useExercise (#4972)
* language:daml-react: deprecate useExercise

We replace 'useExercise', 'useExerciseByKey' with 'useLedger' and expect
a user to call the ledger methods instead.

CHANGELOG_BEGIN
CHANGELOG_END

* addressing martin's comments
2020-03-16 14:55:59 +01:00
Stefano Baghino
1f0534ea4f
Spin-off BuildInfo into its own (micro-)library (#5004)
* Spin-off BuildInfo into its own (micro-)library

CHANGELOG_BEGIN
CHANGELOG_END

* Fix dependencies

* Remove unused dependency
2020-03-16 09:04:00 +01:00
Shayne Fletcher
8f4404b945
Avoid explicit test for package.json exists (#5005)
changelog_begin
changelog_end
2020-03-14 18:00:37 -04:00
Robin Krom
990be40168
docs: typedoc comments for daml-types (#5000)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-13 22:18:54 +01:00
Shayne Fletcher
17d85afbf6
The -p flag to daml2ts is mandatory; create package.json if not there (#5001)
changelog_begin
changelog_end
2020-03-13 17:15:53 -04:00
Robin Krom
b3f20dda8d
docs: Nicer typedocs for daml-ledger (#4993)
This polishes the generated docs for daml-ledger.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-13 22:04:10 +01:00
Gary Verhaegen
364262a7f6
saner scalafmt target (#4985)
The current behaviour of our scalafmt checks compares for changes with
origin/master, which means it is dependent on the state of the local git
repository. This makes it non-reproducible.

Added to the fact that the master branch is not currently green as per
our scalafmt rules, this makes it impossible to rebuild older commits,
which in turn could interfere with our release process.

This PR does two things:

1. Fix our codebase to agree with our formatting rules.
2. Add a flag to `fmt.sh` to enable scalafmt's diff behaviour, and
   change the default to a full scan.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-13 17:37:13 +01:00
Shayne Fletcher
0db2273a53
Make it so you can run in a known location (#4989)
changelog_begin
changelog_end
2020-03-13 12:12:22 -04:00
Moritz Kiefer
4e99f18613
Introduce a DA.Test.Sandbox module for managing sandbox in tasty (#4986)
We previously had 3 slightly different but consistently shitty logic
for handling this in the tests for daml-helper daml repl and the
Haskell ledger bindings. This PR introduces a module that is flexible
enough to capture all their needs and hopefully is somewhat less
shitty.

changelog_begin
changelog_end
2020-03-13 15:35:03 +01:00
Shayne Fletcher
33e384869a
Change code generation scheme for DAML enums (#4975)
changelog_begin
changelog_end
2020-03-13 09:30:52 -04:00
Robin Krom
b748f88fa2
language: docs: use typedoc comments for daml-react (#4977)
Using the correct typedoc comment form gives nice generated
documentation.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-12 21:54:33 +01:00
Robin Krom
f6936a6d6e
docs: fix: include html assets for generated docs (#4974)
The tar command in the bazel rule would not dereference links in the
documentation folder which let to missing assets. This is fixed with the
additional command line flag `-h` to tar.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-12 19:37:17 +01:00
Shayne Fletcher
cdc31ebe81
rename package 'build-and-lint' to 'build-and-lint-test' (#4957)
changelog_begin
changelog_end
2020-03-11 17:38:56 -04:00
Robin Krom
254ee50b7c
docs: better readme's for ts libraries (#4954)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-11 21:35:42 +01:00
Leonid Shlyapnikov
19f5b86c79
Websocket stream to emit last seen offset instead of heartbeat message (#4886)
* Capture lastSeenOffset in the @volatile var

CHANGELOG_BEGIN
[JSON API - Experimental] Websocket stream now emits last seen offset instead of the heartbeat message.
``{"heartbeat": "ping"}`` is replaced by ``{"events":[],"offset":"<last seen offset>"}``. See #4510.
CHANGELOG_END

* updating docs

* moving the last seen offset into the stream, WIP

* adding in-stream state

* minor docs

* cleanup the heartbeat logic

* minot cleanup

* Change live and heartbeat msg handling + some debug logging (to be removed)

* fixing ts tests, cleaning up

* Adding todo with the reference to the follow-up ticket

* Adding todo with the reference to the follow-up ticket
2020-03-11 19:53:05 +00:00
Shayne Fletcher
901b28777e
Reduce repetition by reuse of 'modPath' (#4951)
* Reduce repetition by reuse of 'modPath'

changelog_begin
changelog_end

* Use intercalate not joinPath; import </> from FilePath.Posix

These two changes should remove any ambiguity about whether paths are
treated any differently on Windows vs 'nix.
2020-03-11 15:21:01 -04:00
Shayne Fletcher
793609fd04
client module import ideal syntax (#4946)
changelog_begin
changelog_end
2020-03-11 13:33:16 -04:00
Moritz Kiefer
ed5886051d
Support TLS in daml-helper (#4943)
This PR adds TLS support to DAML helper both via client certs and
without (although the latter is not tested so far since atm this is
not supported by sandbox). The CLI options follow the scheme used by
navigator/extractor/… with the addition that you can just pass `--tls`
which will turn on TLS without custom root certs or client certs.

changelog_begin

- [DAML Assistant] You can now connect to ledger via TLS for ``daml
  deploy`` and ``daml ledger`` commands. See
  https://docs.daml.com/deploy/generic_ledger.html for more information.

changelog_end
2020-03-11 17:58:05 +01:00
Shayne Fletcher
7634b7cfad
Do not require the user type '@' (#4939)
changelog_begin
changelog_end
2020-03-11 11:27:15 -04:00
Leonid Shlyapnikov
953d55f338
Replace AsyncWordSpec with WordSpec, thanks @rautenrieth-da, see #4938 (#4941)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-11 14:58:38 +00:00
Robert Autenrieth
42adfb1887
Fix tests not running with bazel test (#4938)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-11 15:15:02 +01:00
Shayne Fletcher
2e8f8e5192
make generated packages easier to use (#4926)
changelog_begin
changelog_end
2020-03-11 09:03:13 -04:00
Robin Krom
8fe497ea5d
lanugage:docs: include generated docs on docs.daml.com (#4924)
This includes the generated docs for the typescipt libraries daml-react,
daml-ledger and daml-types in the documentation presented on
docs.daml.com. Next step is to create better readmes in this libraries.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-10 18:46:54 +01:00