* Move 'withEnv', call it from daml2ts tests
changelog_begin
changelog_end
* Fix withEnv call to ensure that TASTY_NUM_THREADs is set
withEnv replaces the whole environment so we need to set everything we
care about.
* withEnv replaces the whole environment so we need to set everything we
care about.
* Apparently applying the same fix has destabilized Windows
* Try even harder to get daml assistant tests passing on Windows again
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Update the TypeScript version we're using in `daml2ts` and its support
libraries. Also update the versions of ESLint and its TypeScript
plugin we're using for linting the generated code. (And remove more
occurrences of `HashMap Text Text` at the same time.)
CHANGELOG_BEGIN
CHANGELOG_END
* Remove old time model from ledger config
CHANGELOG_BEGIN
- [Ledger API] Fields related to the old ledger time
model have been removed from the configuration
management service and the ledger configuration service.
CHANGELOG_END
* Update ledger/ledger-api-test-tool/src/main/scala/com/daml/ledger/api/testtool/tests/LedgerConfigurationService.scala
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
As I said previously, all instances of the dreaded `HashMap Text Text`
can go away if write our code more direct. Here we go.
CHANGELOG_BEGIN
CHANGELOG_END
We do not need to have templates in the `.daml` files we generate
during these templates, any serializable type is good enough. Thus,
let's make the tests shorter by using a simple serializable type
instead of templates.
Also make the tests easier to debug when expected failures of `daml2ts`
do not happen in the anticipated way.
CHANGELOG_BEGIN
CHANGELOG_END
* daml2ts: Fix the A:X vs A.B bug
Currently, the TypeScript generated by `daml2ts` does not compile when
there are a modules `A` and `A.B` which both contain serializable types.
(Note: there can't be a type called `B` in `A` due to the name collision
check.)
Since fixing this within the current layout using TypeScript
namespaces would be very challenging since we would need to inject
subnamespaces into existing namespaces. Thus, we switch the code
generation to use ES2015 modules. Since TypeScript namespaces are
considered (somewhat) deprecated, this is a good move anyway.
As a result of this change to ES2015 we need to write `index.ts` files
at each node of the module hierarchy. These `index.ts` files reexport
the module at the current node (if present) and all child modules.
At the same time we also remove the default export of the top-level
`index.ts` file. It is not clear if they are actually useful and to
add them to the new scheme requires a technique that feels like a hack.
To give an example, assume we have modules `A` and `A.B.C`. We now write
the following directory structure:
```
src/
+- index.ts <- reexports A
+- A/
+- index.ts <- reexports module.ts and B
+- module.ts
+- B/
+- index.ts <- reexport C
+- C/
+- index.ts <- reexports module.ts
+- module.ts
```
If you import the package like
```typescript
import * as foo from '@daml.js/foo-1.0.0';
```
you can acces the `A` and `A.B.C` modules as `foo.A` and `foo.A.B.C`.
CHANGELOG_BEGIN
CHANGELOG_END
* Adapt build-and-lint-test
CHANGELOG_BEGIN
CHANGELOG_END
* Add new test
CHANGELOG_BEGIN
CHANGELOG_END
Instead of writing the TypeScript for DAML module `A.B.C` into the file
`src/A/B/C.ts` we now write it into `src/A/B/C/module.ts`. This is in
preparation for also writing a file `src/A/B/C/index.ts`, which
re-exports `src/A/B/C/module` but also `src/A/B/C/D` for each `D`.
We also make sure to use the correct path separator on Windows.
CHANGELOG_BEGIN
CHANGELOG_END
Instead of reaching into to the internals of external dependencies,
we no import these package via their root `index.ts` file. This gives
us the possibility of the changing the internal structure without
having to think about this kind of import as long as we keep the
interface the same. This is exactly what I plan to do in the next PR
which will fix the `A` vs `A.B` bug.
More precisely, instead of writing two imports like
```typescript
import * as pkgXYZ_A_B from '@daml.js/foo-0.0.1/lib/A/B';
import * as pkgXYZ_C from '@daml.js/foo-0.0.1/lib/C';
```
we only write one import
```typescript
import pkgXYZ from '@daml.js/foo-0.0.1';
```
and replace use sites of `pkgXYZ_A_B` and `pkgXYZ_C` with `pkgXYZ.A.B`
and `pkgXYZ.C`, resp.
CHANGELOG_BEGIN
CHANGELOG_END
This prepares a followup change in which I want to import external
dependecies entirely through their `index.ts` file. This will become easier
once the differences between internal and external imports are more
apparent.
Importing external dependencies through their `index.ts` files is part
of a restructuring the file layout in order to solve the bug where we
cannot have modules `A` and `A.B` at the same time.
CHANGELOG_BEGIN
CHANGELOG_END
There's no more reason to have the package id in a separate file
`packageId.ts`. This was different before we had an `index.ts`.
CHANGELOG_BEGIN
CHANGELOG_END
Contributes to #4194.
Closes#4231.
Closes#5022.
CHANGELOG_BEGIN
- [Ledger API] The protobuf fields ledger_effective_time and maximum_record_time have been removed from
command submission. These fields were previously deprecated following the introduction
of a new ledger time model. See issue `#4194 <https://github.com/digital-asset/daml/issues/4194>`__.
[Java Bindings] removed the usage of ledgerEffectiveTime and
maximumRecordTime, and instead added minLedgerTimeAbsolute and
minLedgerTimeRelative in CommandSubmissionClient and CommandClient
CHANGELOG_END
* sandbox: Capture timing metrics for API server calls.
`timer` is a superset of `meter`, so this doesn't lose any existing
behavior; just adds new behavior.
CHANGELOG_BEGIN
- [Ledger API Server] Added timing metrics for all GRPC endpoints.
CHANGELOG_END
* sandbox: Rename SandboxClientResource to GrpcClientResource.
* sample-service: Clean up warnings.
* sandbox: Add tests for MetricsInterceptor.
* sandbox: Split the API metrics interceptor from the naming.
* sandbox: Use `MetricRegistry.name` instead of string interpolation.
* rs-grpc-akka: Restrict the test library to the DAML workspace.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Tighten result type
Command execution can't result in a sequencer error
* New helper method for extracting used contracts
* New error clause
* Add a DAO query for the maximum time of contracts
* Implement algorithm for finding ledger time
CHANGELOG_BEGIN
CHANGELOG_END
* fixup ledgerTimeHelper
* Use new ledger time algorithm
* Mark LET/MRT as deprecated
CHANGELOG_BEGIN
- [Ledger API] DAML ledgers have switched to a new ledger time model.
The ledger_effective_time and maximum_record_time fields of command submission are deprecated,
the ledger time of transactions is instead set automatically by the ledger API server.
Ledger time is no longer strictly monotonically increasing, but only follows causal monotonicity:
ledger time of transactions is greater than or equal to the ledger time of any used contract.
See `#4345 <https://github.com/digital-asset/daml/issues/4345>`__.
CHANGELOG_END
* Add ledger time skew check
* Remove command updater
LET/MRT are now deprecated, this class is now useless
* Remove old time model validator
* Switch to new time model check: kvutils
* Switch to new time model check: in-memory ledger
* Switch to new time model check: SqlLedger
* Use initial ledger config
* Ignore user provided LET
* Use TimeProvider in submission services
* Use deduplication_time in daml-script runner
- Also remove unnecessary command completion output of CommandTracker.
- Remove usage of maximum record time in CommandTracker.
* Use arbitrary default value for deduplication time
* Use built-in Instant ordering
* Remove obsolete test
* Remove obsolete test: CommandStaticTimeIT
* Refactor test: TransactionMRTCompliance
* Disable test: CommandTrackerFlow timeout
* thread maxDeduplicationTime through to CommandTracker
* Improve test
* Refactor command client configuration
* Deduplication time should always use UTC
* Add missing method in TimedIndexService after rebase
* Put more details into the deduplication error response.
* Use system time for command dedup submittedAt.
* Use explicit UTC time source in command validator
* Revert CommandTracker[Flow] to previous completion-recovering-behavior
* Adapt scala client command config to new config params
Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
Currently, we write exceptions for using namespaces and all exceptions in
`index.ts` at the place where they occur. This is pretty noisy.
After this PR, we write all exceptions at the beginning of the file.
CHANGELOG_BEGIN
CHANGELOG_END
Some Option2Iterable ignore annotations are not needed, others were needed for unused methods.
In a few occasions we were ignoring the warning for the very purpose for which is was there,
i.e. avoiding an implicit conversion. I'm all for not verifying this rule if we agree we
don't need it.
For ProcessFailedException it was a bit gratuitous, I changed the way in which the exception
message is built.
CHANGELOG_BEGIN
CHANGELOG_END
changelog_begin
- [DAML Compiler] The default output DAML-LF target version is now
1.8. You can target 1.7 by specifying ``--target=1.7`` in the
``build-options`` field in your ``daml.yaml``.
changelog_end
Fix daml2ts tests
I've received some feedback that the "itself" in
```
Generating <some hash> as itself
```
is confusing. Also, the first noun in each line said what is being
_processed_ not what is being _generated_. I've fixed that too.
CHANGELOG_BEGIN
CHANGELOG_END
* language:daml-react: tests for useStreamQuery/useStreamFetchByKey
This adds tests for `useStreamQuery` and `useStreamFetchByKey` of the
`@daml/react` library.
CHANGELOG_BEGIN
CHANGELOG_END
* renamed hooks.test.ts -> index.test.ts
We don't use TypeScript's project references. Hence there's no point in
passing `--build` to `tsc`. Let's stop doing that hence.
By default, the TypeScript compiler type checks the `.d.ts` of all
dependencies of a project. This is painfully slow. For the packages
generated by `daml2ts` it also doesn't make a lot of sense since we control
all dependencies except for `@mojotech/json-type-validation`, which is
written in TypeScript itself and hence has very sane typings anyway.
This default behaviour can be disabled by setting `skipLibCheck` to `true`.
Doing that decreases the compilation time of _every_ single package generated
by `daml2ts` for the DAVL project from ~10s to ~2s. Let's get that time back!
CHANGELOG_BEGIN
CHANGELOG_END
Avoiding `damlc compile/package` commands (which we would like to deprecate), and replace with plain `damlc build` together with a post dar->dalf extraction step in the couple of places where we actually want the .dalf for testing.
changelog_begin
changelog_end
* sandbox: Fail to start if a time mode is not explicitly specified.
CHANGELOG_BEGIN
- [Sandbox] Sandbox is switching from Static Time mode to Wall Clock
Time mode as the default. To ensure that our users know about this,
for one version, there will be no default time mode. Instead, users
will have to explicitly select their preferred time mode by means of
the `--static-time` or `--wall-clock-time` switches. In the next
release, Wall Clock Time will become the default, and users who are
happy with the defaults will no longer need to specify the time mode.
CHANGELOG_END
* daml-script|triggers: Specify time mode when testing against Sandbox.
* daml-assistant: Default the Sandbox to wall clock time.
CHANGELOG_BEGIN
- [DAML Assistant] Initializing a new DAML project adds a switch to
``daml.yaml`` to ensure Sandbox can continue to start with ``daml
start``::
sandbox-options:
- --wall-clock-time
CHANGELOG_END
* docs: Update the DAML Script and Triggers docs to use Wall Clock time.
It's now what Sandbox will use by default when using `daml init`.
* docs: Change the Quickstart to run Sandbox in wall clock time.
This explains why the contract IDs may vary.
It also updates the manual release testing script to match.
A "stable offset" in the context of the Participant Server is the offset
that was provided by the ledger backend (be it kvutils, corda, daml on sql).
The Participant Server does not keep a participant-local offset anymore.
In a single domain/kvutil setup, this makes offsets stable across participants,
since all participants will see the same offset for the same transaction.
The following changes were needed to achieve this:
- The participant server always uses the offset provided by the backend
AS IS (no more +1 magic).
- Offsets provided to the Ledger API in requests must be treated as
startExclusive and endInclusive (previously beginInclusive and
endExclusive).
CHANGELOG_BEGIN
[Ledger API]: Offsets have been redefined. Instead of being represented
by a number or a structured string, an offset is now an opaque string
that can be compared lexicographically.
[DAML Integration Kit]: The bounds for ``Dispatcher`` are now
startExclusive and endInclusive.
CHANGELOG_END
---------
ledger api:
ledger_offset.proto
Changed definition of offsets, since they can now be compared
lexicographically.
---------
participant-state-api:
Offset:
Changed from Array[Long] to ByteString. Ledgers need to make sure that the
offsets produced are strictly monotonically increasing according to
lexicographical order.
---------
akka-streams:
Dispatcher, DispatcherImpl, SubSource:
Changed interval handling to exclusive/inclusive.
---------
ledger-on-memory:
InMemoryLedgerReaderWriter, InMemoryState:
Changed interval handling to exclusive/inclusive.
---------
ledger-on-sql:
CommonQueries, SqlLedgerReaderWriter:
Change interval in query and boundary handling.
---------
kvutils:
KeyValueParticipantStateReader, KVOffset:
Convenience functions for kvutils to add or remove sub-indexes for
offsets.
KV ledger implementations can use KVOffset to construct a structured offset.
---------
Participant Server:
JdbcLedgerDao:
Use Offset instead of Long.
Fetch offsets directly as Offset from the database with proper anorm
integration.
Change interval handling to exclusive/inclusive.
CommandCompletionsReader, CommandCompletionsTable:
Change interval handling to exclusive/inclusive.
BaseLedger:
Use Offset instead of Long.
Change interval handling to exclusive/inclusive.
Conversions:
Anorm integration for using Offset in queries and result parsers.
JdbcIndexer:
Remove references to "extenalLedgerEnd" and participant-local Long
offset (headRef).
---------
sandbox:
In general:
Use the Offset type everywhere instead of Long.
SQL migrations:
Change all offset columns to bytea or BINARY.
LedgerBackedIndexService:
Proper bounds checking has been pushed down to Dispatcher, which
allowed simplifying the acceptedTransactions implementation.
InMemoryLedger, LedgerEntries:
Change interval handling to exclusive/inclusive.
Transaction lookup by ID is now O(n) because transaction IDs are not
necessarily the same as the offset.
SqlLedger:
Remove external offset references.
* Simplify docs, use mapped types for enums.
changelog_begin
changelog_end
* remove serializale check
* add 'keys' property to enums
* Simplify docs just a little bit more
* language:daml-react: deprecate useExercise
We replace 'useExercise', 'useExerciseByKey' with 'useLedger' and expect
a user to call the ledger methods instead.
CHANGELOG_BEGIN
CHANGELOG_END
* addressing martin's comments
The current behaviour of our scalafmt checks compares for changes with
origin/master, which means it is dependent on the state of the local git
repository. This makes it non-reproducible.
Added to the fact that the master branch is not currently green as per
our scalafmt rules, this makes it impossible to rebuild older commits,
which in turn could interfere with our release process.
This PR does two things:
1. Fix our codebase to agree with our formatting rules.
2. Add a flag to `fmt.sh` to enable scalafmt's diff behaviour, and
change the default to a full scan.
CHANGELOG_BEGIN
CHANGELOG_END
We previously had 3 slightly different but consistently shitty logic
for handling this in the tests for daml-helper daml repl and the
Haskell ledger bindings. This PR introduces a module that is flexible
enough to capture all their needs and hopefully is somewhat less
shitty.
changelog_begin
changelog_end
The tar command in the bazel rule would not dereference links in the
documentation folder which let to missing assets. This is fixed with the
additional command line flag `-h` to tar.
CHANGELOG_BEGIN
CHANGELOG_END
* Capture lastSeenOffset in the @volatile var
CHANGELOG_BEGIN
[JSON API - Experimental] Websocket stream now emits last seen offset instead of the heartbeat message.
``{"heartbeat": "ping"}`` is replaced by ``{"events":[],"offset":"<last seen offset>"}``. See #4510.
CHANGELOG_END
* updating docs
* moving the last seen offset into the stream, WIP
* adding in-stream state
* minor docs
* cleanup the heartbeat logic
* minot cleanup
* Change live and heartbeat msg handling + some debug logging (to be removed)
* fixing ts tests, cleaning up
* Adding todo with the reference to the follow-up ticket
* Adding todo with the reference to the follow-up ticket
* Reduce repetition by reuse of 'modPath'
changelog_begin
changelog_end
* Use intercalate not joinPath; import </> from FilePath.Posix
These two changes should remove any ambiguity about whether paths are
treated any differently on Windows vs 'nix.
This PR adds TLS support to DAML helper both via client certs and
without (although the latter is not tested so far since atm this is
not supported by sandbox). The CLI options follow the scheme used by
navigator/extractor/… with the addition that you can just pass `--tls`
which will turn on TLS without custom root certs or client certs.
changelog_begin
- [DAML Assistant] You can now connect to ledger via TLS for ``daml
deploy`` and ``daml ledger`` commands. See
https://docs.daml.com/deploy/generic_ledger.html for more information.
changelog_end
This includes the generated docs for the typescipt libraries daml-react,
daml-ledger and daml-types in the documentation presented on
docs.daml.com. Next step is to create better readmes in this libraries.
CHANGELOG_BEGIN
CHANGELOG_END
* Deprecate ledger initialization with scenarios
CHANGELOG_BEGIN
[Sandbox] Initializing the sandbox with scenarios is now deprecated in
favor of using DAML Script. The scenario parameter will be removed in
the near future. A warning is logged on startup.
The DAML SDK templates and quickstart guide are using DAML Script.
See the DAML Script migration guide for more information:
https://docs.daml.com/daml-script/index.html#using-daml-script-for-ledger-initialization
CHANGELOG_END
* Shift comment and metadata write calls around
Also remove licence line from package.json files
changelog_begin
changelog_end
* Broken Windows cache. Unbreak it. NOT FOR MERGING TO MASTER.
* Restore Windows cache; safe now to merge.
Currently, the version of `@daml/types` used by the generated TS is
configurable. This is not very helpful since `daml2ts` and `@daml/types`
are developed in lock step and still moving quite fast.
This PR changes `daml2ts` to always use the SDK version for as the
version of `@daml/types`. This also requires us to fix the test
for DAVL which used a hard coded version of `@daml/types` so far.
CHANGELOG_BEGIN
CHANGELOG_END
* daml2ts: Factor out assertFileExists in tests
The pattern `assertBool "..." =<< doesFileExist ...` appears so many
times that it definitely deserves its own function.
CHANGELOG_BEGIN
CHANGELOG_END
* Remove assertions for existence of DAR files
CHANGELOG_BEGIN
CHANGELOG_END
Keep all fields in the workspace `package.json` given to `daml2ts`
instead of only `private` and `workspaces`.
This also allows for removing the `sed` hack in the `build-and-lint`
test and use the `resolution` field of the workspace `package.json`
to point to our local versions of `@daml/types` and `@daml/ledger`.
CHANGELOG_BEGIN
CHANGELOG_END
* Always return error on duplicate submissions
* Remove unnecessary submission information
Now that duplicate submissions always return an error,
we don't need to store the original submission result.
CHANGELOG_BEGIN
CHANGELOG_END
* Rename ttl to deduplicationTime/deduplicateUntil
* Store absolute deduplicateUntil in domain commands
* Fix my own initials
* Remove CommandDeduplicationEntry
Instead, use CommandDeduplicationResult everywhere,
removing the extra layer.
* libs-scala/ports: Wrap socket ports in a type, `Port`.
* sandbox: Use `Port` for the API server port, and propagate.
CHANGELOG_BEGIN
CHANGELOG_END
* extractor: Use `Port` for the server port.
* ports: Make Port a compile-time class only.
* ports: Allow port 0; it can be specified by a user.
* ports: Publish to Maven Central.
* Freeze DAML-LF 1.8
Two minor points that I did not mention in the previous PR:
We also include the renaming of structural records to `struct` and the
renaming of `Map` to `TextMap`.
There are some minor changes around the LF encoder tests which need to
be able to emit package metadata properly so I’ve added it to the
parser. Sorry for not splitting that out.
Following the process used for the DAML-LF 1.7 release, this does not
yet include the frozen proto file.
changelog_begin
- [DAML-LF] Release DAML-LF 1.8:
* Rename structural records to ``Struct``. Note that
structural records are not exposed in DAML.
* Rename ``Map`` to ``TextMap``.
* Add type synonyms. Note that type synonyms are not serializable.
* Add package metadata, i.e., package names and versions.
Note that the default output of ``damlc`` is stil DAML-LF 1.7. You
can produce DAML-LF 1.8 by passing ``--target=1.8``.
changelog_end
* Update encoder
* Update java codegen tests
* Update comment in scala codegen
* Handle TSynApp in interface reader
* Bump lf_stable_version to 1.7
* Fix kvutils tests
This reduces the number of GHCs to 2 on Linux (regular and DWARF) and
1 on macOS. Given that each derivation is > 1 GB this should hopefully
help a bit.
changelog_begin
changelog_end
* move BeginBookmark to util
* adding offsets to steps
* offsetAfter belongs in Txn, not InsertDeleteStep
* make transaction stream a ContractStreamStep.Txn stream
* add several ContractStreamStep append cases
* rewrite 'render' to emit offset in the right places
* make ContractStreamStep#append total again
* check for offset in a few tests
* revert useless whitespace changes
* missed argument
* simpler mapPreservingIds
* rewrite states for new "live" format
* remove invalidated "events" block structure assertions
* make shutdown in withHttpService deterministic, to try to catch race condition
* exhaustiveness checking somehow disabled; fixed fetch flow and all is well
* documentation and changelog
CHANGELOG_BEGIN
- [JSON API - Experimental] Remove ``{"live": true}`` marker from websocket streams;
instead, live data is indicated by the presence of an "offset".
See `issue #4593 <https://github.com/digital-asset/daml/pull/4593>`_.
CHANGELOG_END
* be more specific about what liveness marker may be in docs
* fix daml2ts websocket tests
* mention type rules for all cases in offset documentation
* Add TTL field to protobuf
* Add command deduplication to index service
* Wire command deduplication to DAO
* Implement in-memory command deduplication
* Remove Deduplicator
* Implement JDBC command deduplication
* Add TTL field to domain commands
* Deduplicate commands in the submission service
CHANGELOG_BEGIN
- [Sandbox] Implement a new command submission deduplication mechanism
based on a time-to-live (TTL) for commands.
See https://github.com/digital-asset/daml/issues/4193
CHANGELOG_END
* Remove unused command service parameter
* fixup protobuf
* Add configuration for TTL
* Fix Haskell bindings
* Rename SQL table
* Add command deduplication test
* Redesign command deduplication queries
* Address review comment
* Address review comment
* Address review comments
* Make command deduplication test optional
* Disable more tests
* Address review comments
* Address review comments
* Refine test
* Address review comments
* scalafmt
* Truncate new table on reset
* Store original command result
* Rename table columns
... to be consistent with other upcoming tables
* Rename migrations to solve conflicts
Fixes#4193.
* @daml/react: Remove loading indication from useExercise hook
The indicator was a stupid idea of mine in the first place. Sharing the
loading indicator between potentially concurrent calls to the function
returned by the hooks does not make any sense.
`useExerciseByKey` has the same problem and it's fixed here as well.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix doc comments
CHANGELOG_BEGIN
CHANGELOG_END
Right now, the `I` type parameter of `CreateEvent` is omitted in all
the hooks and hence set to its default value `string`. This is very
unfortunate in upgrading settings where you end up with multiple
versions of the same template that are basically only distinguishable
by their template id.
CHANGELOG_BEGIN
CHANGELOG_END
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
* daml2ts: Don't fest @daml/types from npmjs.com in tests
We now copy the compiled version of `@daml/types` into the yarn
workspace instead of getting it from npmjs.com.
I verified that it works if I change the `VERSION` file to contain
0.13.55. Thus, we're definitely not going to npmjs.com.
CHANGELOG_BEGIN
CHANGELOG_END
* Disable tests on windows
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Avoid opening a server to the world when finding a free port.
This is very annoying on macOS because we get a focus-stealing popup for
a split second, asking for permission to allow the server through the
firewall. The popup pretty much always disappears before it can even be
read, when the server is closed.
This is almost certainly not an attack vector, because:
- we only do this in tests,
- the server is open for only a few milliseconds,
- nothing is served,
- and finding the port is tricky, because it's effectively random.
Nevertheless, it's very annoying.
CHANGELOG_BEGIN
CHANGELOG_END
* Extract a Bazel package for finding free ports.
We seem to do it in 4 different places, which I think is enough to
remove the duplication.
Add a test for `useQuery` that ensure that we don't call the JSON API
if the component calling the hook changes without changing the query.
CHANGELOG_BEGIN
CHANGELOG_END
* Step (1) Add error detection for different names/same package
changelog_begin
changelog_end
* Step (2) : Generate TS for a package once and only once.
Unfortunately, we need to work around some bazel issues which lead to
confliciting versions of react in our tests. This workaround cannot be
used the tests are invoked via `yarn test`. Thus, we only use it when
we the tests from bazel. We use the existence of the environment
variable `TEST_WORKSPACE` as a proxy for whether we run from bazel.
CHANGELOG_BEGIN
CHANGELOG_END
* @daml/react: Add an initial test suite
This only tests that the `useQuery` hook does the right thing once the
call to `Ledger.query` has resolved.
CHANGELOG_BEGIN
CHANGELOG_END
* Workaround for "Invalid hook call."
* Fix lint in BUILD file
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
Currently, we're using the first heartbeat as a guesstimate when we're
switching from the ACS to the live stream. These days, the JSON API has
a marker, namely `{live: true}`, to indicate this switch. This PR,
makes use of that marker.
CHANGELOG_BEGIN
CHANGELOG_END
Currently, contract ids are simply presented by strings. Thus, it is very
easy to accidentally mix up contract ids of different templates. This PR
is an attempt to provide more safety in this regard. It prevents contract
ids of template types which are not in a _structural_ subtyping
relationship from being mixed up. This is far from perfect, but clearly
better than what we have now.
CHANGELOG_BEGIN
CHANGELOG_END
This is one of the tests that keeps breaking every time we change the
number of packages `damlc build` outputs by default which is quite
annoying.
The actual change is trivial, we just read the number of packages from
the manifest. The diff is mostly just propagating this through everything.
changelog_begin
changelog_end
This is pretty much a verbatim copy of the `useStreamQuery` hook but it
works against the `/v1/stream/fetch` endpoint of the JSON API.
CHANGELOG_BEGIN
CHANGELOG_END
* Add type-level strings in DAML.
This PR adds a `PromotedText` stable package, with `PromotedText` type, which is used to encode type-level strings from DAML into DAML-LF. The reason for this is to preserve the `HasField` instance argument. This PR adds a test that `HasField` is succesfully reconstructed incontexts, during data-dependencies, which wasn't possible before.
changelog_begin
changelog_end
* adresss comments
* fix overly specific tests
* kvutils: Extract a committer from the uses of `SubmissionValidator`.
This makes the clock injectable too.
* kvutils: Provide logging contexts in the `Runner`.
* sandbox: Remove the `StaticAllowBackwards` time provider type.
It's not used anywhere.
* sandbox: Fix warnings in CliSpec.
* sandbox: Ensure that we cannot specify both static and wall-clock time.
* sandbox-next: Crash if wall clock time is not specified.
* sandbox-next: Document more known issues in the new Sandbox.
* sandbox: Add a Clock (and some tests) to TimeServiceBackend.
* sandbox-next: Support static time.
CHANGELOG_BEGIN
- [Sandbox Next] Re-establish static time mode.
CHANGELOG_END
* ledger-on-(memory|sql): Expect a `() => Instant`, not a `Clock`.
There's no point in first testing for the length of the array and then
testing all elements individually. That's what arrays are for. :)
CHANGELOG_BEGIN
CHANGELOG_END
Instead of only streaming the events we now primarily stream the set of
active contracts for the `Ledger.streamQuery` methods and the contract
pointed to by the key for the `Ledger.streamFetchByKey` method. The
events that lead to the latest state change are streamed as the second
argument to the event handler as well.
We also rename the event name from `'events'` to `'change'` since the
former is not longer accurate and also confusing and the latter captures
the generality of the streams we deal with here.
CHANGELOG_BEGIN
CHANGELOG_END
Shouldn’t really make a difference due to laziness but at least it
makes it explicit if we need to decode the archive to the AST or just
need to get the package id.
changelog_begin
changelog_end
* Return proper code for invalid authentication
CHANGELOG_BEGIN
[Sandbox] If authentication is enabled, requests without a valid
authentication are going to be rejected with an ``UNAUTHENTICATED``
return code instead of ``PERMISSION_DENIED``.
CHANGELOG_END
* Reduce logging noise from java-rxbindings tests
* Fix rxjava bindings tests to match new behavior
* Fix extractor tests to match new behavior
* Address https://github.com/digital-asset/daml/pull/4485#discussion_r378507478
* sandbox: Move more resource acquisition into the `owner`.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Reimplement SandboxClientResource as a resources.Resource.
* codegen: Use resources in TestUtil.
* sandbox: Manage PostgreSQL in tests with ResourceOwners.
Unfortunately, the development server of `create-react-app` does not
proxy websockets properly. Thus, we need to connect to the JSON API
directly when using it. This now be done by setting the
`REACT_APP_JSON_API_PORT` environment variable in `.env.development`.
CHANGELOG_BEGIN
CHANGELOG_END
* Translate unsupported kinds to a special Erased type
This should simplify `data-dependencies` and avoid issues like #4470
since we can match on the type instead of having to guess which types
can and which cannot be translated back to DAML.
changelog_begin
changelog_end
* sandbox: Don't hold on to old resources when resetting.
Now there's one hell of a memory leak.
CHANGELOG_BEGIN
- [Sandbox] Fixed a memory leak when using the ResetService; not
everything was cleaned up correctly.
CHANGELOG_END
* sandbox: Split out SandboxClientResource from SandboxServerResource.
Gonna replace SandboxServerResource with a ResourceOwner acquisition.
* sandbox: Don't capture the API server in the SandboxServer resource.
When we reset, this is stored forever, leading to a memory leak.
Tested by rewriting the SandboxServerResource to use
`SandboxServer.owner`.
* sandbox: Revert the test client resource to calling `shutdownNow()`.
* sandbox: Make sure the fixture is recreated properly on each test run.
* sandbox: Make `SandboxState` a non-case class.
The `toString()` was unnecessarily heavy.
* sandbox: Futures, futures everywhere.
Avoid a race condition where the server is stopped before it starts by
storing a `Future[SandboxState]` rather than the `SandboxState` itself.
This doesn't trigger the same memory leak as storing a
`Resource[SandboxState]` because we don't capture the object itself in
the `flatMap` in the same way with `Future`.
* sandbox: Remove an unused parameter left in for debugging.
* sandbox: Replace `@VisibleForTesting` with a comment.
* sandbox: Add more comments to the weird logic in SandboxServer.
* sandbox: Get rid of the `Port` type alias; it was confusing.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Not sure why the typechecker didn't catch this one.
I'm assuming this was changing an unused `laoding` field and the real
`loading` field was never used (or inaccurate) for `fetchByKeyResults`.
changelog_begin
changelog_end
* rename "contracts" to "events" in JSON API exercise response
CHANGELOG_BEGIN
- [JSON API - Experimental] Exercise response field "contracts" renamed to "events".
See `issue #4385 <https://github.com/digital-asset/daml/issues/4385>`_.
CHANGELOG_END
* more events in doc
- pointed out by @leo-da; thanks
* Explicitly add readme to packages
* npm packages repository metadata
As specified in https://docs.npmjs.com/files/package.json#repository
CHANGELOG_BEGIN
CHANGELOG_END
* Bazel format
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Build commonjs format for npm packaging
CHANGELOG_BEGIN
CHANGELOG_END
* Custom commonjs typescript library rule
The npm packages generated by rules_nodejs' native ts_library rule use
the UMD package format. This breaks webpack which attempts to determine
dependencies by static code analysis and fails on UMD. To avoid this we
call `tsc` directly to ensure generation of commonjs modules.
* Enable module mapping on ts_commonjs_library
* Replace ts_library by da_ts_library
* Add dummy typescript/index.bzl on Windows
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
In particular, the `preCheckCredentials` function is very specific to the
bad style of authentication we use in DAVL. Without that function, we can
also drop the `ledgerId` field and reach a point where inlining the
definition of `Credentials` makes sense.
CHANGELOG_BEGIN
CHANGELOG_END
Currently, we're not tracking references from key types. This becomes a
problem if the key is the only place referencing a module.
CHANGELOG_BEGIN
CHANGELOG_END
`TemplateStore` and `TemplateStore.addEvents` are at the heart of
`@daml/react`. They definitely warrant such a big test suite.
The removed tests for the `LedgerStore` were test for the `TemplateStore`
in disguise and are covered by the new tests.
CHANGELOG_BEGIN
CHANGELOG_END
- Replace `Ledger.fetchAll` by `Ledger.query` without `query` argument.
- Rename `Ledger.lookupByKey` to `Ledger.fetchByKey`.
- Add `Ledger.fetch`.
- Add tests and improve documentation.
The first three changes bring us closer to the JSON API, which is about to
rename its functions to the same schema.
CHANGELOG_BEGIN
CHANGELOG_END
Since `{} !== {}` in JavaScript, the `useQuery` hook used to indefinitely
reload all contracts when called without its second and third parameter.
Having a stable name for `{}` fixes the issue. Manual tests on
`create-daml-app` confirm this. An automated test suite for `@daml/react`
is still under develeopment.
CHANGELOG_BEGIN
CHANGELOG_END
* language: add daml-react package to ts libraries
This adds the library formerly known as `daml-react-hook` into the
monorepo. We renamed it to `@daml/react`.
The tests sadly don't work with bazel right now because the local
imports aren't resolved correctly. Local testing with `yarn run test`
works as usual.
CHANGELOG_BEGIN
CHANGELOG_END
* address moritz comments
* get rid of DAVL mentions
* fix eslint warnings
* Update language-support/ts/daml-react/tsconfig.json
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
We add a third type parameter `I` to the `Template` interface and use it as
the type of the `templateId` field. We add this parameter to `CreateEvent`,
`ArchiveEvent` and `Event` as well. The reason behind this is to allow for
patterns like
```ts
function handleFooOrBar(event: CreateEvent<Foo, Foo.Key, typeof Foo.templateId>
| CreateEvent<Bar, Bar.Key, typeof Bar.templateId>) {
switch (event.templateId) {
case Foo.templateId: ...
case Bar.templateId: ...
}
}
```
and get exhaustiveness checking. This will become particularly handy when
handling upgrades, where `Foo` and `Bar` would be the old and new version
of a template, resp.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-types.ts: Lint tests as well
Also simplify file selection a bit.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix build command in test script
CHANGELOG_BEGIN
CHANGELOG_END
The current type signature adds complexity which is unjustified in my
opinion. Who would expect meaningful results when specifying a template
by key `undefined`?
CHANGELOG_BEGIN
CHANGELOG_END
* make packages public
This uploads the typescript npm packages of the language support to the
npm registry in the release process.
CHANGELOG_BEGIN
CHANGELOG_END
* address moritz/gary's review
* generate the .npmrc file
* adding debug output
just in case the upload will fail in the next release.
* reverse package order
We rename the typescript packages of the language support as follows:
@digitalasset/daml-json-types -> @daml/types
@digitalasset/daml-ledger-fetch -> @daml/ledger
CHANGELOG_BEGIN
CHANGELOG_END
* Remove language-support/ts/packages/yarn.lock
That file is for local development exclusively.
CHANGELOG_BEGIN
CHANGELOG_END
* yarn args --frozen-lockfile
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Disable all the TS stuff on Windows
changelog_begin
changelog_end
* disable jest explicitly
* more disabling
* :sadpanda:
* Replace @language_support_ts_deps on Windows
Provides dummy content so that `load` commands are still valid on
Windows without `yarn_install`.
* disable daml-ledger-fetch on windows
* shut up buildifier
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Clean the TS vs Bazel stuff a bit
CHANGELOG_BEGIN
CHANGELOG_END
* Fix build issues
CHANGELOG_BEGIN
CHANGELOG_END
* Add dom library back to daml-json-types
CHANGELOG_BEGIN
CHANGELOG_END
* added a package.json to work with yarn workspaces
This adds a package.json files on top of our typescript libraries so
that we can develop locally via yarn workspaces. The package.json that
describes the bazel managed dependencies is moved into a subfolder.
CHANGELOG_BEGIN
CHANGELOG_END
* updated bazelignore
* SDK_VERSION -> SDKVERSION
* language: put sdk versions into package.json
The typescript library versions of our support libraries are now given
by the sdk version.
CHANGELOG_BEGIN
CHANGELOG_END
* removed local field
* better placeholders
* consistent SDK_VERSION
* sed sdkversion in test script
Since the JSON API supports `fetchByKey` and `exerciseByKey` now, we don't
need the `pseudo*` versions of these functions anymore.
CHANGELOG_BEGIN
CHANGELOG_END
* Define eslint test case
* Define eslint_test macro
* Eslint test for daml-ledger-fetch
* Fix lint issues
* Deduplicate eslint on language-support/ts
* Use eslint --parser-options to define tsconfigRootDiro
Allows to avoid hard-coding the Bazel package path into the package.json
file. Instead derives the package root from the location of the
tsconfig.json file.
* document eslint
* pass kwargs to _eslint_test
* Add copyright header
CHANGELOG_BEGIN
CHANGELOG_END
* Exclude daml-json-types tests from lint
Otherwise eslint complains about mismatching config in `tsconfig.json`.
The test files are excluded from the project in `tsconfig.json`.
* Fix linter warning in daml-json-types
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
Currently, the validator for `Optional (Optional _)` would allow the value
`[null]`, which is not the JSON encoding for any value of this type.
This PR fixes the issue. Since it detects companion objects for the
`Optional` by means of their JavaScript class, we can also drop the
`isOptional` property from the `Serializable` interface.
CHANGELOG_BEGIN
CHANGELOG_END
* language: bazel rules for daml-json-types/daml-ledger-fetch
This moves the daml-json-types/daml-ledger-fetch libraries out of the
tests directory and builds them with bazel. We'll rename these libraries
in a follow up PR.
CHANGELOG_BEGIN
CHANGELOG_END
* Update deps.bzl
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* updated package.json
* rename nodejs patch
* update yarn.lock
* update @bazel/bazel dependency
* wrong typescript version in toplevel package.json
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* daml2ts: Isolate tests better
Currently, the `sh_test` for `daml2ts` copies too many files into the
temporary directory where the test is run. This can cause issues with
stale files from development experiments being copied into the test.
This PR addresses the issue by exluding some directories which only
contain generated files.
CHANGELOG_BEGIN
CHANGELOG_END
* Exclude `node_modules` from buildifier check
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* First cut at generalized type
changelog_begin
changelog_end
* Update code gen and test
Co-authored-by: Shayne Fletcher <shayne.fletcher@digitalasset.com>
* Implement exercise by key
ExerciseCommand got a new required element: `reference` of polymorphic type.
* Add test case: exercise Archive by contractKey
* Add test case for ExerciseCommand JSON protocol
* flatten contract reference in ExerciseCommand JSON protocol
* formatting
* Update exercise by key
* Update documentation
CHANGELOG_BEGIN
- [JSON API - Experimental] Support Exercise by Key. See #4009.
CHANGELOG_END
* Address code review comments
* in query argument, rename %templates to templateIds, and nest query under 'query' field
CHANGELOG_BEGIN
- [JSON API - Experimental] In 'search' endpoint arguments, %templates is now templateIds.
Additionally, all contract query fields must occur under 'query'.
See `issue #3450 <https://github.com/digital-asset/daml/issues/3450>`__.
CHANGELOG_END
* fix other old query format usages
* bindings-rxjava: Fix warnings and tidy up code in BotTest.
* bindings-rxjava: Complete BotTest.TestFlowable properly.
This avoids a stack trace in the logs.
* bindings-rxjava: Use `should have size` in BotTest.
Better error messages when it fails.
* bindings-rxjava: Use `Eventually` to wait, rather than `Thread.sleep`.
This should hopefully fix the flakiness in this test.
* CHANGELOG_BEGIN
Type-check type synonyms.
CHANGELOG_END
* placate HLint
* comments
* Add an example that requires the check in kindOf
* check types containing syn-apps are well formed even when there is no expression of that type
* show type mismatch error after synonyms are expanded
* typeOf calls expandTypeSynonyms; track vars bound by TForall during expansion
* test interaction of syn-expansion and free-vars; add one bigger testcase
* extend bigger example with pointed typeclass, having functor as a super class
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* test cases: domain.TemplateId JSON serialization to JsString
* JSON protocol updated
* Fixing json-api test cases
* test cases: domain.TemplateId JSON serialization to JsString
* JSON protocol updated
* Fixing json-api test cases
* Adapt daml2ts and support libraries
* Update documentation
CHANGELOG_BEGIN
[JSON API - Experimental]
- Use JSON string to encode template IDs. Use colon (``:``) to separate parts of the ID.
The request format, with optional package ID:
- "<module>:<entity>"
- "<package ID>:<module>:<entity>"
The response always contains fully qualified template ID in the format:
- "<package ID>:<module>:<entity>"
See #3647.
CHANGELOG_END
* Minor documentation formatting changes.
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Currently, the generated decoder for, say, `Either` looks like
```ts
() => jtv.oneOf(
jtv.object<Either<a, b>>(...),
jtv.object<Either<a, b>>(...),
)
```
After this PR, the generated code will look like
```ts
() => jtv.oneOf<Either<a, b>>(
jtv.object(...),
jtv.object(...),
)
```
That saves us a few type annotations but nothing major.
CHANGELOG_BEGIN
CHANGELOG_END
* daml2ts: Support lookupByKey
We'll add `fetchByKey` and `exerciseByKey` in a separate PR. We'll remove
the `pseudo*` methods in another PR after that one.
CHANGELOG_BEGIN
CHANGELOG_END
* daml2ts: make `lookupByKey` inaccessible for templates without key
* Aligning DB contract table with domain.ActiveContract class
adding key,signatories, observers and agreement_text to DB contract table
removing witnessParties
Reading signatories and observers from contracts table
Updating doc, removing witnessParties
Address code review comments, thanks @S11001001
CHANGELOG_BEGIN
[JSON API - Experimental]
- Align ``contract`` table with ``domain.ActiveContract`` class.
The database schema has changed, if using ``--query-store-jdbc-config``,
you must rebuild the database by adding ``,createSchema=true``. See #3754.
- ``witnessParties`` field is removed from all JSON responses.
CHANGELOG_END
* Fix TypeScript domain models
remove witnessParties and workflowId fields. workflowId has be removed
from JSON output a while ago.
So far, we've used the version of damlc installed with the SDK on the
devs machine. This lead to hard to debug version mismatches. This PR
changes `watch-damlc.sh` to use `bazel run //:damlc` instead.
CHANGELOG_BEGIN
CHANGELOG_END
* Move most of the remaining serializable types to stable LF packages
The only serializable types left in DAML stdlib after this PR are the
following:
- DA.Upgrade:MetaEquiv
- DA.Random:Minstd
- DA.Next.Set:Set
- DA.Next.Map:Map
- DA.Generics:MetaSel0
- DA.Generics:MetaData0
- DA.Generics:DecidedStrictness
- DA.Generics:SourceStrictness
- DA.Generics:SourceUnpackedness
- DA.Generics:Associativity
- DA.Generics:Infix0
- DA.Generics:Fixity
- DA.Generics:K1
- DA.Generics:Par1
- DA.Generics:U1
- DA.Internal.Prelude:Optional
Ignoring the Generics stuff which isn’t very urgent imho and the
Upgrade stuff which is probably going to change significantly anyway,
this leaves us with the weird Random module, the wrappers around
TextMap which will go away anyway and DA.Internal.Prelude:Optional
which shouldn’t exist in the first place (I’ll address that in a
separate PR).
CHANGELOG_BEGIN
- [DAML Compiler] Move more types from daml-stdlib to standalone LF
packages. The module names for the types have also changed
slightly. This only matters over the Ledger API when you specify the
module name explicitly. In DAML you should continue to use the
existing module names.
- The types from ``DA.Semigroup` are now in a separate package under
``DA.Semigroup.Types``
- The types from ``DA.Monoid` are now in a separate package under
``DA.Monoid.Types``
- The types from ``DA.Time` are now in a separate package under
``DA.Time.Types``
- The types from ``DA.Validation` are now in a separate package
under ``DA.Validation.Types``
- The types from ``DA.Logic` are now in a separate package under
``DA.Logic.Types``
- The types from `DA.Date` are now in a separate package under
`DA.Date.Types`.
- The `Down` type from `DA.Internal.Prelude` is now in a separate
package under `DA.Internal.Down`.
CHANGELOG_end
* Fix serializability of RelTime
* fix daml-docs
* Fix tests
* Change variant json encoding,
adding integration test
* Add DamlLfTypeLookup dependencies
* Add MetadataReader
* Add test WIP
* Add serialize test cases
* Add serialize test cases, WIP
* Test for variant encoding decoding
* Solving merge conflicts
* Updating roundtrip test
* Minor cleanup
* Addressing code review comments
Add JsonVariant custom matcher
* Update specification
* Update link
* Add test case, WIP
* Add proper template key resolution
* Got rid of choice record ID resolution, resolving choice type and key type
* Fixing logging
* Add Contract Key decoding tests
* cleanup
* cleanup
* Update JSON variant encoding tests
* Add more contract key JSON decoding tests
* Fix variant JSON encoding
* Change value predicate to support new variant encoding
* Change value predicate to support new variant encoding
* Add lookup by contract key test case
where contract key contains variant and record
Add `requiredResource` to bazel utils
CHANGELOG_BEGIN
- [JSON API - Experimental] Change variant JSON encoding. The new format is ``{ tag: data-constructor, value: argument }``.
For example, if we have: ``data Foo = Bar Int | Baz``, these are all valid JSON encodings for values of type Foo:
- ``{"tag": "Bar", "value": 42}``
- ``{"tag": "Baz", "value": {}}``
See #3622
- [JSON API - Experimental] Fix ``/contracts/lookup` find by contract key.
- [JSON API - Experimental] Fix ``/command/exercise`` to support any LF type as a choice argument.
See #3390
CHANGELOG_END
* minor cleanup
* Fix copy/paste
* Renaming
* Got rid of DAML LF identifier resolution
resolving DAML LF Type based on command type
* Address code review comments, thanks @S11001001
* Address code review comments, thanks @S11001001
Do not include any error handling here; this partial function should
only match the successful case, JsonVariant.
* Address code review comments, thanks @S11001001
comment
* Address code review comments, thanks @S11001001
using `JsonVariant` for variant encoding/decoding
* Address code review comments, thanks @S11001001
replace `find` and `map` chain with collectFirst
* Update docs/source/json-api/lf-value-specification.rst
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <scompall@nocandysw.com>
* sandbox: Create a monadic `ResourceOwner` to manage resources.
* sandbox: Rewrite `ResourceOwner` to be async.
* sandbox: Make sure failed resources are closed immediately.
* sandbox: Better naming in `Open`.
* sandbox: Rename `Open` to `Resource`, and open/close to acquire/release.
* sandbox: Convert `() => AutoCloseable` into `ResourceOwner`.
* sandbox: Refactor the LedgerApiServer in terms of resources.
* sandbox: Explicitly convert `() => AutoCloseable` to `ResourceOwner`.
Explicit > Implicit, right?
* sandbox: Create helpers for converting things to ResourceOwners.
Because I tried to start using them and there was so much code being
written at once.
* sandbox: Simplify construction of JdbcLedgerDao.
* sandbox: Releasing resources should be idempotent.
In that we should only do it once.
* sandbox: Fix the ResetService by closing the API services _first_.
They need to be shut down before the gRPC server.
* sandbox: Don't try and shut down PostgreSQL twice in tests.
* sandbox: Actually run the assertions in ResourceOwnerSpec.
Facepalm.
* sandbox: Test `Resource.sequence` more rigorously.
* sandbox: Move the helpers around `Resource` into `Resource.apply`.
* sandbox: Convert LedgerApiServer resource owners to classes.
* sandbox: Make `ResourceOwner` a monad too, delegating to `Resource`.
* sandbox: Turn `LedgerApiServer` into a ResourceOwner.
* sandbox: Simplify the public signature of `Resource.apply`.
* sandbox: Use ResourceOwners to simplify DB resource management.
This is one hell of a change. Sorry.
* sandbox: Try not to nest `Await.result` calls.
Causes issues when running in a `DirectExecutionContext`.
* sandbox: Turn index subscriptions into resources.
* sandbox: Fix warnings in RecoveringIndexerSpec.
* sandbox: Always release before recovering the indexer.
* sandbox: Add `flatten` and `transformWith` to `Resource`.
* sandbox: If releasing twice in parallel, the second should wait.
* sandbox: If the indexer recovers, clean up the old subscription.
* sandbox: Convert StandaloneIndexerServer into a resource owner.
* sandbox: Convert StandaloneApiServer into a resource owner.
* reference-v2: Rewrite ReferenceServer in terms of resources.
CHANGELOG_BEGIN
- [Reference v2] On an exception, shut down everything and crash.
Previously, the server would stay in a half-running state.
CHANGELOG_END
* sandbox: Rewrite SandboxServer in terms of resources.
* sandbox: Write the port file in a Future.
* sandbox: JdbcIndexer no longer needs to manage the actorSystem.
* sandbox: Shut down the LedgerApiServer when closing the Sandbox.
* sandbox: Rename `Resource.pure` to `Resource.successful`.
* sandbox: Rename `Resource.sequence_` to `sequenceIgnoringValues`.
* sandbox: Delete `CloseableResource`.
It's only used in once place. Just inline it.
* sandbox: `LedgerDao` no longer needs to be closeable.
* sandbox: Delete implicit materializers where they're not used.
* http-json: Wait for the Sandbox to start in tests.
* sandbox: Convert `scheduleHeartbeats` into a ResourceOwner.
* reference-v2: Explain why we steal ownership of the actor system.
* sandbox: Document why we only release resources once.
* sandbox: Add clues to ResourceOwnerSpec.
* http-json: Fix HttpServiceTestFixture to pass auth service through.
* codegen-sample-app: In ScalaCodeGenIT, wait for the server to start.
* Upgrade to Akka 2.6.1, akka-http 10.1.11 and Scala 2.12.10
Akka 2.6.1 Upgrade Changes
- Materializer in place of ActorMaterializer
- Source.future instead of Source.fromFuture
- The Scheduler.schedule method has been deprecated in favor of selecting scheduleWithFixedDelay or scheduleAtFixedRate
- onDownstreamFinish(cause: Throwable)
- ActorAttributes.supervisionStrategy(...) in place of ActorMaterializerSettings.withSupervisionStrategy
See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html
* Akka 2.6.1 Upgrade Changes
- onDownstreamFinish(cause: Throwable)
See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html
* code review: remove unnecessary supervision strategy
* Move Any wrappers and Archive to stable packages
There are no actual API or functionality changes in this PR but the
logic for locating the stable packages has slightly changed since the
Any wrappers package only makes sense for LF 1.7. To address this, we
simply filter out stable packages for newer LF versions since it
doesn’t make sense to depend on those anyway.
CHANGELOG_BEGIN
- [DAML Compiler] Move ``Archive`` type to a separate DALF.
CHANGELOG_END
* More comments
* Fix java codegen tests
* fix more tests
* Force cache reset on Windows
* Revert "Force cache reset on Windows"
This reverts commit 9f2b7d70b2.
What was called `Contract` until now is actually a create event. Hence the
renaming. We'll most likely get a `Contract` type in DAML as well and this
renaming will avoid a name conflict. Also, create events are not part of
DAML-LF values and hence not generated by `daml2ts`. Thus, there's not need
to have them in `daml-json-types`. Instead, they should be in a future
`daml-json-api-types` package which captures the types used by the JSON API.
* language: daml2ts: support for enum types
This adds support for enum types to daml2ts. daml-lf enums are converted
to typescript enums and a decoder defined in the same namespace.
* static serializable check for enums and better tests
* sandbox: Move SandboxServer's helper classes to the companion object.
And make them `private`, `final`, and non-`case`.
* sandbox: Drop SandboxServer.apply and just call the constructor.
* sandbox: Move the SandboxServer#SandboxState class into the object.
Leave the `resetAndRestartServer` method behind, though.
In the code generated by `daml2ts`, every template companion object lists
all its choices and every choice has a pointer back to the companion object
of its template. Thus, there's a knot to tie.
So far, we initialized the choices as `undefined` and later mutated them to
point to the template companion object. This feels kind of hacky,
particularly since we end up with cyclic values.
This PR pushes the pointer from the choice back to the template companion
object behind a lambda. This makes the hack unnecessary and removes the
cyclic values.
* Add package_entries table
* Change PublicPackageUpload event to cover list of packages.
Add PublicPackageUploadRejected.
* Produce new package update events in KeyValueConsumption
* Update signature of uploadPackages
* Cleanup InMemoryKVParticipantState. Add submissionId to uploadPackages.
* Fix up InMemoryKVParticipantStateIT
* Initial ledger dao changes for package entries
Drop the participant_id as we never expect to see
entries of other participants. This should be done
for party_entries as well.
* Drop UploadPackagesResult
* Implement getPackageEntries and refactor callers
* Add maxRecordTime to uploadPackages
* First cut at updating ApiPackageManagementService
* Update tests, wire through the packageEntries
* Don't extend IndexPackagesService in InMemoryPackageStore
It does not implement the full interface and it isn't used
directly as one anyway.
* Drop maximum_record_time from package_management_service
Adding maximum record times touches the whole stack. Leaving
this change to another PR.
* Wire through the removal of maximum_record_time.
And remove dead code from InMemoryKVParticipantState
* Remove checking for duplicate package uploads
This aligns with the behaviour of WriteService.
* Reformat
* Fix PackageManagementService after adding of submission_id to the service
CHANGELOG_BEGIN
- [Sandbox] Restore 0.13.38 logging behaviour.
- [Navigator] Restore 0.13.38 logging behaviour.
- [Extractor] Restore 0.13.38 logging behaviour.
- [Internals] As of 0.13.39, we merged a number of internal JAR files in
the SDK tarball to reduce its size. These jars used to be standalone
JARs you could invoke as e.g. ``java -jar sandbox.jar <args>``. As a
result of merging the jars, they lost their individual ``logback.xml``
configuration file. Although running the jars directly was (and is
still) not supported, note that you can now achieve the same behaviour
with e.g. ``java -Dlogback.configurationFile=sandbox-logback.xml -jar
daml-sdk.jar sandbox <args>``.
CHANGELOG_END
* scala codegen: add GenMap support
* scala-codegen: use InsertOrdMap as underlying of Generic Map binding
* Address Stephen's comments
* make TextMap a strict subtype of immutable.Map to avoid overlap with GenMap
Otherwise, attempted GenMap encodings like
`Value.encode(InsertOrdMap("foo" -> "bar"))` (and any contract
containing such a structure) are either ambiguous or, worse, yield the
wrong result.
* ensure better return types for InsertOrdMap and similar wrappers' operations
* if is an expression
* adapt various primitive encodings to newtype TextMap
* easy cleanups of the TextMapApi
* proper alias for deprecated Map
* update deprecation releases
* Revert "proper alias for deprecated Map"
This reverts commit e85aa85b960c4bf5c4f9624896183ec6e2182bba.
* remove useless invisible deprecated notice
* add generic map tests for scala codegen
* please restart CI
* Adding choice result to the exercise response, WIP
* Adding choice result to the exercise response, WIP
* Refactoring towards #3390,
ExerciseCommand argument does not always have to be a Record, changing
typearg: JsObject -> JsValue
* Cleanup
* exercise-with-result endpoint
* todo
* Switching /commands/exercise to use SubmitAndWaitForTransactionTree,
populating archived and created from TransactionTree
* removing debug println
* Fixing tests
* Removing `/command/exercise-with-result` endpoint
this one returns only exercise result, `/command/exercise` now returns
exercise result and events
* Updating docs
* Updating docs
CHANGELOG_BEGIN
- [JSON API - Experimental] Expose exercise result. Changed the output
of the ``/command/exercise``. Note ``exerciseResult`` and ``contracts``
in ``{"status":200,"result":{"exerciseResult": ...,"contracts":[...]}``.
See #3314
CHANGELOG_END
* Move all datatypes out of daml-prim
This moves the remaining two modules DA.Types and GHC.Tuple to
separate LF packages with stable identifiers.
The only data types remaining are the ones for typeclasses which will
disappear once we move this to type synonyms.
CHANGELOG_BEGIN
- [DAML Compiler] The modules DA.Types and GHC.Tuple from daml-prim
have been moved to separate packages.
CHANGELOG_END
* Fix codegen tests
* Fix DarReader test
* Fix kvutils tests
* Fix jdbcdao tests
* Fix hs ledger bindings tests
* Add ledger and participant ID to claims
CHANGELOG_BEGIN
- [Ledger] AuthService implementations can now restrict the validity of access tokens to a single ledger or participant.
- [Sandbox] The sandbox JWT authentication now respects the ledgerId and participantId fields of the token payload.
CHANGELOG_END
* Add tests for ledger and participant in claims
* Address review comment
* Address review comment
* Fix tests
* Fix tests
* Ensure the access token is initialized when constructing a client
CHANGELOG_BEGIN
- [Java Client] Ensure the access token is initialized when using a
deprecated constructor.
CHANGELOG_END
* Improve phrasing and grammar
* daml2ts: Add E2E tests
* Attempt to fix failing test
* Don't try to delete the temp directory
* Cleanup block can't be empty
* Leave TMP_DIR before removing it
* RxJava Bindings: Allow bot to run on a scheduler
It seems that having many bots results in a some sort of deadlock
or blocking of data flow within the flowable network.
Adding some async boundaries to allow for concurrent processing
seems to help.
Fixes#2356
CHANGELOG_BEGIN
- [RxJava Bindings] Added a method to the ``Bot`` class allowing users to specify a ``Scheduler`` to use for running the bot. See `issue #2356 <https://github.com/digital-asset/daml/issues/2356>`__.
CHANGELOG_END
* Go easy, test!
* Fix compiler warnings in generated enums
* Fix warnings in generated equals method for parameterized types.
* Remove warning in equals for records without fields.
CHANGELOG_BEGIN
- [Java Bindings] Removed warnings in code emitted by the Java Codegen.
CHANGELOG_END
* fix compilation error in tests
* daml assistant expected auth token in Bearer format
* Daml assistant does no validation of the auth token before passing in on to the ledger.
* clarify code with newtype Token
This is a first step towards making sure that the package ids for
types defined in daml-prim and daml-stdlib don’t change. This PR
mostly adds all the necessary infrastructure for that and moves
GHC.Types and GHC.Prim to make sure it works.
Until data-dependencies are really solid and we have verified that we
no longer have performance issues with an increasing number of Haskell
packages, we still include the source files in daml-prim and then just
rewrite the references.
We will also need to add tests that these packages really have stable
ids but I’ll leave that for separate PRs since this doesn’t make that
much sense anyway until all of the types have moved to stable
packages.
CHANGELOG_BEGIN
- [DAML Compiler] The modules GHC.Prim and GHC.Types from daml-prim
have been moved to separate packages.
CHANGELOG_END
