As we strive for more inclusiveness, we are becoming less comfortable
with historically-charged terms being used in our everyday work.
This is targeted for merge on Dec 26, _after_ the necessary
corresponding changes at both the GitHub and Azure Pipelines levels.
CHANGELOG_BEGIN
- DAML Connect development is now conducted from the `main` branch,
rather than the `master` one. If you had any dependency on the
digital-asset/daml repository, you will need to update this parameter.
CHANGELOG_END
@sofiafaro-da is taking care of 1.8.0-snapshot.20201215.5907.0.a6ed34c5 (#8306), so they get pushed back to the end of the line.
Please do not merge this before #8306.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
@S11001001 is taking care of 1.8.0-snapshot.20201209.5848.0.e27026ee (#8209), so they get pushed back to the end of the line.
Please do not merge this before #8209.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Previously, each Protobuf source JAR was considered independent, and did
not require dependencies. This meant that the kvutils Protobuf sources
were published without the associated DAML-LF sources, making them
useless.
This change adds the source JARs as runtime dependencies to ensure that
when publishing a Protobuf source JAR, the release process will fail if
the dependencies are not also published.
CHANGELOG_BEGIN
- [Ledger API] We now publish Protobuf sources in JARs for DAML-LF
types to Maven Central, with the group "com.daml". The artifact names
follow the format "<name>_proto_jar".
CHANGELOG_END
@stefanobaghino-da is taking care of 1.8.0-snapshot.20201201.5776.0.4b91f2a6 (#8126), so they get pushed back to the end of the line.
Please do not merge this before #8126.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
NPM specialcases the latest tag which is used by default and defaults
to that in things like `npm add`. Given that we don’t want people to
rely on snapshots, tagging them as next (common convention on npm)
seems like a good idea.
I have no idea how to test this reasonably so the next snapshot will
have to tell if it works or not.
changelog_begin
changelog_end
CHANGELOG_BEGIN
- [Integration Kit] The kvutils Protobuf definition is now published to
Maven Central in a JAR, under the group "com.daml", with the artifact
name "participant-state-kvutils-proto".
CHANGELOG_END
We don't use them, and they are broken anyway, because Maven demands
Javadoc JARs and our "plain" jars don't provide them.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api: Use `proto_jars`.
CHANGELOG_BEGIN
- [Ledger API] The Scala JARs containing the gRPC definitions no longer
contain the *.proto files used to generate the ScalaPB-based classes.
CHANGELOG_END
* Create a source JAR for *.proto files in `proto_jars`.
* ledger-api: Publish the protobuf sources as "ledger-api-proto".
CHANGELOG_BEGIN
- [Ledger API] The *.proto files containing the gRPC definitions are now
provided by a new Maven Central artifact, with the group "com.daml"
and the artifact name "ledger-api-proto".
CHANGELOG_END
* release: We don't need the "main-jar" option.
* Bazel: Proto JARs will always have a Maven artifact suffix.
* Bazel: Simplify Protobuf source file TAR and JAR targets.
* Bazel: Extract out Protobuf functions.
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.
* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.
* Bazel: Factor out Java/Scala protobuf class generation into a helper.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/archive: Use `proto_jars`.
* Bazel: Remove the visibility modifier from `proto_jars`.
It's too confusing. Just make everything public.
* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.
* Bazel: Add comments to the various parts of `proto_jars`.
* daml-assistant: Do unpleasant things with `location` in Bazel.
@cocreature is taking care of 1.8.0-snapshot.20201124.5709.0.dabd55d0 (#8055), so they get pushed back to the end of the line.
Please do not merge this before #8055.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
@garyverhaegen-da is taking care of 1.8.0-snapshot.20201117.5661.0.76fae40c (#7994), so they get pushed back to the end of the line.
Please do not merge this before #7994.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
@SamirTalwar-DA is taking care of 1.7.0-snapshot.20201110.5615.0.b35c9fcb (#7937), so they get pushed back to the end of the line.
Please do not merge this before #7937.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Document new integrity checker options
* Remove v2 from the bazel target in README
* Refactor integrity checker into published "tools" lib + binary
CHANGELOG_BEGIN
CHANGELOG_END
* Refine split
* Release tools lib
* Generalize IntegrityChecker.run
* Use glob for test source definition
* Widen glob for tools-tests to all (future) tool tests, not just integritycheck
* Simplify test source globs
* Simplify tools lib source glob
* Fix build
Although I _really like_ making releases, particularly the testing on Windows, I think it's only fair if I don't have all of them for myself but share them with the team.
CHANGELOG_BEGIN
CHANGELOG_END
@aherrmann-da is taking care of 1.7.0-snapshot.20201027.5530.0.bdbf8977 (#7822), so they get pushed back to the end of the line.
Please do not merge this before #7822.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
@nickchapman-da is taking care of 1.7.0-snapshot.20201020.5481.0.03a03957 (#7754), so they get pushed back to the end of the line.
Please do not merge this before #7754.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* resources: Move builders into //ledger/ledger-resources.
Keep the actual constructors in a trait, but instantiate it when working
with ledger code.
This allows us to later introduce an extra "context" type parameter to
ResourceOwner.
* resources-akka: Move the builders in to //ledger/ledger-resources.
* resources: Introduce an abstract `Context` parameter for owners.
This replaces the concrete `ExecutionContext`. While it _can_ be an
execution context, it really doesn't matter as long as we can get at one
somehow.
This is being introduced so we can wrap the context in a container,
either for type tagging or to include extra information.
Because our current context _is_ `ExecutionContext`, and an implicit is
provided to extract it, we can end up with two ways to get the same
value. We use shadowing to prevent this. This problem should go away in
the near future when a new context type is added.
CHANGELOG_BEGIN
- [Integration Kit] The `ResourceOwner` type is now parameterized by a
`Context`, which is filled in by the corresponding `Context` class in
the _ledger-resources_ dependency. This allows us to pass extra
information through resource acquisition.
CHANGELOG_END
* ledger-resources: Move `ResourceOwner` here from `resources`.
* ledger-resources: Remove dependencies from outside //ledger.
* ledger-resource: Wrap the acquisition execution context in `Context`.
So we can add a logging context to it.
* resources: Pass the Context, not the ExecutionContext, to Resource.
* Avoid importing `HasExecutionContext`.
* ledger-resources: Publish to Maven Central.
* resources: Make the small changes suggested by @stefanobaghino-da.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-resources: Pull out a trait for test resource contexts.
Saves a few lines of code.
* Restore some imports that were accidentally wildcarded.
* resources: Replace an `implicit def` with a couple of imports.
* participant-integration-api: Simplify the JdbcLedgerDaoBackend tests.
Try and use the right execution context where possible.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
@robin-da is taking care of 1.7.0-snapshot.20201013.5418.0.bda13392 (#7676), so they get pushed back to the end of the line.
Please do not merge this before #7676.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Update release instructions to follow newest GSG
We've lately integrated the codegens into `daml start`, which made the GSG significantly shorter in turn. Reflect these changes in the release instructions. Also reword some parts where I found the instructions ambiguous on first reading.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions
CHANGELOG_BEGIN
CHANGELOG_END
@hurryabit is taking care of 1.6.0-snapshot.20201006.5358.0.0c1cadcf (#7588), so they get pushed back to the end of the line.
Please do not merge this before #7588.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* conservatively move daml-script, trigger SValue interpreters to common library
* introduce expect and JavaList pattern for converters
* clean up trigger Converter Command interpretation
* add Church Free monad
* add an action language for trigger updates
* add expectE to remove some of the joins
* convert more of the converters to expect
* tool for unrolling Free/Roll
* split handleStepResult up and clean up its pattern
* handleStepFreeResult to interpret TriggerF
* replace Free Church with Pure/Roll free from Script
* newtype for ActionTrigger
* replace update in low-level Trigger with Free TriggerF
* submit one Commands at a time
* boolean blindness strikes again
* log missed TriggerF steps
* comment actual Submit contents
* match #7501 fromPureSExpr sig change in 00b80b8ea3
* avoid using forwardPort in runTrigger
* push State back into DAML, so it can be excluded from the action list
* push Message back into DAML, unifying the action language for initialState and update
* bringing TriggerF into initial state
* really add TriggerF into initial state, with all ports, tested
* add ActionTrigger class, express initialState in its terms
* add all TriggerF actions to existing TriggerA
* Trigger.rule will no longer have Time argument
* rename getS, setS to get, put, matching C.M.T.State from transformers
* make high-level Rule evaluate to the underlying TriggerF sequence
* Assert's testRule doesn't have a transform yet
* move DamlTuple2 to common converter library
- suggested by @cocreature; thanks
* combine the two Frees, provide from Script
* remove time argument from integration tests
CHANGELOG_BEGIN
- [Triggers] The ``Time`` argument was removed from the trigger rule function; instead, it
can be fetched within the ``TriggerA`` ``do`` block by ``getTime``, as with ``Update``
and ``Scenario``. The ``LowLevel`` trigger interface has been redesigned; such triggers
need to be rewritten or ported to high-level triggers.
See `issue #7456 <https://github.com/digital-asset/daml/pull/7456>`_.
CHANGELOG_END
* add trigger rule simulator to support Assert module
* missed new Free module
- left in script per @cocreature
* remove retract as we ended up using foldFree for that purpose instead
- suggested by @cocreature; thanks
* throw ConverterException instead of RuntimeException
- suggested by @cocreature; thanks
* remove Time argument from coin-upgrade-trigger
* port trigger service tests
* port trigger scenario test
* put TriggerSetup and TriggerRule into LowLevel.Trigger instead of unboxed Free
- suggested by @cocreature; thanks
* remove Time argument from trigger compatibility test
* submit commands as soon as each `emitCommands` is sequenced
- we still collect a list, but only for tracking commandsInFlight
* filter out compatibility tests for triggers before now
* remove commented imports, libraries from new shared converter
* make the TriggerF interpreter tail-recursive
* remove unused compatibility trait
* add back new state logging
* remove refactoring comment
* rewrite some LowLevel initialStates in do
* hide Daml.Script.Free from docs
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* remove forwardPortInitialState
- suggested by @cocreature; thanks
* manually port low-level updates
- suggested by @cocreature; thanks
* remove forwardPort
- suggested by @cocreature; thanks
* fail faster on unrecognized TriggerF
- suggested by @cocreature; thanks
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
@sofiafaro-da is taking care of 1.6.0-snapshot.20200929.5303.0.f1e58206 (#7522), so they get pushed back to the end of the line.
Please do not merge this before #7522.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* concurrent: Tag DirectExecutionContext.
1. Tag `DirectExecutionContext` as `ExecutionContext[Nothing]`, thereby
stating that it works for any tagged `Future`.
2. Move `DirectExecutionContext` to the _libs-scala/concurrent_
library, as it requires it and it's tiny.
CHANGELOG_BEGIN
CHANGELOG_END
* concurrent: Fix the privacy of `DirectExecutionContextInternal`.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
With the introduction of the standalone JAR, we cannot rely on the
assistant anymore to pass the default logback config. Users can still
override the logback config with `-Dlogback.configurationFile` if they
need something else but this provides a more sensible default logging
config than seeing a ton of debug logs from netty.
changelog_begin
changelog_end
- Followup to #7338, adjusting release test instructions for changes to the Java
quickstart, mismatches found testing #7462. Compare to
docs/source/app-dev/bindings-java/quickstart/template-root/daml/Main.daml for line #
references.
CHANGELOG_BEGIN
CHANGELOG_END
@S11001001 is taking care of 1.6.0-snapshot.20200922.5258.0.cd4a06db (#7462), so they get pushed back to the end of the line.
Please do not merge this before #7462.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* add phantom-tagged ExecutionContext and Future to scala-utils concurrent package
* many new operations for Futures
* Future, ExecutionContext combinators from porting ledger-on-sql
- picked from 546b84ab9cdf4de2d93ec5682bdee6cfd6b385f8
* move Future, ExecutionContext companions into normal package
* lots of new docs
* many new Future utilities
* working zipWith
* tests for ExecutionContext resolution, showing what will be picked under different scenarios
* even more tests for ExecutionContext resolution
* tests showing some well-typed and ill-typed Future combinator usage
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* missed scalafmt
* one more doc note
* split concurrent package to concurrent library
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Improve release instructions
- Add more notes on Remmina and how to use ad-hoc machines
- Fix Windows quickstart testing after change to template
changelog_begin
changelog_end
* Fix typo
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
@stefanobaghino-da is taking care of 1.6.0-snapshot.20200915.5208.0.09014dc6 (#7410), so they get pushed back to the end of the line.
Please do not merge this before #7410.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Please indulge my OCD.
And apologies to whoever was counting on these for their steganographed
[Whitespace] program.
[Whitespace]: https://esolangs.org/wiki/Whitespace
CHANGELOG_BEGIN
CHANGELOG_END
We reach the timeout on pretty much every release lately and we need to
retry to entire release process two or three times to get through. This
is us giving up on Maven, not Maven itself crashing, so I4d like to try
giving it more time.
CHANGELOG_BEGIN
CHANGELOG_END
@cocreature is taking care of 1.5.0-snapshot.20200908.5166.0.1623baec (#7348), so they get pushed back to the end of the line.
Please do not merge this before #7348.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
NPM doesn’t actually need this (at least it didn’t for me locally and
hopefully CI agrees) to pick up changes and it emits a very
scary-looking warning if you do pass it.
changelog_begin
changelog_end
@rohanjr is taking care of 1.5.0-snapshot.20200901.5116.0.4460cb5e (#7296), so they get pushed back to the end of the line.
Please do not merge this before #7296.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
@leo-da is taking care of 1.5.0-snapshot.20200825.5071.0.d33e130f (#7233), so they get pushed back to the end of the line.
Please do not merge this before #7233.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
@garyverhaegen-da is taking care of 1.5.0-snapshot.20200818.5027.0.1b33d374 (#7173), so they get pushed back to the end of the line.
Please do not merge this before #7173.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR attempts to add some automation around assigning release
management. The PR adds a file `release/rotation`; each week, the
updated CI cron job will:
- Open a PR for the new release [as current].
- Assign the first user in the file to that PR.
- Add the Standard-Change label to the PR.
- Start the build for that PR [as current].
- Open a new PR that rotates the `release/rotate` file, i.e. pushes back
the first line to the end of the file.
This PR also adds mentions of the "release handler" (the first line of
`release/rotation`) to the various messages we send to Slack along the
release process.
The initial state of the `release/rotation` file has been created by
listing all the volunteers (Language team, Application Runtime team, as
well as @SamirTalwar-DA and @stefanobaghino-da) and piping the file
through `shuf`. (Then I put myself at the top so I can hopefully iron
out the issues with the first attempt.)
CHANGELOG_BEGIN
CHANGELOG_END
* Factor out tar/gzip reproducibility flags
* use mktgz in package-app
* Bazel managed tar/gzip
* Remove quiet = True
As stated in the comment this is no longer required with Bazel >= 3.0.
* Build package-app as a sh_binary
This way Bazel will manage the runtime dependencies tar, gzip, mktgz,
and patchelf.
package-app.sh changes directory so it needs to make sure that all paths
are absolute and that the runfiles tree/manifest location is forwarded
to programs called by package-app.sh.
* Avoid file path too long errors
* Fix readlink -f on MacOS
* Document abspath
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Moving `Statements.discard` from //ledger-server/http-json into //libs-scala/scala-utils
changelog_begin
changelog_end
* Add new module to the published artifacts
* `com.daml.scalautil` instead of `com.daml.scala.util`
@S11001001: That's because if this is in classpath and you import com.daml._,
you have a different scala in scope than the one you expect.
* Extend `daml new` to accept template as an option
The two positional arguments keep confusing users so this PR changes
things to allow the template to be passed via `--template`. Using a
positional argument still works so this is not breaking.
I’ve updated all docs to use the less confusing syntax.
changelog_begin
- [DAML Assistant] You can now use ``daml new project-name
--template=template-name`` instead of ``daml new project-name
template-name``. The old CLI syntax continues to be supported.
changelog_end
* Update docs/source/getting-started/index.rst
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Move public code into daml-integration-api
CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
This was never intentional, nobody even knew that this was possible
and we have an alternative, documented way of getting this via github
releases.
To avoid introducing this issue again, I’ve removed non-jar artifact
types from the Maven upload script.
fixes#448
changelog_begin
changelog_end
* Add `platform-version` field to `daml.yaml`
This PR adds the `platform-version` field to `daml.yaml`. Based on the
approach agreed upon in #6558, the logic for this all sits in
`daml-helper` and there are no changes to the assistant.
The details of how the logic work are in a comment so I’m not going to
repeat them here but the commands that are affected are:
- `daml sandbox`
- `daml sandbox-classic`
- `daml json-api`
- `daml start` (but only for sandbox and the JSON API, not for
Navigator or anything else)
For tests, I’ve added a test to the compat workspace that installs two
SDKs simultaneously and tries out various combinations and verifies
that we get the correct version. Open to other ideas for testing this
but that seemed like the most sensible option that actually tests what
we run.
changelog_begin
- [DAML Assistant] You can now specify the version of Sandbox and the
JSON API independently of your SDK version by setting
``platform-version`` in your ``daml.yaml``. This is useful if you
are deploying to a ledger that is running components from a
different SDK version. See
https://docs.daml.com/tools/assistant.html#project-config-file-daml-yaml
for details.
changelog_end
* Run platform-version tests
changelog_begin
changelog_end
* Fix tag globbing
changelog_begin
changelog_end
* fmt
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* Try to fix env vars
changelog_begin
changelog_end
* Remove hardcoded references to 1.2.0
changelog_begin
changelog_end
* Rephrase doc comment
changelog_begin
changelog_end
* get things to compile
changelog_begin
changelog_end
* maybe fix things for realz
changelog_begin
changelog_end
* Remove debugging output
changelog_begin
changelog_end
* Get angry at windows
changelog_begin
changelog_end
* why is windows
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
Buildifier now comes with a handy attachment to catch single `\`
characters inside strings and replace them with `\\` if the escape
sequence is invalid. Skylark/Python will do this at runtime anyway; this
just makes it clearer what the actual behavior is.
I needed to change `\` characters at the end of lines to `\\` manually
in order to stop Buildifier from simply concatenating the lines
together. Everything else was automatic.
CHANGELOG_BEGIN
CHANGELOG_END
* Address https://github.com/digital-asset/daml/pull/6507#discussion_r446113575
* drop blindinginfo.proto
changelog_begin
changelog_end
* drop BlindingCoder
* Remove blindinginfo Protobuf definition JAR
changelog_begin
[DAML-LF] The blindinginfo Protobuf definition JAR, which was previously unused, has been pulled from the artifacts released on Maven
changelog_end
Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
* LF: rename library transaction-scalacheck to transaction-test-lib
CHANGELOG_BEGIN
CHANGELOG_END
* move files in com/daml
* missing change in release/artifacts.yaml
* remove 'com/dam' from the path
* Add release checklist to release instructions
This includes a couple of checks that internal projects have been
tested on the release candidate.
changelog_begin
changelog_end
* Update release/RELEASE.md
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* release: Compute missing deps by using one Bazel query, not many.
* release: Publish all JARs to the local Maven repository at once.
This is way, way faster than running `mvn install:install-file` in a
loop.
It works by creating a POM that instructs the `install:install-file`
plugin command to install the JARs in roughly the same way as before,
and then running `mvn initialize` once.
CHANGELOG_BEGIN
CHANGELOG_END
* release: Make the `Maven` file easier to read with less nesting.
And also more nesting.
* release: Always validate the Maven artifacts, as before.
* release: Explain `maybeMissingDeps` a little better.
* release: Delete the comment on excluding scenario and script services.
* release: Use `withCreateProcess` when calling `mvn initialize`.
* release: Build the TypeScript packages in one `bazel` call.
* release: Remove an unncessary `liftIO`.
This small PR makes a few QoL improvements to the release.sh script:
1. The snapshot command will now work for any commit. Previously, it
would refuse to print the snapshot suffix for commits that were not
ancestors of the `master` branch. The new version will print a
warning if the commit does not seem to be part of a release branch,
but will still print the result.
2. On checking the LATEST file, the script will now print a slightly
more useful error message if the file format is not valid.
3. The snapshot command will now print the entire line to be added into
the LATEST file, rather than just the version suffix.
CHANGELOG_BEGIN
CHANGELOG_END
The `gawk`-based method currently referenced does not work in a
multi-line `LATEST` world. I don't want to scare people with the command
that would work here.
CHANGELOG_BEGIN
CHANGELOG_END
* equalz Scalatest matcher in new daml-lf/scalatest-tools library
* equalz typing tests
* a 'should' replacing design
* a 'MatcherFactory1' design
- this fails because the TC parameter should be a type member to avoid
scala/bug#5075 but it is not
* MatcherFactory1 with chained Lub+Equal typeclass
- requires partial-unification at point of use, which is not great
* LubEqual's extra tparam is probably unneeded
* better LtEqual
* demonstrate that HK LubEqual's resolve with DMT should + MatcherFactory
* remove unneeded 3rd param from LubEqual, again
* update dependency specs and license headers
* allow use with should, shouldNot in some cases, preserving the shouldx/shouldNotx alternatives
* move Equalz to libs-scala/scalatest-utils
* rename bzl targets and place in com.daml.scalatest package
* add scalatest-utils to release
* move *SpecCheckLaws, Unnatural to scalatest-utils
* missed scalacheck dep in scalatest-utils
* downstreams of *SpecCheckLaws now get them from scalatest-utils
* test equal-types case as well
* update LF documentation
CHANGELOG_BEGIN
CHANGELOG_END
* whitespace error
trigger all releases from master
The 1.1.0 release went wrong and we had to trash it and release 1.1.1
instead. This is an attempt at identifying and correcting the root
cause behind that incident.
To understand the situation, we need to know how releases worked before
1.0. We had a one-line file called `LATEST` that specifies the git SHA and
version tag for the latest release. A change to that file triggered a
release with the specified release tag, built from the source tree of
the specified commit. The `LATEST` file looked something like:
```
f050da78c9 1.0.0-snapshot.20200411.3905.0.f050da78
```
To mark a release as stable, we would change it to look like this:
```
f050da78c9 1.0.0
```
i.e. simply drop the `-snapshot...` suffix. Even though the commit (and
thus the entire source tree we build from) is the same, we would need to
rebuild almost all of our release artifacts, as they embed the version
tag in various places and ways. That worked well as long as we could
assume we were doing trunk-based development, i.e. all releases would
always come from the same (`master`) branch.
When we released 1.0, and started work on 1.1, we had a few bug reports
for 1.0 that we decided should be resolved in a point release. We
decided that the best way to handle that would be to have a branch
starting on the release commit for 1.0, and then backport patches from
`master` to that branch. We adapted our build process to also watch the
`release/1.0.x` branch and, in particular, trigger a new release build if
the `LATEST` file in that branch changed. That worked well.
The plan going forward was to keep doing regular snapshot releases from
the `master` branch, and create support, point releases ("patch" releases
in semver) from dedicated branches.
On April 30, we made a snapshot release as an RC for 1.1.0, by changing
the `LATEST` file in the `master` branch. That release was built on commit
681c862d. On May 6, we decided to take a new snapshot as the RC for
1.1.0; we changed `LATEST` in `master` to designate 7e448d81 as the new
latest release.
On May 11, we noticed an issue that broke our builds. Without going into
details, an external artifact we depend on had changed in incompatible
ways. After fixing that on `master`, we reasoned that this would also
break the build of the final 1.1.0 release if we just tried to build
7e448d81 again. But as the target release date was May 13, we did not
want to take a new snapshot after that fix, as that would have included
one more week of work in the release, and given us no time to test it.
So we did what we did for the 1.0 branch, as it had worked well: we
created a branch that forked from `master` at commit 7e448d81 and called
it `release/1.1.x`, then cherry-picked the one fix to our build process to
work around the broken download. When the time came to make the final
1.1.0 build on May 13, we naturally picked the `LATEST` file from the
`release/1.1.x` branch and dropped the `-snapshot...` suffix. Importantly,
we did not need to update the target commit to include the "broken
download" fix as, in the meantime, the internet had fixed itself, and we
thus reasoned we should go for the exact code of the RC rather than
include an unnecessary, albeit seemingly harmless, change.
Everything went well with the release process. Tests went well too. Then
we got a report that an application that worked against the latest RC
broke with the final 1.1.0. The issue was that we had built the wrong
commit: by branching off at the point of the _target_ commit for the
latest snapshot, we did not have the change to the `LATEST` file that
designated that commit as the target. So the `LATEST` file in
`release/1.1.x` was still pointing to 681c862d.
I believe the root cause for this issue is the fact that we have
scattered our release process over multiple branches, meaning there is
no linear history of what was released and we are relying on people
being able to mentally manage multiple timelines. Therefore, I propose
to fix our release process so this should not happen again by
linearizing the release process, i.e. getting back to a situation where
all releases are made from a single branch, `master`.
Because we do want to be able to release _for_ multiple release branches
(to provide backports and bugfixes), we still need some way to
accommodate that. Having a single `LATEST` file in the same format as
before would not really work well: keeping track of interleaved release
streams on a single file would not really be easier than keeping track
of multiple branches.
My proposed solution is to instead have a multiline LATEST file, so that
all the release branch "tips" can be observed at the same time, and, as
long as we take care to only advance one release branch at a time, we
can easily keep track of each of them. This is what this PR does.
This required a few changes to our release process. Most notably:
- Obviously, as this is the main point of this PR, the build process has
once again been restricted to only trigger new releases from the
`master` branch.
- As our CI machinery cannot easily be made to produce multiple releases
from a single build, the `check_for_release` step will only recognize
a commit as a release trigger if it changes a single line in the
`LATEST` file. This restriction comes in addition to the existing one
that a release commit is only allowed to change either just the
`LATEST` file or both the `LATEST` and
`docs/source/support/release-notes.rst` files.
- The docs publication process has been changed to update _all_
published versions to display the _latest_ release notes page. This
means that the release notes page will always show you all published
versions, regardless of which version of the documentation you're
looking at. This also means that interleaving release notes correctly on
that page is a manual exercise.
- As per the intention of the new process, the `LATEST` file has been
updated to contained all existing post-1.0 stable releases. It should
also include all existing snapshot releases should we have more than one
at a time (say, should we discover an issue with 1.1.1 that required us
to work on a 1.1.2).
- The `release.sh` script has been dramatically simplified as I felt it
was trying to do too much and porting its existing functionality to a
multi-line `LATEST` file would be too hard.
CHANGELOG_BEGIN
CHANGELOG_END
* Extract caching from participant-state as a library
This will be used to keep a cache of values to cut on LF translation cost when serving transactions.
changelog_begin
changelog_end
* Add dependency where missing
Currently, there are quite a few releases that are lacking the
Standard-Change label, even though they did publish artifacts. This
makes our SOC2-compliance tracking a bit harder. For the past two
months, I have manually added the label after-the-fact while preparing
the monthly compliance report, but that doesn't seem like a great
solution.
This PR changes the release process to be more optimistic: assume the
release is going to succeed by putting in the label immediately, and
then (optionally) removing it if the release fails.
Note that the label should only be removed in the rare case where the
release was merged into master but somehow did not produce any artifact.
This can only happen if the Linux build fails quite early, which as far
as I know only happened once over the past two months when we had the
release notes race condition.
CHANGELOG_BEGIN
CHANGELOG_END
* factor TlsConfiguration parser from extractor
* move TlsConfigurationParser to new library
* link extractor to ledger-service/cli-opts properly
* use TlsConfigurationCli in http-json, pass SslContext to ledger-client
* test TLS options as used in http-json
- the TLS config code is shared with extractor, where it is more fully
tested; we just do a sanity check here
* doc TLS options for http-json
CHANGELOG_BEGIN
- [JSON API] New ``--pem``, ``--crt``, ``--cacrt``, and ``--tls`` options
for securing the connection between JSON API server and ledger.
See `issue #2540 <https://github.com/digital-asset/daml/issues/2540>`__.
CHANGELOG_END
* TLS off in daml-script JSON API test
This PR updates two points in the release process:
1. The process for getting release notes for a stable release based on
my experience of doing that for the past 2 releases.
1. It adds testing for the getting started guide. I did not remove the
testing of `quickstart-java`. I do want to test scenarios and
navigator and a bit of VSCode so the only potentially redundant
point there is the `mvn compile` step but that verifies that Maven
artifacts have been published successfully so I think it is still
useful.
changelog_begin
changelog_end
* ledger/metrics: Move metric helpers to their own Bazel package.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use ledger/metrics.
* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.
Importing methods is harder than importing objects.
* metrics: Publish to Maven Central.
* Used `daml codegen java` instead of calling the codegen from maven
This should hopefully fix the issues with mismatched versions of
slf4j.
changelog_begin
changelog_end
* Move config to daml.yaml
* Remove alternative invocation via maven from docs
CHANGELOG_BEGIN
- Move daml2ts, bindings-ts and JSON API out of experimental section in docs
- Rename Experimental to Early Access in docs and assistant
- Reorganise the docs a little bit to de-emphasise the Ledger API
CHANGELOG_END
This PR removes the code for publishing to Bintray and updates the
documentation to point to github releases or Maven central.
I had to slightly change the docs formatting since trying to use
markup in code blocks results in a horrible layout and I don’t want to
fix that atm.
I’ve removed all artifacts that were only published to Bintray.
changelog_begin
changelog_end
* Use com.daml as groupId for all artifacts
CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END
* Add 2 additional maven related checks to the release binary
1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId
* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
* Make new sandbox the default.
changelog_begin
- [DAML SDK] The new sandbox is now the default that runs with ``daml sandbox`` and ``daml start``. The command ``daml sandbox-next`` has been removed. The old sandbox can be invoked via ``daml sandbox-classic`` and ``daml start --sandbox-classic=yes``.
changelog_end
* Update descriptions.
* Change it to a switch
* Change switch help
* Recapitalize
* Make SDK release tarball reproducible
Without these changes multiple factors contribute to a different SDK
release tarball on each rebuild:
* Without `--sort=name` the order of files in the archive is determined
by the OS and essentially random.
* Without the `--owner/group/mtime` flags the archive contains metadata
that depends on the environment.
* Without `-n` `gzip` would write a timestamp into the produced
artifact.
CHANGELOG_BEGIN
CHANGELOG_END
* sdk-release-tarball: zip is unused
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* release: Reformat Markdown to fit 80 characters, and adding punctuation.
CHANGELOG_BEGIN
CHANGELOG_END
* release: Formatting, an extra test step, and a bit of clarification.
* Depend on LF version specific daml-libs
* daml-script.dar build multiple LF versions
CHANGELOG_BEGIN
[DAML Script] The `daml-script` library is now available in multiple LF
versions, namely 1.7, 1.8, and 1.dev.
CHANGELOG_END
* daml-trigger.dar build multiple LF versions
[DAML Triggers] The `daml-trigger` library is now available in multiple
LF versions, namely 1.7, 1.8, and 1.dev.
* Keep daml-script.dar available for tests
* Keep daml-trigger.dar available for tests
* daml-libs LF versions integration test
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* participant-state{,-index}: Move Timed*Service classes from Sandbox.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Metrics for the various read, write, and index
services.
CHANGELOG_END
* kvutils/app: Add timing metrics for read/write/index services.
* participant-state: Move metrics-related code to another Bazel package.
* participant-state-metrics: Add to artifacts.yml.
* participant-state-metrics: Move TimedIndexService back into Sandbox.
Cuts down on dependencies like nobody's business.
* Integrate create-daml-app into the assistant
fixes#4868
changelog_begin
- [DAML Assistant] Add a new ``daml create-daml-app`` command for creating a project based on
`create-daml-app <https://github.com/digital-asset/create-daml-app>`_.
changelog_end
* Try random things hoping to fix windows
* Try random things hoping to fix things on macos
* daml-assistant: Add `daml sandbox-next`.
CHANGELOG_BEGIN
- [DAML Assistant] You can now run a pre-release version of Sandbox with
``daml sandbox-next`` so you can test it out and verify everything is
working as expected. Running this will launch Sandbox rebuilt on a
more modern architecture. An upcoming release of DAML will switch over
to the new implementation by default.
CHANGELOG_END
* daml-assistant: Explain that sandbox-next is experimental.
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* daml-assistant: Copy-pasta an integration test for `daml sandbox-next`.
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* sandbox: Fail to start if a time mode is not explicitly specified.
CHANGELOG_BEGIN
- [Sandbox] Sandbox is switching from Static Time mode to Wall Clock
Time mode as the default. To ensure that our users know about this,
for one version, there will be no default time mode. Instead, users
will have to explicitly select their preferred time mode by means of
the `--static-time` or `--wall-clock-time` switches. In the next
release, Wall Clock Time will become the default, and users who are
happy with the defaults will no longer need to specify the time mode.
CHANGELOG_END
* daml-script|triggers: Specify time mode when testing against Sandbox.
* daml-assistant: Default the Sandbox to wall clock time.
CHANGELOG_BEGIN
- [DAML Assistant] Initializing a new DAML project adds a switch to
``daml.yaml`` to ensure Sandbox can continue to start with ``daml
start``::
sandbox-options:
- --wall-clock-time
CHANGELOG_END
* docs: Update the DAML Script and Triggers docs to use Wall Clock time.
It's now what Sandbox will use by default when using `daml init`.
* docs: Change the Quickstart to run Sandbox in wall clock time.
This explains why the contract IDs may vary.
It also updates the manual release testing script to match.
Previously we assumed that the module name was globally unique in the
DAR which is definitely not guaranteed. Now we instead detect the
package id of the trigger library based on the type of the trigger we
are running which doesn’t fall apart if there are multiple versions of
the trigger library.
I’ve also removed the check for the package id of the trigger library
since I’d like the trigger runner to be backwarts compatible from now on (we
didn’t break that in a while).
This is slightly ugly since the Runner class is currently not specific
to a single trigger but only the individual methods are aware of the
specific trigger identifier. I’ll refactor this in a separate PR.
changelog_begin
changelog_end
In the current state of the release instructions, the person in charge
of the release has to figure out how to produce the changelog. This PR
adds more specific (and hopefully simpler) instructions for producing
relevant changelogs.
CHANGELOG_BEGIN
CHANGELOG_END
* Remove damlc migrate
``damlc migrate`` hasn’t worked for quite a while and we emitted a
warning for months so given that we don’t have plans to make it work
again in the near future, I think it does more harm than good to keep
it around.
changelog_begin
- [DAML Compiler] After being deprecated for a while the ``damlc
migrate`` command has now been removed. See
https://docs.daml.com/upgrade/ for up to date documentation
on model upgrades.
changelog_end
fixes#3704 (by removing the tests 😇)
* yeah the windows cache is once again broken \o/
* Revert "yeah the windows cache is once again broken \o/"
This reverts commit 38d7877aa4.
* Freeze DAML-LF 1.8 proto
This is a copy of the 1.dev proto with the following changes that
overall remove all references to 1.dev:
* Change the versions and the reference to the spec.
* Remove `dev` from version history at the top.
* Remove `GENMAP` and corresponding primitives.
* Remove general equality `EQUAL` and the comments that the
type-specific equality primitives are only available in < 1.dev
* Remove experimental text primitives.
changelog_begin
changelog_end
* windows is very bad
* bump windows hashes
* libs-scala/ports: Wrap socket ports in a type, `Port`.
* sandbox: Use `Port` for the API server port, and propagate.
CHANGELOG_BEGIN
CHANGELOG_END
* extractor: Use `Port` for the server port.
* ports: Make Port a compile-time class only.
* ports: Allow port 0; it can be specified by a user.
* ports: Publish to Maven Central.
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
* Add documentation for DAML repl and advertise it
This PR adds some simple docs for ``daml repl`` and adds it to the
release notes.
changelog_begin
- [DAML Repl - Experimental] A new ``daml repl`` command that allows
you to use the ``DAML Script`` API interactively. Take a look at the
`documentation <https://docs.daml.com/daml-repl/>`_ for more
information.
changelog_end
* Update docs/source/daml-repl/index.rst
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* s/Repl/REPL/
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
As mentioned in the title, this is still very experimental and needs
more work before we want to advertise it. However, the code is in a
somewhat reasonable shape, there are tests and I think even in the
current state it is already useful. Also this PR is already getting
very large so I don’t want to hold off much longer before merging this.
It is included in the SDK but hidden from `damlc --help` and `daml
--help` until the most pressing issues are addressed (primarily around
making sure that it doesn’t just shut down if you have a type error
and better error messages in general).
changelog_begin
changelog_end
I verified that there are no semantic changes, only cosmetic ones, by
running the following before and after, and checking the diff.
```
< release/artifacts.yaml ruby -rjson -ryaml -e 'puts JSON::generate YAML::load $stdin' | jq '. | sort_by(.target)' > ~/Desktop/artifacts.{old,new}.json
```
CHANGELOG_BEGIN
CHANGELOG_END
* resources: Allow for startup exceptions without no stack trace.
* sandbox: Remove dependency duplication in the BUILD file.
* ledger-on-(memory|sql): Extract `Main` into its own file tree.
This is so we can remove the dependency on kvutils/app from the library.
logback.xml goes with it.
* sandbox: Implement "Sandbox Next" on top of ledger-on-sql.
CHANGELOG_BEGIN
- [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML
Ledger Integration Kit. Currently not exposed through the CLI.
CHANGELOG_END
* ledger-on-sql: Publish to Maven Central.
* sandbox: If an unknown database is passed to Sandbox Next, handle it.
* resources: Catch errors in owner construction or acquisition.
Sometimes we're not in a future yet.
* resources: Don't run ProgramResource error handling inside the EC.
Otherwise we end up in deadlock which is only broken by
`awaitTermination` timing out.
* Enhanced completions for the assistant
changelog_begin
- [DAML Assistant] The assistant can now do completions for SDK
commands, e.g., ``daml ledger upl<TAB>`` will complete to ``daml
ledger upload-dar``.
changelog_end
* Apply suggestions from code review
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
Originally we ran the release step on both Linux and MacOS to handle
platform dependent artifacts, in particular, damlc.jar. However, we
don’t have any platform dependent artifacts that are uploaded as part
of the release script anymore and I hope we will never have to add any
in the future.
So this PR, removes the code for handling platform dependent artifacts
in the release step and disables the release step on MacOS (while
still setting the variables like we do on Windows).
Currently the release step still costs us ~2 minutes on MacOS which is
already our slowest platform so hopefully this will speed things up a
bit.
changelog_begin
changelog_end
* fix release script
only upload from the linux machine and escape arguments to `publish`,
otherwise they get interpreted by bazel.
CHANGELOG_BEGIN
CHANGELOG_END
* Update release/src/Main.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Update release/src/Main.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* bracket -> bracket_
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* language: add daml-react package to ts libraries
This adds the library formerly known as `daml-react-hook` into the
monorepo. We renamed it to `@daml/react`.
The tests sadly don't work with bazel right now because the local
imports aren't resolved correctly. Local testing with `yarn run test`
works as usual.
CHANGELOG_BEGIN
CHANGELOG_END
* address moritz comments
* get rid of DAVL mentions
* fix eslint warnings
* Update language-support/ts/daml-react/tsconfig.json
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Currently, pretty much all of our builds are bottlenecked on
MacOS (mainly because the builders are slower and have worse
caching). The release step adds > 3min to each build which is a bit
annoying. It turns out that removing the calls to `bazel query` which
are used to check for missing Maven dependencies speeds this up by >
5x. Given that the check is platform independent anyway, we can just
disable it on MacOS.
changelog_begin
changelog_end
* make packages public
This uploads the typescript npm packages of the language support to the
npm registry in the release process.
CHANGELOG_BEGIN
CHANGELOG_END
* address moritz/gary's review
* generate the .npmrc file
* adding debug output
just in case the upload will fail in the next release.
* reverse package order
* Remove damlc.jar
We never advertised or published this externally and our only internal
user has moved off this months ago already.
changelog_begin
changelog_end
* Remove dependency from navigator test lib on damlc jar
* libs-scala/resources: Move `Resource` and friends to their own package.
CHANGELOG_BEGIN
- [Ledger API Server] Publish the resource management code as a library
under ``com.digitalasset:resources``.
CHANGELOG_END
* libs-scala/resources: Add to the artifacts list.
* libs-scala/resources: Factor out the test helpers into test/lib.
* libs-scala/resources-akka: Make the Akka resources an optional dep.
* Start on daml test-scripts
* Run all `Script a` as test cases
* LedgerClient: Expose PackageManagementClient
To enable DAR uploads
* Upload the DAR to the ledger
* Start sandbox if no ledger specified
* Format daml test-script
* Fix deprecation warning on ActorMaterializer
* Add test-case //daml-script/tests:test_daml_script_test_runner
* Add daml test-script command
CHANGELOG_BEGIN
- [DAML Script - Experimental] Allow running DAML scripts as test-cases.
Executing ``daml test-script --dar mydar.dar`` will execute all
definitions matching the type ``Script a`` as test-cases.
See `#3687 <https://github.com/digital-asset/daml/issues/3687>`__.
CHANGELOG_END
* daml-test-script enable logging
* Remove outdated TODO comment
* daml script-test More elaborate test-caseo
Compare to expected output and add failing test-case
* daml test-script Don't abort on test-failure
Before the test runner would abort on the first failed test-case. This
occasionally introduce additional test-failures if the sandbox was
torn down half-way through execution.
* ./fmt.sh
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Moves CommandCompletionIT to Ledger API Test Tool
Closes#2841
Also refactors test observers into a single API and unifies the existing implementations of the DirectExecutionContext
* Allow explicit offsets to get checkpoints
* Raise checkpoint test timeout
* Consolidate checkpoint tests in a single test case
* Fix compilation issues
* Exclude command submission/completion tests for rev-v2
* Drop test not relevant for Ledger API implementations
* Fix compilation errors
CHANGELOG_BEGIN
- [Sandbox] Restore 0.13.38 logging behaviour.
- [Navigator] Restore 0.13.38 logging behaviour.
- [Extractor] Restore 0.13.38 logging behaviour.
- [Internals] As of 0.13.39, we merged a number of internal JAR files in
the SDK tarball to reduce its size. These jars used to be standalone
JARs you could invoke as e.g. ``java -jar sandbox.jar <args>``. As a
result of merging the jars, they lost their individual ``logback.xml``
configuration file. Although running the jars directly was (and is
still) not supported, note that you can now achieve the same behaviour
with e.g. ``java -Dlogback.configurationFile=sandbox-logback.xml -jar
daml-sdk.jar sandbox <args>``.
CHANGELOG_END
* ledger-api-scala-logging: Fix errors in IntelliJ IDEA.
The Bazel plugin for IntelliJ doesn't seem to be smart enough to be able
to handle a Scala library that is part `src` directory and part
generated code from another Bazel rule. It just ignores the second part.
This means that IntelliJ cannot find the *ServiceLogging classes, as
they're not represented on the Bazel-generated classpath, and so
complains with lots of errors when working on the equivalent Api*Service
files.
To fix this, we can split these in two, compiling the base traits to
`ledger-api-scala-logging-base` and then the generated code separately.
It does result in an extra Bazel dependency for the users of
ledger-api-scala-logging, as Bazel doesn't realise transitive
dependencies for us.
* Release: Add `ledger-api-scala-logging-base` to the Maven list.
In #3706 we fixed the SDK versions of the daml-trigger an daml-script
libraries and (rightfully) stopped filtering out 0.0.0 from the SDK
version check in `damlc build`. However, this broke daml-sdk-head
since we still distributed the DARs in daml-sdk-head with the
Sdk-Version set to whatever the current release is rather than 0.0.0.
This still contains the main class so you can use it like you would
use the fat jar but publishing fat jars to maven central is apparently
bad practise and some peple have asked for the library.
This includes some slight tweaks to the scala_docs rule to make it
capable of coping with the generated source file and a hack in the
release script to avoid it complaining about the scenario proto
library not being published to Maven even though it is included in the
transitive deps.
* Spin off TokenHolder into a new library
Avoids having weird dependencies between different packages, makes TokenHolder available on Maven
* Fix auth-utils path
Previously, we constructed the tarball used by daml-sdk-head by first
building the regular SDK release tarball only to the extract it, patch
the version file and recompress it. This adds about 30-60s to every
invocation of daml-sdk-head.
This PR changes this by factoring out the logic from building the
release tarball into a macro that we instantiate twice, once with the
proper version file and once with a dummy HEAD version file set to
0.0.0.
This does change the format of the sdk-head-tarball slightly, in
particular the files are now located under sdk-0.0.0 instead of
sdk-head. However, this doesn’t matter anyway afaik and I think the
new format makes more sense anyway considering that the regular
release tarballs have something like sdk-0.13.38.
* ledger-api-test-tool: Increase the duration when watching health.
This should hopefully stop CI from flaking out.
* reference-v2/sandbox: Avoid unnecessary companion object constructors.
I like indirection… when it does something.
* ledger: Propagate empty health checks throughout the services.
* reference: Remove duplication from the ReferenceServer object.
* ledger-api-common: Actually query a "reporter" in the health service.
* ledger-api-common: Report health per-component when required.
* ledger-api-health: Use a Map to represent components for health checks.
* sandbox: Fix warnings in SqlLedgerSpec.
* ledger-api-common: Throw GrpcHealthService errors inside the Future.
* ledger: Implement health checks against the PostgreSQL connection.
Without proper testing, because I am not great at this.
* sandbox: Remove duplication and fix warnings in PostgresAround.
* sandbox: Test the SQL Ledger's health reporting on failure.
* sandbox: Don't report as unhealthy until 3 connections fail.
* ledger-api-health: Remove unused parts of the API.
Bit of premature design there.
* sandbox: Rename the "ledger" health check to "write".
* participant-state: Add the ReportsHealth trait to ReadService.
* ledger-api-common: `Future.fromTry(Try(…))` -> `Future(…)`.
* ledger-api-common: Make it clearer that StubReporter closes over health.
* ledger-api-common: Explain the HealthService watch tests with comments.
* sandbox: Clean up SqlLedger a bit.
* sandbox: Don't try and stop PostgreSQL twice in PostgresAround.
* bazel_tools: Windows rlocation lookups need to be with forward slashes.
* release: Fix case of "true".
* ledger-api-common: Make `GrpcHealthService::matchResponse` return a Try.
* ledger-api-common: Make `GrpcHealthServiceSpec` async.
* sandbox: Make a couple of DB classes final.
* sandbox: Avoid importing `X._` in PostgresAround.
* sandbox: Add clues to the SqlLedgerSpec's multiple assertions.
* sandbox: If PostgreSQL doesn't come back up, keep retrying.
* sandbox: Remove duplication in SqlLedgerSpec.
* sandbox: In SqlLedgerSpec, actually wait for the health to change.
* sandbox: In PostgresAround, make stopping PostgreSQL idempotent.
* sandbox: Simplify the SqlLedgerSpec to make it work on CI.
It's worth a shot.
* ledger-api-common: Simplify the GrpcHealthServiceSpec a little.
And add a changelog.
CHANGELOG_BEGIN
- [Ledger API Server] Add a health check endpoint conforming to the
`GRPC Health Checking Protocol <https://github.com/grpc/grpc/blob/master/doc/health-checking.md>`_.
- [Ledger API Server] Add health checks for index database connectivity.
- [Participant State API] Add a mandatory ``currentHealth()`` method to
``IndexService``, ``ReadService`` and ``WriteService``.
CHANGELOG_END
* sandbox: Improve the Javadoc layout for DbDispatcher.
* sandbox: Capitalize constants in SqlExecutor.
* ledger-api-health: Convert HealthStatus to an abstract class.
* Start moving all the jars into a single jar.
* Fold navigator into daml-sdk jar
* include sandbox
* Remove unnecessary compileDeps.
CHANGELOG_BEGIN
- [DAML SDK] Reduced the size of the DAML SDK by about
60% uncompressed, 70% compressed, by deduplicating Scala
dependencies.
CHANGELOG_END
* update copyright header
* buildifier fix
* Add participant-state configuration protobuf
- Move the "DamlConfiguration" from kvutils into participant-state/protobuf/ledger_configuration.proto.
- Add version number and spec (ala transaction.proto)
This is a preparation for indexing the configuration and having one canonical serialization for it.
* Initial thoughts on indexing the configuration
* Implement indexing of ledger configuration changes
* Add record time to all Updates. Wire through participant id.
and rename V7__Add_configuration to V8.
* Add ledger_configuration_java_proto to release artifacts
* Fix up release of ledger_configuration_java_proto
* Suggestions from review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* address rebase issues
* Resolve compilation errors after rebase/merge
* happy formatting for scala and bazel and friends
* Drop "openWorld" setting from configuration
And refactor the tests to allocate parties explicitly.
* Fix up migration and tests
* Drop authorizedParticipantIds from configuration
Implement configuration authorization in kvutils using the previous
the participant id of the previous configuration.
* Post-rebase fixes
* Add missing migrations
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Add missing mavenUpload to ledger-api-server
* Remove stateUpdateRecordTime
* Address code review
- Address PR review
- Merge TimeModelImpl and the traits. Remove TimeModel from ledger-api-common.
- Throw `Err` from KeyValueConsumption on parse errors instead of assert/sys.error
* Reformat
* Add missing protobuf file
* Fix compilation after TimeModel changes. Add version logs to participant-state{,-kvutils}.
* Fix TestUtil.scala build
* Apply suggestions from code review
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Address review
- synchronized access to ledgerConfiguration in InMemoryLedger
- store rejection if configuration generation is wrong
* Update ledger/participant-state/protobuf/ledger_configuration.rst
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
