* Remove unused class `JavaConf`
* Adapt tests to add template references for every type
* Minimize the data passed to `collectDamlLfInterfaces`
* Separate `CodeGen` from `CodeGenRunner`
* Refactor interfaces collection and resolution
* Refactor Java codegen
* Return `InterfaceType`s from `transitiveClosure`, move `DefTemplateWithRecord` to the Scala codegen
* Remove unused abstractions from the Java codegen
changelog_begin
changelog_end
* Minimize diff: move as much as possible to `CodeGenRunner`
* Revert "Adapt tests to add template references for every type"
This reverts commit 37c8c827179daa29668fa843c79afc8cef09e431.
* Fix compilation errors
* Minimize diff: restore imports in `ClassForType`
* Minimize diff: get rid of unnecessary refactorings
* Further simplications and fixing compilation errors
* Minimize diff: move `collectDamlLfInterfaces` back to its previous position
* Address https://github.com/digital-asset/daml/pull/13440#discussion_r846301958
* Remove unused `name` field from `InterfaceTreeBuilder`, inline builder construction
* Remove unused `Traverse` instance for `TypeDeclOrTemplateWrapper`
* Remove unneccessary type parameter from `TypeDeclOrTemplateWrapper`
* Remove some unnecessary copies `.toList`
* Remove `TypeDeclOrTemplateWrapper`, use `Either` directly as a wrapper
* Use `bimap` instead of explicitly wrapping and unwrapping the `Either`
* Remove unnecessary type `Mode`
* Remove unnecessary type parameter from `WriteParams`
* Remove unnecessary binding
* Pull `OrderedDependencies` out of `Graph`
* Move computation of relevant packages to `codegen-common`
* Move `templateCount` from `LFUtil` into `CodeGen`
* Comment the purpose of `TransitiveClosure`
* Pull the variance cache out of `LFUtil`
`LFUtil`'s purpose is that of limiting passing around certain
values. While the purpose is questionable, the variance cache did
not need to access any of those values and it's quite easy to
move to a separate class where the purpose and the scope are clearer.
* Make `DependencyGraph` work directly on type declarations
* Add constructor for a collection of interfaces to `EnvironmentInterfaces`
* Factor the error message from `cyclicDependencies` out of `DependencyGraph`
changelog_begin
changelog_end
* WIP
* First working version of java codegen daml interface support
* Update language-support/java/codegen/BUILD.bazel
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Fix compile errors
* Simplify code massivly, enjoy less duplication
changelog_begin
- The Java codegen now has basic support for daml interface definitions. Converting a contract id of a template implementing an interface to a contract id of the interface is possible and both executing interface choices is possible on the contract id of the interface and implementing template.
changelog_end
* Rename the test file to reflect it is a test file & enhance the inner test name & extend it further
* Fix test
* Refactor parts of TemplateClass.scala into multiple files
* Format that files!
* Minimize duplication further
* Remove unused comment
* Simplify code
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Update language-support/java/codegen/src/main/scala/com/digitalasset/daml/lf/codegen/CodeGenRunner.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Further refactoring and renaming of the TemplateClassSpec to ContractIdClassBuilderSpec
* Fix formatting
* Add interface docs
* Remove unnecessary code generation of the Contract class for interface types
* Use less bool flags and more good function names :)
* Fix build
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
The Scala codegen code lives under `com.daml.codegen` but the common
library and the Java codegen both live under `com.daml.lf.codegen`.
In order to share code while preserving package private access of
certain facilities (i.e. `DependencyGraph`) the package needs to
be standardized. I chose to go the Java way mostly to reduce noise
on work currently on its way on the Java codegen (#13366).
Contributes towards https://github.com/digital-asset/daml/issues/13324
changelog_begin
changelog_end
Specifically for the Java codegen, builds on top of #12977.
Makes sure we don't rely on PackageName.fromString and PackageVersion.fromString
to blow up when passed an empty string.
changelog_begin
changelog_end
* Fix parsing of java codegen module prefixes
I admit I have no idea what I thought when I wrote that split but it
was clearly nothing sensible.
changelog_begin
- [Java Codegen] Fix parsing of `module-prefixes` to support package
names containing dashes.
changelog_end
* reveiw comments
* Inline nameVersion
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* Get rid of deprecated components
Removes:
- mentions of the Node.js bindings
- mentions of the Scala bindings
- usage of the Scala codegen as an SDK tool
- Java bindings "reactive components"
changelog_begin
The Node.js bindings are no longer supported as part of the Daml SDK
The Scala bindings are no longer supported as part of the Daml SDK
The Java bindings reactive components are no longer supported as part of the Daml SDK
The `daml codegen` command has dropped support for Scala
changelog_end
* Fix typo noticed by @cocreature
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Address https://github.com/digital-asset/daml/pull/12167#issuecomment-995714093
* Remove quickstart-scala from the SDK tarballs
* Fix failing codegen test
* Address https://github.com/digital-asset/daml/pull/12167#pullrequestreview-833991243
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Handle module-prefixes in Java codegen
changelog_begin
- [Java Codegen] The Java codegen will now pick up the
`module-prefixes` field from `daml.yaml` which can be used to handle
module name collisions between different DALFs.
changelog_end
* Reference ticket in todos
changelog_begin
changelog_end
* less stupid map initialization
changelog_begin
changelog_end
* Pull out string assembly
changelog_begin
changelog_end
* Update language-support/java/codegen/src/main/scala/com/digitalasset/daml/lf/codegen/CodeGenRunner.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update language-support/java/codegen/src/main/scala/com/digitalasset/daml/lf/codegen/CodeGenRunner.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* fewer views
changelog_begin
changelog_end
* more assert
changelog_begin
changelog_end
* more private
changelog_begin
changelog_end
* fmt
changelog_begin
changelog_end
* fix scala 2.12
changelog_begin
changelog_end
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
The one thing that is still missing is making the generated Scala code
from the codegen compatible with Scala 2.13 so the examples are
excluded for now.
changelog_begin
changelog_end
* Support DAML_PROJECT in daml codegen java
fixes#8406
We need to resolve the DAR path relative to the project
root. Otherwise, we’ll just fail to find the file if daml codegen java
is invoked from a different directory.
changelog_begin
changelog_end
* Fix test
changelog_begin
changelog_end
* fmt lalala
changelog_begin
changelog_end
* fix another test
changelog_begin
changelog_end
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.
This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.
Apologies for the giant PR, I don’t see a way to keep it smaller.
changelog_begin
changelog_end
* add -Xsource:2.13, -Ypartial-unification to common_scalacopts
* add now-referenced scalaz-core where needed
* work around bad type signatures in scalatest Aggregating, Containing
* unused Any suppression
* work around bad partial-unification wrought by type alias
* remove unused Conversions import
- not required in 4f68cfc480 either, so unsure how it's survived this long
* work around Future.traverse; remove unused show import
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused bounds
* remove -Ypartial-unification and -Xsource:2.13 where they were explicitly passed
* longer comment on what the options do
- suggested by @stefanobaghino-da; thanks
* forget Future.traverse, just use scalaz, it knows how to do this
Packages com.digitalasset.daml and com.daml have been unified under com.daml
Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.
CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
* Use com.daml as groupId for all artifacts
CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END
* Add 2 additional maven related checks to the release binary
1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId
* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
* Update bazel-common to fix javadoc issues
Specifically, to fix the following error
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
javadoc_library(name = 'rs-grpc-bridge_javadoc')
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```
* Define Maven deps using rules_jvm_external
* Pin artifacts
* Remove bazel-deps generated targets
* Remove bazel-deps
* Switch to rules_jvm_external targets
* update bazel documentation
* pom_file: There are no more bazel-deps targets
* BAZEL-JVM.md `maven_install` typo
* language-support/scala: add support for specifying roots explicitly
By default, code-gen will use all templates in the given package as roots, and
generate code for all types used. This patch adds the ability to specify roots
explicitly via a set of regex.es to match the (fully qualified) template names.
This is an experiment, tests and UI need to be adjusted later.
* scala codegen rootfilter: move filtering to envIFace handling
* doc for --root option
* scalafmt
* test filterTemplatesBy
* release note for --root Scala codegen option
* Extract codegen-common module, #166
* Scala Codegen Main using the same option parser as Java Codegen, #166
There is one important difference, Scala Codegen does not allow mapping
dars to different package names, all dars have to be mapped to the same
package name.
Replace Scala Codegen println's with scala logging, respecting the
configured codegen verbosity
* Fix bazel formatting
* Update the release dry run script
* Releasing codegen-common
* Improving Scala Codegen error reporting (code review)
* Addressing codereview comments
* Make it explicit that we skip not supported option
