* Move packaging tests to a separate test suite
Originally, we had these as part of the integration test suite since
`bazel run damlc build` couldn’t locate `ghc-pkg` but that has been
fixed for a while.
Moving it to a separate test suite speeds things up as the integration
tests are rerun very often and also makes development much more
convenient since the new test suite supports `-p` properly to filter
to specific tests.
For now, I’ve left the upgrading tests as part of the integration
tests. I expect that we probably want to factor those out to another
test suite as well.
* fix warnings
* language: fix for tuple types in source generation
We correct the source generation from dalfs containing TupleN data
types. A test is added to the integration tests of daml assistant and
the `generate-src` command is improved to make this test possible.
* check that tuple type comes from DA.Types
* check that compilation succeeded
* pattern match golf
* check for tuples between 2 and 20
This change allows to only import `module A` from the instances package
if data dependencies are present instead of `module A` and `module
AInstances`.
* language: introduce data-imports
Right now the user experience for importing dalfs and dars from
different sdks is quiet confusing. This PR tries to solve this. We add
an additional field `data-imports` to daml.yaml. These imports can come
from different SDK's and we will generate interface files containing the
data types and their Template instances.
This also simplifies the migration command, as it now always imports the
respective packages as `data-imports`.
* language: cross sdk dalf/dar imports
The final piece for cross sdk imports. With this PR we can import the
data types of packages and dalfs that were created with different sdks.
This is done by generating interface files from dalfs and an 'instances'
package that contains the template instance definitions of template data
types. The instances itself are defined via the `external` keyword,
which is inlined to proper daml-lf instance definitions given in the
respective dalf package.
We test that cross sdk imports work by importing the `simple-dalf` in
the daml-assistant integation tests and running a scenario.
Currently we hide modules, for which we don't define an alias, when we
specify one alias for *one* module. With this change, all modules
without aliases are still exposed.
* language: refactoring of iface file generation and package db setup.
This is a refactoring of the damlc part that creates the package
database and the code generation for interface files. This is a
preparation for the cross sdk imports.
We also add an internal command to damlc to generate generic instances
code and use simple copying via {..} in the migration command. An
additional test checks that migration via generics still works.
* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/Upgrade.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* clearer description for generate-gen-src
* updated documentation
* correct copy/pasta mistake
* added a comment on different build options in migration command.
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* resolve dalf paths from dar manifest
* added a comment on different headers in upgrade modules.
* removed monoid instance for ExtractedDar
* Update bazel-common to fix javadoc issues
Specifically, to fix the following error
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
javadoc_library(name = 'rs-grpc-bridge_javadoc')
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```
* Define Maven deps using rules_jvm_external
* Pin artifacts
* Remove bazel-deps generated targets
* Remove bazel-deps
* Switch to rules_jvm_external targets
* update bazel documentation
* pom_file: There are no more bazel-deps targets
* BAZEL-JVM.md `maven_install` typo
* language: dalf imports and a test
This adds the possibility to directly import dalfs in a project. We test
that we can import the `simple-dalf` in the daml-assistant integation
tests. For now we only check that data type generation works, not yet
the template instance.
The following was fixed: When rewriting package self references, this
changes the hash of the package later on and leads to different package
hashes. Also we need to be careful to write the orignal binary
representation to this and not re-encode it because the encoding might
have changed with a different sdk.
* addressing moritz's comments.
* windows doesnt like bazel paths
* Add a first draft of documentation for DAML triggers
The API will still change in a bunch of ways but I’d rather get some
docs in place now and update them as we change things than not have
any docs at all.
* Fix path to daml.yaml
* s/bot/trigger/
* fix source code markers
* Fix tests
* Update docs/source/triggers/index.rst
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* language: fix: don't look for daml files in hidden directories
Fixes#3134. When locating daml source files, we need to make sure we're
not recursing into the .daml directory, because it contains source files
of the dependencies.
* Update daml-assistant/integration-tests/src/Main.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This moves the creation of a package database from given dalfs out of
the migrate command and into the init command. In particular, this makes
the process of creating a package database independent of the migrate
command.
It also changes the way this package database is created to be only
dependend on given dalf files.
* Flattening results of the `/contracts/search`,
returning a `ActiveContract` instead of `GetActiveContractsResponse`
* Fixing the test,
it is an empty array instead of empty GetActiveContractResponse object
* Updating examples
* Updating release notes
* Update unreleased.rst
Accepting suggested changes
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
* Resolving master merge conflicts
This should make `daml start` a bit more useful since you don’t have
to switch to starting all processes separately once you start
deviating from the defaults, e.g., I found myself wanting to specify a
custom ledger id during the hackathon. This is part 1 of ##2993.
The filename of the dar is not something that you should rely on as
evidenced by the fact that we have a -o option to change it to
something completely different.
* Use a consistant include dir for cwd
See https://github.com/digital-asset/ghcide/pull/114 for the actual
fix.
This PR just bumps ghcide and adds a regression test. I’ll change the
revision before merging, I just want to test CI for now.
fixes#2929
* Switch to proper ghcide revision
* writeIfacesAndHie no longer exists
* Add changelog entry
* Maybe I should try to compile code before committing but I don’t want to
* Fix ghcide exe
* language: support for automatic embeddings/projections in migation
This adds support to automatically generate embeddings/projections if an
optional field to a contract template is added. This allows generation
of migration contracts for these kind of updates.
* only project None entries
* added a test for optional field embedding/projection.
* filter out exception stack traces logged at debug
* Adding logback.xml for DAML Assistant release
* Fixing readme
* Fixing readme
* Use the config file in the assistant
* add Numeric.java
* ledger-api: rename `decimal` field to `numeric` in value protobuf
* Address Gerolf's comment
* ledger-api: add missing renammings
* ledger-api: relax syntax of numbers that can be sent as numerics
* extractor: fix
* leger-api: change format of number though ledger api
* daml-lf: fix numeric regexp
* ledger: fix tests
* language: compile everything in the source directory
This removes the need to specify a 'main'. Instead we 'source' in
daml.yaml should point to the source root directory.
* language: smoothing out the migration experience
Some improvements to make the migration process simpler when migrating
between packages with the same name, e.g. foo-1.0.0 -> foo-2.0.0. This
is the main use case.
* language: append the version to the output dar name by default.
We now by default output foo-1.0.0.dar instead just foo.dar. Also the
maven coordinate default naming got removed.
* fixing integration tests and quickstart.dar occurences
* Only change project root once
As described in #2449, calling withProjectRoot' twice breaks with
relative paths and is also just silly so this PR fixes this by
factoring out the actual logic from init from execInit which does the
project root thingy.
* Add a regression test
* Add link to deploy URL in daml ledger --help and daml deploy --help
* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
Despite, a lot of searching around, I couldn’t find any information on
why exactly quoting "code" causes issues but removing the quotes seems
to fix the issue ¯\_(ツ)_/¯
* Fix daml build for usernames with spaces
This PR fixes two issues caused by having spaces in your username:
1. On Windows, we need to quote the path to the daml binary in the
batch wrapper (quotes are not valid in usernames, so no need to worry
about escaping them).
2. Invoking ghc-pkg via callCommand broke since shells are
terrible. Luckily, we can easily get away with just using callProcess
here.
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* language: put hash in package directories
We put the package id of the main dalf of a package into the directory
names, where we store the files of that package in the package database.
This way we make sure that two equally named packages don't overwrite
their dependencies and files.
* Supporting producing sdist tarballs for the HS ledger bindings
The README.md has an explanation for how you can use this.
This should hopefully allow others to experiment with the bindings.
Currently, if you do `daml init foo` and there's a file `foo/Main.daml`,
you'd end up with a file `foo/daml.yaml` whose `source` entry is
`foo/Main.daml`when it should actually just be `Main.daml`.
This PR fixes this problem.
* daml deploy: first version
* fix build: replace callCommand with runProcess_
* abstract/share: doBuild, getDarPath
* hide deploy command for now
* make sandbox port configurable for both deploy & start daml-helper commands
fixes#2142
It turns out that typed-process has the behavior we want so rather
than rolling our own version of `withCreateProcess`, I just switched
to that.
extractTarball tries to restore the ownership stored in the
tarball. This goes wrong when the tarball is fetched from CI since the
uid/gid might not exist locally.
* Bazel: 0.24.0 -> 0.27.0
* Update rules_haskell for Bazel 0.27 compatibility
* Update bazel-deps and bazel-watcher
* Windows escape JVM flags
* load commands at top of .bzl file
Bazel 0.27 no longer allows load commands that are not at the beginning
of the file.
* Update Bazel rules
* subpackage boundary
* native is not defined in BUILD files
* yarn: @bazel/hide-bazel-files
Seems to be required since latest rules_nodejs version. Otherwise, yarn
fails with errors about existing BUILD or BUILD.bazel files.
* grpc-java plugin visibility
* Update fat_cc_library
* Nix Python3 toolchain
* Iteration over depset
* dev_env_package: Create symlinks one level deeper
To prevent symlinking the BUILD file as well. The nested BUILD file
confuses Bazel as of 0.27 and rules_nodejs cannot find the node
executable anymore.
* Update rules_nodejs
* Add managed_directories for node_modules
* hie-bios: Extract bazel-genfiles from bazel info
Bazel 0.27 changed the genfiles location which breaks the hie-core test
on macOS.
* update cc_wrapper to Bazel 0.27
* bazel info -> bazel info bazel-genfiles
* Fix typo in BUILD
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* language: upgades: generate upgade code from dalfs
This switches the `migrate` command to read dalf files instead source to
generate upgrade code. The dalf files are read, data types are converted
back to haskell source, and from this we can generate generic instances
again.
There are two known issues that we want to address next:
- Duplicate generic instance definitions will make the build fail.
I.e this only works for data declarations that have no generic
instances
- Data declarations of the form `data Foo = Foo ()` can not be
converted back to haskell source.
* Update rules_haskell and static GHC
Remove patches that have been upstreamed or are no longer required.
Update still required patches to match the new rules_haskell version.
Previously we patched rules_haskell to coerce GHC into using static
Haskell libraries in most places. In particular we moved hs-libraries
entries into extra-libraries entries in the package configuration files.
A much cleaner approach is to compile GHC with a static RTS, then GHC
will by itself choose to load static Haskell libraries.
* Remove haskell_cc_import
* da-hs-daml-cli -> daml-cli
* da-hs-damlc-app -> damlc-app
* windows: hanging GRPC FFI call problem resolution
* windows: fix visual test
* windows: enabled back again some of disabled daml-assitant tests
* marking daml_ghc_integration_test as large
language: upgrades: dont derive generics for data types having the instances already
We only derive generic instances on the fly for data types that don't
have them already.
* Add support for haskell_repl targets in da-ghci
* Change default target
* Revert newline loss.
* bazel fetch before bazel query
* Make a top level repl target the default.
* Add da_haskell_repl dependency in //BUILD
* Fix syntax error
* Fix bazel formatting...
* Rename DamlHelper modules to make //:repl work
* DamlHelper -> DamlHelper.Run
* Update the import in DamlHelper.Main
* Fix bazel rules again
* Update DamlHelper import in integration-tests
* language: generate module containing generic instances
The migrate command generates generic instances of data types and puts
them in a separate module. For now this only works if the data type
doesn't have a generic instance already, we'll deal with this case in
the next PR.
* Add --install-assistant option
* Better doc on iActivate
* Determine whether to install assistant based on new flag
* Fix logic and update install warning
* Remove --activate from install.sh
* Add missing ]
* Change deprecation warning.
* Add release notes for new option.
* Update help text for install-assistant
* Fix release notes
* language: feature: initial implementation of a 'migrate' command
We add a 'migrate' command to daml assistant that generates a project
that allows to migrate contract instances from package1 to package2.
This first version reads both package1 and package2 from source. As a
next step we read only the dalfs from package1, because it might have
been created with a different compiler.
* Simplify daml version output
* Make sure project version is really from daml.yaml
* Calculate latest version more thoroughly.
* Only show latest version if available
* Add test case for damlc test-files
* Separate damlc test-files from damlc test
* Fix rules_daml/daml.bzl daml_test
* damlc test-files --> damlc test --files
The project options are still relevant for the --files case, as it may
be necessary to change into the project root directory in order to
locate the project package database.
This was already possible before via the DAML_PROJECT environment
variable but for users that want to call damlc directly, e.g., via
damlc.jar a CLI flag can be more convenient.
* List all available versions.
* Add --all flag in daml version
* Save version list to cache
* Update version cacheing logic.
* Linting error
* PR revisions.
* Update release notes.
* Update daml-assistant/src/DAML/Assistant/Version.hs
* Update docs/source/support/release-notes.rst
Co-Authored-By: Beth Aitman <bethaitman@users.noreply.github.com>
* no var no problem
further refactor
introduced InfraState
a bit less vars
encapsulating closes
SandboxServer starts automatically
rebase fixup
collecting state into a single object
some cleanup
removing exposed materializer
LedgerBackend is closed in SandboxServer
changed ownership of Ledger
fixing perf tests
fixing some compile errors
formatting
removing unused method
fixing integration test to use correct dar file
fixing issue with PostgresFixture and SandboxResource
Fix integration tests on Windows
* fixing rebase artifacts
* Add an option to not modify PATH in daml install --activate
This is particularly useful in test suites where you install to a
temporary directory that should not be added to the registry.
* Use yes/no/auto
* Make check more robust
* Separate version logic out of DAML.Assistant.Env.
* Refactoring some of the exception handling.
* Update daml version command.
* Uncommit linting atrocity.
* Started working on daml init.
* Implement daml init.
* Nicer messages and nicer field generation.
* Cleaning up a duped definition.
* Review revisions
* Catch all synchronous exceptions when making network requests.
* Wrap all of getLatestVersion.
* wrapErr wraps sync exceptions as well.
* Pass through exit codes.
* Add two failing getDispatchEnv tests.
* Fix getDispatchEnv idempotency.
* Fix new test formatting.
* Make getDamlAssistantPath look in env first.
* Fix daml env var overriding.
* Test all the Nothing cases of env var dispatching.
* Fix dispatchEnv and getDamlEnv for Nothings.
* Add hlint rule to avoid future setEnv debacles.
* Fix other uses of setEnv.
* Fix type error.
* Fix reviewer comments
* setEnv comment
* Do not ask user to install when they are already running the install command.
* Avoid looking at unecessary information when doing daml install.
* Update daml-assistant/exe/DAML/Assistant.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Remove unnecessary import.
* Move code shared between lib and tests to a DAML assistant library
Previously we were compiling the same code twice, once for the binary
and once for the test suite.
* Move tests and binary to separate directories
The lack of sandboxing on Windows in combination with
-Wmissing-home-modules breaks our Windows build otherwise.
