Commit Graph

516 Commits

Author SHA1 Message Date
Samuel Williams
6a88119637
Multi package test (#17746)
* Multi package test scaffold and first batch

* Add next set of tests

* Rearrange, add source testing

* Explicit imports

* Install sdk in tests

* Blank setEnv is different to normal setEnv >:(

* Fix paths for windows

* Add daml.yaml changes tests (that fail)

* Update from comments, add sdk-version check

* Update tests to match changes to CLI behaviour
2023-12-19 11:03:29 +00:00
dylant-da
96485898f4
Fix issues with damlc with custom release version querying network (#17991)
Co-authored-by: Samuel Williams <samuel.williams@digitalasset.com>
2023-12-06 19:35:10 +00:00
dylant-da
40b5eb8476
Fix split release for deps (#17968)
* Resolve release version for sdk build checks

* lint

* lint

run-full-compat: true

* Simple test for using daml-script in release versions

* Fix build issues in tests using pSdkVersion

run-full-compat: true

* Fix build issues with DamlcIntegration

* fix bad sdk version being an invalid version

run-full-compat: true

* Fix the linux "mmap 4096 bytes at (nil): Cannot allocate memory" error

* Fix compat tests on Windows

run-full-compat: true

* test windows os correctly

run-full-compat: true

* temporarily disable canton_3x

run-full-compat: true

---------

Co-authored-by: Paul Brauner <paul.brauner@digitalasset.com>
2023-12-05 23:46:27 +01:00
mziolekda
34743570bd
Rewire the build to use ledger-api within canton (#17959)
* Rewire the build to use ledger-api within canton

* same for canton-3

* fix buf

* fix buf again

* remove references to canton from canton-3x
2023-12-04 16:11:13 +01:00
dylant-da
4d8e4995df
Add --install-with-custom-version, rename --allow-install-non-release (#17931)
* Add --install-with-custom-version, rename --allow-install-non-release

* Test install tarball via custom version in compatibility tests

run-full-compat: true
2023-12-04 10:31:28 +00:00
mziolekda
7e2aa7cece
Adopt the repository to the movement of java bindings (#17913)
* Make java bindings compile in new place

* fix a mysterious grpcurl error
2023-11-30 22:43:35 +01:00
Gary Verhaegen
6f457e872b
Allow install non release does not look for matching version (#17909) (#17918)
Co-authored-by: dylant-da <106664681+dylant-da@users.noreply.github.com>
2023-11-27 12:00:49 +01:00
dylant-da
8faad711fe
Fix split release breaking create-daml-app test (#17898)
Adding --allow-install-non-release=yes to withSdkResource fixes issues with integration tests in 2.8.0 release
2023-11-23 11:36:08 +00:00
dylant-da
234d003f0c
Supporting version splitting in Daml Assistant and Daml Compiler (#17682)
* initial implementation supporting version splitting in assistant

stubbed out implementations of resolveReleaseVersion/other resolution

* implement resolveReleaseVersionFromGithub

* continue work on fixing SDK/release version split

* First commit that successfully runs `daml-head install <split version>`

* fix tests

* Ignore snapshot/metadata for isHeadVersion

* remove log debugging

* Fix tests for getSdk

* refactor DA.Daml.Project.Types.defaultSdkPath

* enable incremental changes to version cache

* Allow resolveReleaseVersionFromGithub to fail via Either

* Split getSdkVersionFromSdkPath into get{Sdk,Release}VersionFromSdkPath

* Add resolveReleaseVersionFromDamlPath/Github to resolveReleaseVersion

* Add mock sdk config with version

* Remove getInstalledSdkVersions dependency on Cache

* add override for github version api endpoint, useful for mocking

* Add alternate-download to provide alternate tarball install resolution

* initial work on comprehensive autotester

* Copyright header

* fix build generation to have valid Main.daml

* improve error message for check_daml_install_nonzero

* Make killing miniserve processes more robust

* remove breakpoint

* Implement `daml build` tests

* Solve TODO in installExtracted to resolve sourceSdkVersion coherently

Supply useful error message with it.

* Return SdkVersion from sdkVersionFromReleaseVersion for typechecking

* Finish tests for tarball build, drop installed_already_behaviour

* Try to force-reload cache on tar failure, fix `Possible fix:` notes

* Add AllowInstallNonRelease as a flag

* Test allow_nonrelease, refactor, clean up setup_sandbox

* Move no_cache_override_github_endpoint API response into file

* Remove dead code, vestigial code

* Remove TODOs that no longer are relevant

* fix iAllowInstallNonRelease in InstallOptions for autoInstall

* Refactor alternateVersionLocation

* (Try to) lint language-support/ts/codegen/BUILD.bazel

* fix daml-assistant/test

* Enable allow_nonrelease post-build checks

* Remove writeFile debugging from DA.Daml.Assistant.Version

* Replace ../test-daml-yaml-install/test.sh with test-all-installs.sh

* lint

* more lint

run-full-compat: true

* Stop tracking API response in test-daml-yaml-install

* Move ReleaseResolution into Version

* remove extra deps, fixes daml-project-config-cabal-file-matches

* Move InstallLocation to minimize changes to DA.Daml.Project.Types

run-full-compat: true

* Create cachePath in daml-assistant tests that need it

* Bind to unused ports in test-all-installs.sh

run-full-compat: true

* Override via files instead of just URLs

run-full-compat: true

* Remove unused RELEASES_ENDPOINT

* Use dict keys instead of hardcoding in test-all-installs listing

* Refactor to remove check_daml_install_from_tarball_after_cache_reload

* Improve comment on update_cache

* Move shift before cases

* Add comment for unsafeParseReleaseVersion

* Rename unsafeParseReleaseVersion into unsafeParseOldReleaseVersion

* remove done todo

* Add some hungarian notation to resolveReleaseVersionFromDamlPath

* drop redundant let

* Define ordering over ReleaseVersion

run-full-compat: true

* use sdkVersion for codegen

* Use `urls` attribute in http_file, `url` is unsupported on Windows (!)

run-full-compat: true

* Remove unnecessary check for cache reload

run-full-compat: true

* Try use daml executable directly without daml-sdk-0.0.0

run-full-compat: true

* try force daml.exe to daml

run-full-compat: true

* Use daml.exe when windows is detected

* add windows tarballs for snapshots

* Fix most tests on windows, "line too long" breaks some tarball tests

run-full-compat: true

* Point to more recent snapshot post Moises's fixes

* Add os-specific tarball paths and alternate-download

run-full-compat: true

* Fix windows autoinstall with 0.0.0

run-full-compat: true

* Fix error message, remove daml_install_output catching

* Detect "The input line is too long" in other post_failed commands

* Fix missing releases endpoint, handle "cannot find the path specified"

run-full-compat: true
2023-11-22 14:24:13 +00:00
mziolekda
b81ee69abe
disentangle ledger api (#17831)
* make testing-utils independent of ledger-api

* remove dead files

* move non ledger-api stuff to libs-scala
2023-11-14 22:44:10 +01:00
Marton Nagy
6933514df5
Switch from Akka to Pekko (#17814)
* Automated renames by bash script

This commit is exclusively contains changes by the bash script.
For the bash script is present at the pull request.

* Manual pekko migration changes

* adapt fully qualified name references
* adapt pekko package declarations
* adapt bazel files with dependency changes
* adapt canton pekko lib shade_rule
* adapt logger configuration declarations
* pin maven dependencies
* revert incorrect changes by script to compatibility module

Workarounds for further TODOs:
* disable http-json-perf and libs-scala/gatling-utils modules to maintain clean pekko dependencies (without akka)
* disable GraphQLSchemaSpec test (sangria library needs to be upgraded)

* Formatting
2023-11-13 23:22:22 +00:00
Remy
6ef062dac9
Revert "Nmarton/pekko rebased (#17789)" (#17796)
This reverts commit d310692d1b.

@remyhaemmerle-da messed up and merged the PR too fast.
2023-11-09 21:23:18 +00:00
Marton Nagy
d310692d1b
Nmarton/pekko rebased (#17789)
* Scala upgrade to 2.13.11
* Switch from akka to pekko
* Also disable gatling based libs to start from a clean stage
2023-11-09 18:17:33 +01:00
Moisés Ackerman
3512b5eb4d
Embed bash and zsh completion scripts into daml-assistant (#17675)
* Update optparse-applicative dep to 0.17.1.0

* Update entropy dep to 0.4.1.10

* Embed bash and zsh completion scripts into daml-assistant
2023-10-27 17:23:20 +02:00
dylant-da
845b486e56
Add test-daml-version-and-install script, fix more updating issues (#17599)
* Add test-daml-version-and-install script, fix more updating issues

* Test `daml install`, improve checks for snapshots.txt

run-full-compat: true

* bring diff into scope for test-daml-version-and-install.sh

* Use runfiles to locate daml in test-daml-version-and-install.sh

* Bound versions.is_at_least("2.7.5") for test-daml-version-and-install.sh

run-full-compat: true

* Add logged diffing to diagnose issue

run-full-compat: true

* Add diff to data of test-daml-version-and-install.sh

run-full-compat: true

* Fix `daml install` call breaking - cache should still be refreshed

* Address minor issues: Daml 10.x.y matching, quoting, wrong comment

run-full-compat: true

* Drop printing

* Improve logging for test-daml-version-and-install.sh

* Fix undercased variable name DAML_CACHE, rerun CI

run-full-compat: true

* disable on windows

run-full-compat: true
2023-10-24 11:48:11 +00:00
dylant-da
a0fe6fe532
Fix version caching (#17581)
* Make at least one call API if there is no cache

* document why we don't use /repos/{owner}/{repo}/releases/latest

* Refactoring

* Improve name - envLatestStableSdkVersion is only used in one location

* `daml install latest` overrides update-check in daml-config.yaml

* Improve comments for cacheAvailableSdkVersions

* lint

* Drop lookup for UpdateCheckNever

https://dev.azure.com/digitalasset/daml/_build/results?buildId=151669&view=logs&jobId=870bb40c-6da0-5bff-67ed-547f10fa97f2&j=870bb40c-6da0-5bff-67ed-547f10fa97f2&t=fdc575a5-c2b4-5f52-8465-2c7aa3e12574
reports
daml: Snapshot versions list from https://api.github.com/repos/digital-asset/daml/releases does not contain valid JSON
  context: Fetching list of available SDK versions
  details: Error in $: expected Array, but encountered Object

I assume this is because the request returned some sort of failure
2023-10-17 17:32:05 +01:00
Gary Verhaegen
4c1631e89c
fix get-daml for macOS/BSD sed (#17562) 2023-10-16 08:47:46 +00:00
Samuel Williams
03bb8bfee2
Update daml-assistant to support new versioning (#17553) 2023-10-12 09:12:49 +01:00
Gary Verhaegen
3ff37fc0b6
get.daml.com: support dissociated daml <> sdk versions (#17550) 2023-10-11 18:14:56 +02:00
Samuel Williams
5eb2696deb
Implement multi-package.yaml CLI (#17267)
* First working model (requires ghcide changes)

* Bug fixes, add logging

* Use pinned ghcide

* Comment up a storm

* Add multi sdk version support

* Implement project.yaml approach

* Cleanup

* Add projects to multi-package.yaml

* Add flags for build all projects in multi-project, and no caching

* Delete ghcide

* Start migration from daml multi-build to daml build

* Add non --all daml build flow

* Finish migration over to daml build

* Add comments

* Change --all and --no-cache to switches

* Fix caching issue with source set to .

* Address initial comments

* Add daml.yaml source as a file caching warning

* Forward sub-assistant call errors to multi-build

* Update comment

* Update merged ghcide
2023-10-11 13:50:55 +01:00
Paul Brauner
a5cd445d3b
Introduce LF2 major version (#17348)
- Introduces a new major version, "2", in the daml_lf proto
- Adds new major versions to the compiler and the engine
- Updates all code that assumes only one major version
- Updates all code that assumes only one dev version
2023-09-13 10:01:42 +02:00
Gary Verhaegen
4bf7693f42
fix cache (#17377)
Looks like the issue was Bazel 5.2.0.
2023-09-08 10:31:42 +02:00
Moisés Ackerman
6ae0e0111e
Upgrade to Bazel 5 (again) (#17319)
* Update to rules_haskell v0.16
* Update comments re bazel patches
* clean up bazel overrides
* Upgrade to Bazel 5.2.0
* Remove '--distinct_host_configuration=false'
* Update buildifier to 6.3.2
* Suffix macos and ubuntu caches with yyyymm
* bump windows cache to v14
* [REVERTME] bump linux/macos/darwin timeout to 4h
2023-09-01 15:15:36 +02:00
Samuel Williams
9179b2e7d0
Jar licensing (#17299)
* Add _distribute.jar target with correct license
Use it everywhere

* Fixes from review

* Change NOTICES to NOTICES.txt for consistency
2023-08-22 15:56:29 +01:00
azure-pipelines[bot]
2f4b494fa5
bump canton to 2.8.0-snapshot.20230804.10940.0.ve26becd2 (#17236)
bump canton to 2.8.0-snapshot.20230804.10940.0.ve26becd2

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2023-08-06 13:32:51 +02:00
dylant-da
760f879e35
Extract available snapshot versions from Github (#17029)
* Extract available snapshot versions from Github

* Refactor slightly, add comments

* Implement caching

* Make getLatest commands query the minimum number of times from Github

* lint

* filter out prereleases, fix naming

* Fix outdated reference `extractVersionsFromSnapshots` to extractReleases

* Retrieve all versions so that getLatest can prune old versions' patches

* Hide --snapshots, --all, and --force-reload flags from daml version

* Get isPrerelease check correct way around

* Thread UseCache to `daml install` and other commands

* Refactor getEnvLatestStableSdkVersion to defer sdk version extraction

* store envLatestStableSdkVersion thunk in struct so tests can override it

* Read environment var early in getLatestStableSdkVersion

I implemented this with `overrideWithEnvVarMaybe`, but it required some
rather opaque abuse of `fmap pure`, `sequence`, and `join`

* Replace hidden with internal so flags don't show up in help
2023-06-28 12:04:00 +00:00
Gary Verhaegen
f6a51dd083
clean up refresh-get-daml (#16994) 2023-06-15 09:49:07 +01:00
Gary Verhaegen
ed202d1861
refresh-get-daml-com: add invalidation & format (#16993)
* refresh-get-daml-com: add invalidation & format

* test update
2023-06-14 18:54:52 +00:00
Krisztian Pinter
fed6e1ab7a
Add cabal support for some components (#16755) 2023-06-05 17:07:31 +02:00
Gary Verhaegen
de6f31c4d8
canton: allow depending on arbitrary sha (#16926) 2023-06-01 15:06:35 +00:00
dylant-da
6058070637
Remove daml visualize / damlc visual (#16901)
* Remove daml visualize / damlc visual

* fix redundant imports

* remove visual-web subtest
2023-05-30 10:50:36 +01:00
Moisés Ackerman
e71c667778
Extract daml-dar-reader (#16786) 2023-05-02 16:17:23 +00:00
Samuel Williams
b9f3a787e0
Increase port timeout and cpu reserve for canton tests (#16742)
* Increase port timeout and cpu reserve for canton tests

* Fix duplicate tag

* Update BUILD.bazel
2023-04-25 11:41:26 +01:00
Remy
2f27f987ad
Increase daml-assistant GRPC timeout (#16751)
10s is definitively too short for the ledger to upload a "big" dar
especially when on load. We bump it to 60s following the one uses in
the scenario service.
2023-04-24 17:20:05 +02:00
Samuel Williams
52b9193631
Convert all daml-helper test suites to canton (#16722)
* Convert all daml-helper test suites to canton
2023-04-21 09:19:43 +01:00
Samuel Williams
3121c95754
Move tests from sandbox to canton (#16640)
* Implement FreePort for Linux

* Refactor FreePort a little, add withCantonSandbox, convert tests

* Implement dynamic port range for windows

* Import readProcess

* Log macOS info

* Switch to locateRunfiles

* Add sysctl for macos

* Bracket correctly

* Change dir to mainworkspace

* Implement MacOS dynamic port range

* Improve regex matches
2023-04-05 15:28:43 +01:00
atriantafyllos-da
e58ea28601
Upgraded circe-yaml to 15.0-RC1. [#16412]. (#16415) 2023-03-13 14:51:43 +01:00
mziolekda
1a3980bf89
Separate test-common from the rest of the ledger code (#16307)
* move files

* format

* wrong file

* fix conformance tests

* add readme

* fix compatibility tests

* fix haskell tests
2023-02-14 15:29:13 +00:00
Moisés Ackerman
bc79f15ae0
[Docs] Introduction to interfaces (#15837)
* Add Cash template

* Add NFT template

* Add Asset interface

* Add interface instance Asset for Cash

* Add interface instance Asset for NFT

* Add tests for Asset interface instances

* add comment for RejectTransfer/WithdrawProposal

* Use unqualified 'view' function in 'view' definition

* Prefix interface types with I and view types with V

* Use verbose names for templates and choices

* Add example for queryInterface

* Add example for queryInterfaceContractId

* Define tests as functions taking 'Parties'

* Define cashAndNftTest reusing mkAssetTest

* cashAndNftTest checks the return value of queryInterface directly

* Use 'debug' on the result of 'queryInterface{,ContractId}'

* Interface instances

* capitalize names

* Polymorphic Daml Script

* Add hint about serializable types under viewtype

Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: carrielaben-da <91496516+carrielaben-da@users.noreply.github.com>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2023-01-12 12:11:53 +01:00
Gary Verhaegen
151e12b81a
bump copyright (#16002)
This is the result of:

- Updating `./COPY` to say `2023`.
- Running `./dev-env/bin/dade-copyright-headers update .`
2023-01-04 18:21:15 +01:00
Chun Lok Ling
8bcd24f945
LineBuffering for stdout, stderr in Assistant/daml-helper (#15931)
* LineBuffering for stdout, stderr

* move to the top

* address comment
2022-12-19 19:40:00 +00:00
Gary Verhaegen
2a796c412c
remove dead code (#15739)
Follow-up on #15373.
2022-11-30 12:06:46 +00:00
Gary Verhaegen
e8e9f5dc36
fix gsg on node 18 (#15373)
CHANGELOG_BEGIN
CHANGELOG_END
2022-11-28 11:52:47 +01:00
nickchapman-da
6e379dbd5a
[script] update doc for daml-script/interfaces (#15683)
* update reference doc (doc-comments in Script.daml) for daml-script/interfaces

* small english fixes

* update user doc and example for daml-script/interfaces

* build/test ScriptExample.daml at 1.15

* dont show party in interface info message

* fix for: //daml-script/export/integration-tests/golden:matches-docs-test

* disable failing integration test
2022-11-28 08:18:44 +00:00
Gary Verhaegen
b749828b43
update & pin puppeteer dependencies for gsg test (#15584)
Not that I relish the added flakiness on CI, but I want to be able to
run those tests locally.
2022-11-25 00:23:56 +01:00
Moisés Ackerman
06892a4ae9
Clean up uses of locateRunfiles (#15446)
* Define DA.Bazel.Runfiles.locateResource as an alternative to locateRunfiles

* Use locateResource to get default dlint rules file

Fixes #15443

* Use locateResource to get scenario-service.jar

* Use locateResource to get ghc-pkg executable

* Use locateResource to get stable-packages directory

* Use locateResource to get ghcversion.h

* Use locateResource to get hpp executable

* Use locateResource to get builtin package dbs

* Drop unused function DA.Daml.Options.Types.getBaseDir

* Use locateResource to get d3{,plus}.min.js

* Use locateResource to get repl-service.jar

* data Resource now takes a runfilesPathPrefix instead of a runfilesPath

* Add hlint warning 'locateRunfiles/package_app'

* Let locateRunfiles crash when used outside of bazel target

* typo
2022-11-10 13:20:43 +01:00
Stefano Baghino
5d6a0355a3
Specify that parties are allocated only if necessary (#15441)
Makes the behavior of `daml ledger allocate-parties` more explicit
2022-11-04 12:53:03 +00:00
Gary Verhaegen
dd5543eaf2
create-daml-app: make tests pass on m1 (#15031)
This is very much a "just make it work" type of patch. The main issues
were:

- A bunch of type errors in the `index.test.ts` file when opened in VS
  Code. I'm not sure how it happened, but it's possible I'm just using a
  more recent TS compiler than whoever edited this last.
- `localhost`, on my machine at least, resolves to `::1`, and therefore
  the connections were just not established. I've opted for replacing
  `localhost` with explicit `127.0.0.1`.
- `npm-cli.js` was not an executable in my `PATH`, and won't be for
  most users, but `npm` is not on our testing infrastructure's `PATH`.

Those tests are obviously not tested by CI, so please review carefully /
test this on your own machine before approving.

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-23 04:57:47 +02:00
dylant-da
5a3cb468f8
Make damlc options fail if used more than once (#15021)
* Make damlc options fail if used more than once

* Fix import issues with optparse-applicative

* Undo `optionOnce` conversion for arguments that should occur many times

* satisfy changelog check

CHANGELOG_BEGIN
CHANGELOG_END

* Improve comments

* Add tests for optionOnce behaviour

* lint
2022-09-21 09:26:59 +00:00
Chun Lok Ling
15c0610d63
Do not send archive commands if there is no active contract (#14978)
* do not send archive commands if there is no active contract

CHANGELOG_BEGIN
CHANGELOG_END

* use null instead of == []

CHANGELOG_BEGIN
CHANGELOG_END

* mis-spelling correction.

* let me try

* let me try again
2022-09-09 17:52:32 +01:00