* Implement FreePort for Linux
* Refactor FreePort a little, add withCantonSandbox, convert tests
* Implement dynamic port range for windows
* Import readProcess
* Log macOS info
* Switch to locateRunfiles
* Add sysctl for macos
* Bracket correctly
* Change dir to mainworkspace
* Implement MacOS dynamic port range
* Improve regex matches
* Define DA.Bazel.Runfiles.locateResource as an alternative to locateRunfiles
* Use locateResource to get default dlint rules file
Fixes#15443
* Use locateResource to get scenario-service.jar
* Use locateResource to get ghc-pkg executable
* Use locateResource to get stable-packages directory
* Use locateResource to get ghcversion.h
* Use locateResource to get hpp executable
* Use locateResource to get builtin package dbs
* Drop unused function DA.Daml.Options.Types.getBaseDir
* Use locateResource to get d3{,plus}.min.js
* Use locateResource to get repl-service.jar
* data Resource now takes a runfilesPathPrefix instead of a runfilesPath
* Add hlint warning 'locateRunfiles/package_app'
* Let locateRunfiles crash when used outside of bazel target
* typo
* Remove reliance on implict party allocation in the authorization tests of the sandbox-on-x
CHANGELOG_BEGIN
CHANGELOG_END
* Remove the HOCON and cli config parameters
* Remove implicit party allocation from the Submission Service
* Remove support for implicit parties from the bridge
* Fix codegen reliance on implicit party allocation
* Fix daml script tests
* Fix trigger tests
* format security-evidence.md post rebase
* Fix json-api tests
* One more json-api test
* Fix haskell tests
* Corrections post-review
* Cosmetic improvements
* Fix race condition in party allocation handling
* Removal of Sandbox Classic in Haskell code.
CHANGELOG_BEGIN
CHANGELOG_END
* Further sandbox-classic removals
* Upload dar files when withSandbox is used.
* fix formatting
* Moved upload of dars in createSandbox.
* Fixed test by passing the token before the token file is created.
* Added implicit party allocation argument for haskell tests.
* DPP-1073 Infer sandbox token from the shared secret.
* Different slicing of runLedgerUploadDar
Co-authored-by: Andreas Triantafyllos <andreas.triantafyllos@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Migrate Haskell tests to Sandbox on X
The diff here looks extremely confusing. sandbox-classic is now
sandbox on x not sandbox classic so this isn’t a typo. It is really
moving to sandbox on x.
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* Add timeouts to polling funcs in daml start.
This PR adds timeouts to some polling functions used in daml start and
the assistant integration tests, and also early exits based on a process
exit status. E.g. waitForHttpServer will make sure some process is
still running, instead of waiting to timeout.
The effect of this is that now whenever there is some error in a
subprocess, daml start and the integration tests will finish early
instead of running forever (or timing out in bazel).
changelog_begin
changelog_end
* missing a readPortFile instance
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* Use the token from incoming requests to update the package list
changelog_begin
changelog_end
* Lazily initialize the ledger client
* Fix ee integration tests
* Fix package reloading behaviour by using a semaphore to check for ongoing updates
* Refactor out the semaphore code into a concurrency utility class
* Use correct locking for the updateTask so every thread always uses an up to date task
* Remove unused imports in utils.Concurrent & remove packages from the tests
* Hide & mark the token file cli option deprecated because we dont need it anymore and only keep it so client deployment code doesn't break
* Fix scala 2.12 build by adding more type annotations
* Update ledger-service/http-json-cli/src/main/scala/com/daml/http/OptionParser.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/PackageService.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Readd pgkManagementClient after it was removed accidentally (but now it's lazy)
* Remove concurrent object & use atomic boolean instead of a mutex because it makes more sense
* Replace semaphore with countdownlatch
* Refactor the caching into a separate class
* Use Instant instead of LocalDateTime
* Remove that ** of bad synchonization and do stupid simple synchronization because it JUST WORKS, besides adapt when we want to reload
* Remove await in tests because it can result in buggy tests
* remove unused code in WebSocketService.scala
* Unhide the access-token-file option as per request of Stefano
* Less implicit jwts per request of Stefano
* Try making some code more readable as by request of Akshay
* Use more shark because it expresses better than flatMaps if I don't need the arg
* Move defs in predicate in WebsocketService.scala around
* Try to minimize diff further in WebsocketService.scala
* Fix build and minimize diff in WebSocketService.scala further
* Minimize diff of function getTransactionSourceForParty in WebSocketService.scala
* Share the ec in WebSocketService.scala to minimize the diff
* Minimize in function predicate in WebSocketService.scala
* Further minimize in function predicate in WebSocketService.scala
* Change some case classes to be normal classes but with apply method
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/PackageService.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Get rid of implicit jwt tokens, the world is already confusing and full of implicits enough
* Improve readability
* Integrate the new LedgerClient which does not depend on a leder id
* Fix tests
* Apply suggestions from code review
thanks to @S11001001
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Apply further review comments
* Remove outcommented code
* Deprecate access token file option in the description too
changelog_begin
- [JSON API] The cli option `--access-token-file` is now deprecated. It
is not needed anymore and you can safely remove it. Reason is that
the operations which prior required a token at startup are now done
on demand using the token of the incoming request.
changelog_end
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Parallelize daml-assistant integration tests.
This mainly involves avoiding changing the working directory or the environment while inside tests.
AFAIK the daml start tests can't be parallelized without starting up a sandbox instance each time, which seems bad. These tests could be separated from the rest...
The magic number "2" was picked via experimentation:
```
1 threads -> 306 s
2 threads -> 199 s
3 threads -> 198 s
4 threads -> 195 s
```
changelog_begin
changelog_end
* buildifier-fix
Each of these installations is about 1gb so it’s not that unlikely
that this plays a significant factor in our out of disk space errors.
Although I think we cleanup partially so I’m not sure how large the
part of the SDK installation is that gets leftover.
changelog_begin
changelog_end
* daml build: add a --access-token-file for remote dependencies
This adds a `ledger.access-token-file` field in the `daml.yaml` project
file and a `--access-token-file` flag to `daml build` to authorize
querying/fetching of remote dependencies.
CHANGELOG_BEGIN
[daml build] A new flag `--access-token-file` is added for the `daml
build` command. It allows the specify the path to an access token to
authenticate against the ledger API. This is needed if the project
depends on a remote Daml package hosted on such a ledger. Alternatively,
the path to the token can also be specified in the `daml.yaml` project
file under the `ledger.access-token-file` field.
CHANGELOG_END
* Merge Maven uploads for different Scala versions
It turns out Maven will abort an existing staging operation if you
create a new one. This means our jobs race against each other. We
could try to fix that by either sequencing the jobs in a clever
way (annoying and can break things like rerunning if only parts
failed), or by creating more profiles (unclear if you can even have
two profiles for the same group id, even if you do, it’s annoying to
merge).
So in this PR I (grudgingly) merged both uploads into the Haskell
script. This isn’t all bad:
1. It moves some logic from bash embedded in yaml string literals into
Haskell code.
2. It duplicates some versions but it removes duplication in other
places so overall not too much worse.
3. It does however, make things slower. We don’t run this stuff in
parallel. That said, the release step is relatively small (< 5min) and
it only runs on Linux.
We could add CLI arguments to make the Scala versions configurable for
local development. Given that this is blocking releases, I wanted to
get something in that works first and then see what we need in that regard.
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* daml ledger: Infrastructure for tests
This adds infrastructure for testing `daml ledger` and moves one test
from the assistant integration tests to the newly created tests.
CHANGELOG_BEGIN
CHANGELOG_END
* Update libs-haskell/test-utils/DA/Test/HttpJson.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Update libs-haskell/test-utils/DA/Test/HttpJson.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* small fixes
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
This gives us visibility into issues w/ sandbox in the
tls/deployment/ledger tests. This change only affects test
output.
changelog_begin
changelog_end
This limits the JVM max memory and initial memory in a few tests that
look like they might be using more than they have two and that run for
a long time so there is a high chance they end up running in parallel
with something else.
changelog_begin
changelog_end
* Fix leakage of SDK installations
This fixes an issue where the integration tests leaked SDK
installations. Easily verified by `ls /tmp/extra-dir*` before and
after running the tests.
changelog_begin
changelog_end
* I hate windows so much
changelog_begin
changelog_end
* Haskell: Add assertFileExists to DA.Test.Util
This PR adds a helper function `assertFileExists` that captures the
`doesFileExist ... >>= assertBool ...` pattern that is very common in
our Haskell test suites. It also adds the inverse
`assertFileDoesNotExist` function. Both functions are now used where
appropriate.
CHANGELOG_BEGIN
CHANGELOG_END
* Add directory dependency to test-util lib
CHANGELOG_BEGIN
CHANGELOG_END
* Split up repl tests to make them faster
This PR splits up the tests into the tests for TLS and Auth and the
tests for the actual functionality.
The func tests use the repl as a library which allows them to be
significantly faster:
1. We only need to start the service process once.
2. We only need to initialize the package db once.
There is at least one other point that I did not address for now and
that is only loading the packages into the repl service once. While
loading them multiple times is a noop, it still has a performance
implication.
Sadly, this has turned out much more messy than I thought it would be
due to various issues with haskeline/repline/tasty/computers. The
details are in a comment in DA.Test.Repl.FuncTests.
changelog_begin
changelog_end
* Try to nuke cache on window selectively
* Enable cache again
- Move deployment tests (deployTest, fetchTest) out of integration-tests.
- Use DA.Test.Sandbox where appropriate.
- Split out code for useful test patterns: i.e. calling commands quietly, getFreePort.
changelog_begin
changelog_end
* Move 'withEnv', call it from daml2ts tests
changelog_begin
changelog_end
* Fix withEnv call to ensure that TASTY_NUM_THREADs is set
withEnv replaces the whole environment so we need to set everything we
care about.
* withEnv replaces the whole environment so we need to set everything we
care about.
* Apparently applying the same fix has destabilized Windows
* Try even harder to get daml assistant tests passing on Windows again
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This ensures that -l and -p work properly in the integration tests
since they no longer depend on the order.
There is lots of other crap to cleanup in those tests but I’m trying
to keep it to small steps.
changelog_begin
changelog_end
* Integrate create-daml-app into the assistant
fixes#4868
changelog_begin
- [DAML Assistant] Add a new ``daml create-daml-app`` command for creating a project based on
`create-daml-app <https://github.com/digital-asset/create-daml-app>`_.
changelog_end
* Try random things hoping to fix windows
* Try random things hoping to fix things on macos
* Support authentication and TLS in DAML repl
changelog_begin
- [DAML Repl - Experimental] You can now connect to a ledger via TLS
by passing ``--tls`` to ``daml repl``
- [DAML Repl - Experimental] You can now connect to a ledger with
authentication by passing the token via ``--access-token-file`` to
``daml repl``.
changelog_end
* try to fix linking on windows
* windows is weird
* gnah
We previously had 3 slightly different but consistently shitty logic
for handling this in the tests for daml-helper daml repl and the
Haskell ledger bindings. This PR introduces a module that is flexible
enough to capture all their needs and hopefully is somewhat less
shitty.
changelog_begin
changelog_end
