* Translate unsupported kinds to a special Erased type
This should simplify `data-dependencies` and avoid issues like #4470
since we can match on the type instead of having to guess which types
can and which cannot be translated back to DAML.
changelog_begin
changelog_end
* Switch to HashSet/HashMap for NormalizedUri/NormalizedFilePath
This matches the change to ghcide in
https://github.com/digital-asset/ghcide/pull/420. Now that we have
optimized Hashable instances it makes sense to use them as much as
possible.
changelog_begin
changelog_end
* debug windows crap
* Fix a bug in vr scenario notifications
* Revert "debug windows crap"
This reverts commit f58fdb92c1.
* attempt to fix windows
who are we kidding, this is not going to work
* expand documentation on WebSockets subprotocols to a more-visible section
CHANGELOG_BEGIN
CHANGELOG_END
* combine reference in websocket section to auth section
* resection
* subprotocols sometimes called protocols
- suggested by @hurryabit; thanks
* Extract partial functions for DABL to re-use
CHANGELOG_BEGIN
CHANGELOG_END
* Actually DABL needs total functions,
they will map them to different endpoints
* sandbox: Don't hold on to old resources when resetting.
Now there's one hell of a memory leak.
CHANGELOG_BEGIN
- [Sandbox] Fixed a memory leak when using the ResetService; not
everything was cleaned up correctly.
CHANGELOG_END
* sandbox: Split out SandboxClientResource from SandboxServerResource.
Gonna replace SandboxServerResource with a ResourceOwner acquisition.
* sandbox: Don't capture the API server in the SandboxServer resource.
When we reset, this is stored forever, leading to a memory leak.
Tested by rewriting the SandboxServerResource to use
`SandboxServer.owner`.
* sandbox: Revert the test client resource to calling `shutdownNow()`.
* sandbox: Make sure the fixture is recreated properly on each test run.
* sandbox: Make `SandboxState` a non-case class.
The `toString()` was unnecessarily heavy.
* sandbox: Futures, futures everywhere.
Avoid a race condition where the server is stopped before it starts by
storing a `Future[SandboxState]` rather than the `SandboxState` itself.
This doesn't trigger the same memory leak as storing a
`Resource[SandboxState]` because we don't capture the object itself in
the `flatMap` in the same way with `Future`.
* sandbox: Remove an unused parameter left in for debugging.
* sandbox: Replace `@VisibleForTesting` with a comment.
* sandbox: Add more comments to the weird logic in SandboxServer.
* sandbox: Get rid of the `Port` type alias; it was confusing.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Shuffle participants used in tests
* Add no-wait-for-parties flag to the test tool
This flag is needed for privacy-preserving ledgers that do not expose
party allocations to non-submitting participants.
* Add shuffle-participants flag to test-tool
CHANGELOG_BEGIN
CHANGELOG_END
* Address code review. Make --no-wait-for-parties hidden.
We've created a cool video that happens to have the wrong link in it
(missin l in the extension), and changing videos is more expensive than
changing text files.
Also, the current URL is a tad long, so I've added a shorter one.
Note: as this is part of the docs bundle (actually generating an HTML
redirect page), this won't be live until next version is published.
CHANGELOG_BEGIN
CHANGELOG_END
* Support multiple-packages in `damlc ide`
changelog_begin
- [DAML Studio] You can now open DAML Studio in the root of a
multi-package project instead of opening it separately for each
package. Take a look at the documentation for details on how to set
this up.
changelog_end
There are a few caveats here:
1. You need a ``daml.yaml`` in the root of your project directory. I
think this is somewhat sensible but we should add a warning to VSCode
if you open it in a directory that does not have a ``daml.yaml`` (in a
separate PR).
2. Changes are not picked up accross dependencies. This is a larger
undertaking and given the current setup simply impossible (we don’t
know that the source files of one package belong to the DAR referenced
in the ``dependencies`` field of the other package. We can make this a
bit better by at least detecting that the ``.dar`` has changed but
let’s do that separately.
3. Since ``daml init`` runs once on startup, it will run in the root
directory instead of initializing the package db of the individual
packages. This is fixable but will conflict with #4391 so let’s
address this separately.
I’ve added docs to the daml studio section that explain the caveats.
* Use the proper sdk version in lsp-tests
`fakeDynFlags` has been removed from ghcide. We still need it since we
set up our environment in weird ways and want it to be platform
independent so this PR just copies the definition into `daml`.
changelog_begin
changelog_end
Not sure why the typechecker didn't catch this one.
I'm assuming this was changing an unused `laoding` field and the real
`loading` field was never used (or inaccurate) for `fetchByKeyResults`.
changelog_begin
changelog_end
* ci: temp machines for scheduled killing experiment
Based on our discussions last week, I am exploring ways to move us to
permanent machines instead of preemptible ones. This should drastically
reduce the number of "cancelled" jobs.
The end goal is to have:
1. An instance group (per OS) that defines the actual CI nodes; this
would be pretty much the same as the existing ones, but with
`preemptible` set to false.
2. A separate machine that, on a cron (say at 4AM UTC), destroys all the
CI nodes.
The hope is that the group managers, which are set to maintain 10 nodes,
will then recreate the "missing" nodes using their normal starting
procedure.
However, there are a lot of unknowns I would like to explore, and I need
a playground for that. This is where this PR comes in. As it stands, it
creates one "killer" machine and a temporary group manager. I will use
these to experiment with the GCP API in various ways without interfering
with the real CI nodes.
This experimentation will likely require multiple `terraform apply` with
multiple different versions of the associated files, as well as
connecting to the machines and running various commands directly from
them. I will ensure all of that only affects the new machines created as
part of this PR, and therefore believe we do not need to go through a
separate round of approval for each change.
Once I have finished experimenting, I will create a new PR to clean up
the temporary resources created with this one and hopefully set up a
more permanent solution.
CHANGELOG_BEGIN
CHANGELOG_END
* add missing zone for killer instance
* add compute scope to killer
* authorize Terraform to shutdown killer to update it
* change in plans: use a service account instead
* .
* add compute.instances.list permission
* add compute.instances.delete permission
* add cron script
* obligatory round of extra escaping
* fix PATH issue & crontab format
* smaller machine & less frequent reboots
In a committed transaction (i.e. a transaction with only absolute
contract ids), local divulgence is a non-concept.
Therefore local divulgences don't need to be treated and all code
related to it can be removed.
This also removes some duplication between InMemoryLedger and SqlLedger.
CHANGELOG_BEGIN
CHANGELOG_END
* move InsertDeleteStep to util
* turn append cid argument into a typeclass
* use Cid for appendForgettingDeletes as well
* add secondary data to the deletes in InsertDeleteStep with new covariant tparam
* make StepAndErrors hold ArchivedContracts so ws archives look like exercise archives
* add changelog
CHANGELOG_BEGIN
- [JSON API - Experimental] The format of ``archived`` responses from WebSocket endpoints
has changed to include template IDs, similar to exercise responses.
See `issue #4383 <https://github.com/digital-asset/daml/issues/4383>`_.
CHANGELOG_END
* Update CI nix version
For `--option http2 false` to take effect requires Nix 2.3.2.
CHANGELOG_BEGIN
CHANGELOG_END
* Set option `http2 = false` dev-env nix config
This is less likely to overlook an instance than manually adding
`--option http2 false` to each Nix invocation.
Setting `--option htt2p false` also had no effect on the multi-user Nix
installation on the Linux CI machines due to
```
WARNING: option '--disk_cache' was expanded to from both option '--config linux' (source /nix/store/2xnfb2l39d2b4nxw5vwmqz5hjwhw0caw-daml-bazelrc) and option '--config linux' (source /nix/store/2xnfb2l39d2b4nxw5vwmqz5hjwhw0caw-daml-bazelrc)
```
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Fix indexing of transactions referencing divulged contracts
When a transaction referenced a divulged contract that had no contract
metadata (e.g. it was only known to the participant via divulgence),
the indexer failed with InactiveDependencyError.
This fixes the issue by modifying lookupContractLet query to return
LetUnknown if the contract is known, but has no metadata.
No changelog entry to add as this bug did not affect any users as this
surfaced with privacy extensions. Canton not affected as they always
include divulged contracts, even if they already have been divulged.
CHANGELOG_BEGIN
CHANGELOG_END
* Failing test-case for lookupContractLet
* Address PR review
* Second pass addressing review
As part of our SOC2/ISO certification, we need to be able to evidence a
list of "Standard Changes" for the DAML SDK project. This commit adds a
script that extracts, for a given month, the list of Standard Changes
that happened along with relevant information (author, reviewer, etc.).
It also adds a definition for a monthly cron job on Azure to run the
script and send the result to Slack, @-mentioning Martin.
CHANGELOG_BEGIN
CHANGELOG_END
* Prevent building of en/de-coders of Cid, Val, and Nid outside Value/Transaction Coder
* DAML-LF: remove constrains on Cid for LF encoder/decoder
* Address Samir coments
* rename "contracts" to "events" in JSON API exercise response
CHANGELOG_BEGIN
- [JSON API - Experimental] Exercise response field "contracts" renamed to "events".
See `issue #4385 <https://github.com/digital-asset/daml/issues/4385>`_.
CHANGELOG_END
* more events in doc
- pointed out by @leo-da; thanks
* Remove phantom archive when streaming events filtered by contract key
* Add fetch test cases, WIP
* Add fetch test cases, WIP
* Add fetch test cases, WIP
* Add fetch test cases, WIP
* Test case to validate phantom archive filter for stream/fetch
* Use `scan` instead of `statefulMapConcat` to filter out phantom archives
advantage(???) immutable state vs mutable
* make `com.digitalasset.http.WebsocketEndpoints.handleWebsocketRequest` public
so DABL can call it directly, `private[http]` does not work since DABL
is under com.projectdabl, see #4190
* Fixing typos
* Scalafmt
* Scalafmt
* Removing unused function
* Update docs
* Remove unused type alias
* CHANGELOG_BEGIN
[JSON API - Experimental]
- Added streaming version of fetch by key: ``/stream/fetch``. See #4075.
CHANGELOG_END
* Address code review comments
* Add test case with todo to address consistent handling of empty requests
* Fixing merge conflicts
* Addressing code review comments
* Addressing code review comments
Use `partition` instead of consecutive intersect and diff
We need to use runActionSync everywhere except for the IDE. Pretty
sure I’ve fixed all cases of those. We also need to use useE instead
of use_ since apparently exceptions shortcircuit shake meaning that
runActionSync doesn’t actually do sync. I’ve only fixed the cases that
we hit in the tests so we probably should do a more thorough pass
after this (I want to get this merged soon since it is quite flaky on
master).
changelog_begin
changelog_end
[JSON API - Experimental]
Added a command line option to allow overriding default TTL.
``--default-ttl <value>`` Optional Time to Live interval to set if not provided in the command. Examples: 30s, 1min, 1h. Defaults to 30 seconds
CHANGELOG_END
* resources: Allow for startup exceptions without no stack trace.
* sandbox: Remove dependency duplication in the BUILD file.
* ledger-on-(memory|sql): Extract `Main` into its own file tree.
This is so we can remove the dependency on kvutils/app from the library.
logback.xml goes with it.
* sandbox: Implement "Sandbox Next" on top of ledger-on-sql.
CHANGELOG_BEGIN
- [Sandbox] Preliminary work to rebuild Sandbox on top of the DAML
Ledger Integration Kit. Currently not exposed through the CLI.
CHANGELOG_END
* ledger-on-sql: Publish to Maven Central.
* sandbox: If an unknown database is passed to Sandbox Next, handle it.
* resources: Catch errors in owner construction or acquisition.
Sometimes we're not in a future yet.
* resources: Don't run ProgramResource error handling inside the EC.
Otherwise we end up in deadlock which is only broken by
`awaitTermination` timing out.
* add SearchForeverRequest with one-or-many JSON model
* the least structured way to flatten a query union into a single stream
* add somewhere for indices to go in StreamQuery
* unzipMap utility
* doc Positive
* collect indices for SearchForever predicate, and use only one Map
* interface for rendering positions
* finish propagating positives from the predicate to the rendering phase
* add matchedQueries to every `created` in the searchForever results
* test that matchedQueries indices are included in query stream blocks
* document query union
* add changelog
CHANGELOG_BEGIN
- [JSON API - Experimental] ``/contracts/searchForever`` accepts multiple queries,
and includes with each ``created`` result the ``matchedQueries`` indicating which
queries matched.
See `issue #4363 <https://github.com/digital-asset/daml/pull/4363>`_.
CHANGELOG_END
* remove unused unzipMap
* Disable http2 with Nix to work around segfaults
CHANGELOG_BEGIN
CHANGELOG_END
* Disable http2 in dev-env calls to nix-build as well
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Added checks for unit id conflicts.
changelog_begin
changelog_end
* Only report conflicting keys
* lint
* fix indentation
* Show conflicting package ids also
I’ve spend several hours trying to get them to fail on Linux and
Windows and have failed completely (apart from making my machine roun
out of memory, thanks crashplan). I also inserted explicit delays to
make the failure more often frequent without any sucess (at making it
fail). I don’t really have a satisfying explanation of why this is
gone now. My best guess is that the `ghcide` upgrade fixed this since
it included some fixes and changes around how we kill and start the
shake process.
fixes#4328
changelog_begin
changelog_end
* sandbox: Fix RecoveringIndexer so it doesn't wait too long.
If the indexer restart delay is too short, RecoveringIndexer will still
wait a second. This is not an issue in production, because the delay is
typically 10 seconds, but in tests, where we use 100 milliseconds, it's
enough to make the test flaky.
We need to wait in one-second increments to allow for quick
cancellation; we don't want the user to have to wait 10 seconds if the
process is terminated while restarting.
The fix is to take the minimum of 1 second and the remaining restart
time.
This also adds STDOUT logging back into the tests; it was invaluable
when debugging the issue.
I am not including a changelog entry, because this only affects tests.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Add an assertion in IndexerIT to ensure it stops quickly.
We already test the logs, but it's probably wise to test that the
indexer doesn't wait 10 seconds.
* Enhanced completions for the assistant
changelog_begin
- [DAML Assistant] The assistant can now do completions for SDK
commands, e.g., ``daml ledger upl<TAB>`` will complete to ``daml
ledger upload-dar``.
changelog_end
* Apply suggestions from code review
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
It turns out that various environments (hello docker) start with a
closed stdin. This means that things like `daml sandbox` exit
immediately. Since users that care about the new behavior are likely
to know about it, we only enable it if a specific flag is given.
changelog_begin
- [DAML Assistant] The new behavior introduced in ``0.13.51`` to shut
down when stdin is closed is now disabled unless you explicitly
enable it by passing ``--shutdown-stdin-close``.
changelog_end
* Explicitly add readme to packages
* npm packages repository metadata
As specified in https://docs.npmjs.com/files/package.json#repository
CHANGELOG_BEGIN
CHANGELOG_END
* Bazel format
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
changelog_begin
- [DAML Script - Experimental] Add a sleep funciton that pauses
the script for the given duration. This is primarily useful in tests
where you repeatedly call query until a certain state is
reached.
changelog_end
fixes#4199
