* Freeze DAML-LF 1.8 proto
This is a copy of the 1.dev proto with the following changes that
overall remove all references to 1.dev:
* Change the versions and the reference to the spec.
* Remove `dev` from version history at the top.
* Remove `GENMAP` and corresponding primitives.
* Remove general equality `EQUAL` and the comments that the
type-specific equality primitives are only available in < 1.dev
* Remove experimental text primitives.
changelog_begin
changelog_end
* windows is very bad
* bump windows hashes
Given that these two group of tests run for the same combination of LF
versions, I don’t see any reason why we should keep them
separate. Merging them, speeds up the packaging tests by > 50s on my
local machine.
changelog_begin
changelog_end
Rather than decoding everything twice, we can only rewrite references
which is measurably faster (> 5s out of 60s on the packaging tests).
changelog_begin
changelog_end
* libs-scala/ports: Wrap socket ports in a type, `Port`.
* sandbox: Use `Port` for the API server port, and propagate.
CHANGELOG_BEGIN
CHANGELOG_END
* extractor: Use `Port` for the server port.
* ports: Make Port a compile-time class only.
* ports: Allow port 0; it can be specified by a user.
* ports: Publish to Maven Central.
* kvutils: Make the `KeyValueParticipantStateReader` tests more rigorous.
If the `offset` is specified as `None`, expect it to be `None`, not
just anything.
* kvutils: Simplify `KeyValueParticipantStateReader#stateUpdates`.
Construct the Source with `Source.apply`, not `Source.fromIterator`.
* kvutils: Use multiple entry IDs in `KeyValueParticipantStateReaderSpec`.
* kvutils: Add basic tests to `KeyValueParticipantStateReaderSpec`.
* kvutils: Add heartbeats to `LedgerReader`'s `events` output.
Heartbeats are optional, to be delivered by the ledger if and when it
deems necessary.
* sandbox-next: An observing time service backend using Akka streams.
* sandbox-next: A regular heartbeat based on Akka Streams' `tick`.
* sandbox: Replace `TimeServiceBackend.withObserver` with `.observing`.
More code, but it's more decoupled, so can more easily be sent to the
underlying backend in Sandbox Next.
CHANGELOG_BEGIN
- [Sandbox] Fixed a bug in the command completions stream when running
Sandbox in static time. Previously, upon updating the time, the old
time was emitted on the completions stream. The new time is now
emitted.
CHANGELOG_END
* sandbox: TimeServiceBackend should only emit accepted changes.
* ledger-on-memory: Use `LedgerRecord` directly.
* ledger-on-memory: Stream heartbeats to the log.
* ledger-on-memory: Encapsulate mutations behind locks at all times.
* ledger-on-memory: Differentiate between reading and writing.
* ledger-on-memory: Factor out appending to the log.
* kvutils: Move the heartbeat test into the base from ledger-on-memory.
* kvutils: Log when the submission validation fails unexpectedly.
* ledger-on-sql: Add a script to hash all migrations.
* ledger-on-sql: Publish heartbeats to the log, and stream them out.
* ledger-on-sql: Log if publishing the heartbeat failed.
* ledger-on-sql: Wrap all queries in `Try`.
Just to make sure that we don't throw from a function that returns `Try`
or `Future`.
* ledger-on-sql: Allow `Long` values as the heartbeat timestamp.
`INTEGER` really does mean 32-bit, apparently.
* sandbox-next: Pipe heartbeats to the ledger.
* ledger-on-sql: Make sure we publish the correct head after a heartbeat.
Off-by-one errors are the best errors.
* ledger-on-(memory|sql): Just accept heartbeats, not their owner.
* sandbox: Update CIDs in tests to account for the extra heartbeat.
* ledger-on-memory: Fix a reference to variable in a comment.
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* ledger-on-sql: `flatMap` over `Try` rather than `Future` when possible.
* sandbox: Make sure the heartbeat queues are thread-safe.
* kvutils: Remove `LoggingContext` from the interfaces.
Keep it internally. This means we'll drop any context, but otherwise
things should work as expected.
* sandbox-next: Pull out the heartbeat interval into a constant.
* ledger-on-sql|sandbox: Clarify large levels of nesting.
Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
Rejected submissions are a user-error and don't indicate a problem with the server not functioning properly. Such user
errors make it hard to spot "real" server-side warnings and errors.
i
Closes#4772
CHANGELOG_BEGIN
- The Ledger API Server now logs rejected submissions at a lower "INFO" level to remove a source of warnings/errors without relation to server health.
CHANGELOG_END
Originally, we introduced this since it made fully cached builds
slightly faster. However, that was a long time ago. Looking at the
numbers now, things actually seem to be significantly faster without
this (possibly due to changes in how Bazel handles this). In
particular on MacOS where I’ve seen a couple of builds with < 20
minutes on CI which I did not see in quite some time without this.
changelog_begin
changelog_end
* Freeze DAML-LF 1.8
Two minor points that I did not mention in the previous PR:
We also include the renaming of structural records to `struct` and the
renaming of `Map` to `TextMap`.
There are some minor changes around the LF encoder tests which need to
be able to emit package metadata properly so I’ve added it to the
parser. Sorry for not splitting that out.
Following the process used for the DAML-LF 1.7 release, this does not
yet include the frozen proto file.
changelog_begin
- [DAML-LF] Release DAML-LF 1.8:
* Rename structural records to ``Struct``. Note that
structural records are not exposed in DAML.
* Rename ``Map`` to ``TextMap``.
* Add type synonyms. Note that type synonyms are not serializable.
* Add package metadata, i.e., package names and versions.
Note that the default output of ``damlc`` is stil DAML-LF 1.7. You
can produce DAML-LF 1.8 by passing ``--target=1.8``.
changelog_end
* Update encoder
* Update java codegen tests
* Update comment in scala codegen
* Handle TSynApp in interface reader
* Bump lf_stable_version to 1.7
* Fix kvutils tests
* Make kvutils work with the new contract id scheme
CHANGELOG_BEGIN
- [KVUtils] uses random contract id. Contract ids are made of 65 hexa decimal characters.
CHANGELOG_END
Co-authored-by: Jussi Mäki <jussi.maki@digitalasset.com>
* Tighten the loop: backend services to return API responses
CHANGELOG_BEGIN
CHANGELOG_END
* Use transaction filter directly
* Remove unnecessary transition through domain objects
* Ensure transient contract remover compares sets of witnesses
* Honor verbosity in request
* Address review https://github.com/digital-asset/daml/pull/4763#pullrequestreview-367012726
- using named parameters when creating the API objects
- renamed EventOps accessors to easily recognizable names
- dropped unnecessary usage of views
- honoring verbosity level in request in all places
- replaced usage of lenses with simple copying where it made sense
We will only include type synonyms and package metadata (which are
used for Cross-SDK model upgrades). Everything else stays in 1.dev for
now.
changelog_begin
changelog_end
This should hopefully fix the failure to delete the temporary zip file
on Windows. See https://github.com/mrkkrp/zip/pull/61
changelog_begin
changelog_end
This PR adds a first version of update documentation and removes the
existing docs which were talking about `damlc migrate`.
So far the docs focus on the high-level approach to upgrades taken in
DAML and give an example of how to structure upgrade contracts.
What is not covered so far (and I’d like to leave that for a separate
PR) is:
1. Technical details: How are things split up into packages, which
restrictions apply to data-dependencies, …
2. Deployment and Running the upgrade via triggers/daml script/…
3. Common patterns for handling this in UIs (e.g. “locking” old contracts)
changelog_begin
changelog_end
I’m not really happy with this “fix” but after having spend way too
much time on this, this was the best I came up with. (The details are
in an inline comment). If anyone has better ideas, I’m all ears.
changelog_begin
changelog_end
Replace `daml-lf-repl validate` in packaging tests with `damlc validate-dar`.
Simplify test setup a little by passing tools (damlc,validate, etc) in a record.
changelog_begin
changelog_end
* Use yarn installation link instead of homepage
* Indentation of Messages segment
* Update git link to downloads page
* Explain why you see friends of friends in the network
* Tweak wording
* Descibe how to run new app more explicitly
* Make change to ui folder more obvious
* Add next steps
changelog_begin
changelog_end
* Fix title levels
Includes a change that should reduce the size of runfiles manifest files
and runfiles trees for targets with runtime dependencies on the Python
toolchain.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
Let's see how far we get with #4745.
It's a bit of a shame I can't retry the same commit multiple times.
Maybe I should have accounted for that in the version format...
CHANGELOG_BEGIN
CHANGELOG_END
This has been running for a few days now and while I have seen a bunch
of these cases, I have not once received a message with a BACKOFF value
different from 512. This means that, likely due to some sort of internal
caching in Azure, retrying in this case is useless and just makes the
build failure take more time, i.e. more time before we can rerun.
Rerunning does usually solve it, though.
I have also noticed that we still get these notifications when the job
has been canceled, which usually means the user has force-pushed (in
which case it makes sense that the commit is no longer available). I'm
not sure we can detect this, but I take this opportunity to print the
JobStatus just in case.
CHANGELOG_BEGIN
CHANGELOG_END
* Update create-daml-app code
* Remove reference to infix elem
* Remove the licenses from the code we tell users to copy
* Tweak conclusion
changelog_begin
changelog_end
Somehow, in the current setup, the publish steps do not get executed on
master. This is what Azure reports:
```
Evaluating: and(succeeded(), eq('$(is_release)', 'true'),
eq(variables['Build.SourceBranchName'], 'master'), eq('linux', 'linux'))
Expanded: and(True, eq('$(is_release)', 'true'),
eq(variables['Build.SourceBranchName'], 'master'), eq('linux', 'linux'))
Result: False
```
So it looks like, in the condition, `${{parameters.is_release}}`
evaluates to the literal string `$(is_release)`. If we look at the point
of invocation of the ~function~ template, we can see:
```
- template: ci/build-unix.yml
parameters:
release_tag: $(release_tag)
name: 'linux'
is_release: $(is_release)
```
so it does not seem completely crazy. However, according to the
documentation, we should expect that to be replaced by the value of the
corresponding variable, as per:
```
variables:
release_sha: $[ dependencies.check_for_release.outputs['out.release_sha'] ]
release_tag: $[ coalesce(dependencies.check_for_release.outputs['out.release_tag'], '0.0.0') ]
trigger_sha: $[ dependencies.check_for_release.outputs['out.trigger_sha'] ]
is_release: $[ dependencies.check_for_release.outputs['out.is_release'] ]
```
What's interesting here is that, within `build-unix.yml`, we are also
using `release_tag` in the exact same way:
```
- bash: ./build.sh "_$(uname)"
displayName: 'Build'
env:
DAML_SDK_RELEASE_VERSION: ${{parameters.release_tag}}
```
and this time output from the build seems to show the value being
correctly substituted:
```
damlc - Compiler and IDE backend for the Digital Asset Modelling
Language
SDK Version: 0.13.55-snapshot.20200226.3266.d58bb459
Usage: <interactive> COMMAND
Invoke the DAML compiler. Use -h for help.
```
My current guess is that the (undocumented, as far as I can tell)
evaluation order is as follows:
1. In the template, syntactically replace all the parameters.
2. In the job definition, replace the call to the template with the code
of the template. So it is as if we had written the template directly in
the `azure-pipelines.yml` file, with `$(release_tag)` and
`$(is_release)`.
3. Run the build. When we reach the time to run this specific job,
we can evaluate the expressions for the variables and replace them in
the rest of the job.
So what is going wrong? I believe the issue is with the quotes,
preventing the substitution of `is_release`. They came directly from the
[documented
syntax](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml#use-a-template-parameter-as-part-of-a-condition),
but if the above evaluation order is correct, they should not be there.
There are actually two things going wrong here. The first one is that
the syntax `$()` is used to substitute a value in what Azure considers a
string. This is the case for `env` keys. However, the `condition` key
is not a string, it is an Azure "expression". Expressions have their own
evaluation rules and syntax, and in particular, `$()` is not a
substitution rule there, so when it sees `$()` in a string in an
expression (due to the quoptes), it leaves it alone.
Removing the quotes does not directly help, though, as we then end with
```
condition: eq($(is_release), 'true')
```
and `$()` is not valid syntax in an expression. The way to use variables
in an expression is `variables.name` (or `variables["name"]`, because
why have only one?).
So that means we have to pass variables to the template in different
ways depending on how they will be used. So much fun.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Name the arguments to `ApiServices.create` for clarity.
* sandbox: Clarify numbers and types in configuration classes.
* sandbox-next: Log the correct port on startup.
* sandbox-next: Connect up the command configuration.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox-next: Wire up TLS configuration.
* sandbox-next: Wire up the maximum inbound message size.
* sandbox-next: Set the global log level if specified.
And if it's not specified, default to the level in logback.xml, INFO.
* sandbox-next: Connect up the submission configuration.
* sandbox-next: Log the correct ledger ID.
* sandbox-next: Use `TimeProvider.UTC`.
The existing approach is a historical accident. The reason for the
additional tarball/install jobs was that, in my original attempt, the
build steps would still build the current commit, as opposed to the
target commit.
This is not such an issue on Linux, but setting up the build environment
on Windows and macOS _again_ for no good reason is a pure waste of time
(and effort in getting it right). Now that the build steps build the
target commit (with the env var set), we can go back to the way things
were previously: just take the build products directly from the build
step.
CHANGELOG_BEGIN
CHANGELOG_END
* Make completion service return checkpoints
The new table for #4681 and the query used to retrieve completions
currently does not return checkpoints. These do not have to match
the application_id and submitting_party query since those fields
are not populated.
CHANGELOG_BEGIN
CHANGELOG_END
* Address https://github.com/digital-asset/daml/pull/4735#discussion_r384713277
* Extract ErrorOps, use liftErr instead of leftMap
JSON error formatting cleanup
CHANGELOG_BEGIN
CHANGELOG_END
* Good we have tests for this stuff
* Apply https://github.com/scala/bug/issues/3664 work-around,
so JsonError can be used instead of JsonError.apply
* error formatting
This doesn’t seem to bring any benefit anymore on cache hits with the
reduced closure size and on cache misses it’s significantly slower.
changelog_begin
changelog_end
This reduces the number of GHCs to 2 on Linux (regular and DWARF) and
1 on macOS. Given that each derivation is > 1 GB this should hopefully
help a bit.
changelog_begin
changelog_end
* move BeginBookmark to util
* adding offsets to steps
* offsetAfter belongs in Txn, not InsertDeleteStep
* make transaction stream a ContractStreamStep.Txn stream
* add several ContractStreamStep append cases
* rewrite 'render' to emit offset in the right places
* make ContractStreamStep#append total again
* check for offset in a few tests
* revert useless whitespace changes
* missed argument
* simpler mapPreservingIds
* rewrite states for new "live" format
* remove invalidated "events" block structure assertions
* make shutdown in withHttpService deterministic, to try to catch race condition
* exhaustiveness checking somehow disabled; fixed fetch flow and all is well
* documentation and changelog
CHANGELOG_BEGIN
- [JSON API - Experimental] Remove ``{"live": true}`` marker from websocket streams;
instead, live data is indicated by the presence of an "offset".
See `issue #4593 <https://github.com/digital-asset/daml/pull/4593>`_.
CHANGELOG_END
* be more specific about what liveness marker may be in docs
* fix daml2ts websocket tests
* mention type rules for all cases in offset documentation
