* add Numeric.java
* ledger-api: rename `decimal` field to `numeric` in value protobuf
* Address Gerolf's comment
* ledger-api: add missing renammings
* ledger-api: relax syntax of numbers that can be sent as numerics
* extractor: fix
* leger-api: change format of number though ledger api
* daml-lf: fix numeric regexp
* ledger: fix tests
This fixes all flakiness in `damlc test` that I was able to
reproduce. Previously, I got it to fail in about 10% of the cases
whereas now I have successfully run tests 200 times under load without
issues.
There were two issues at play here:
1. We run scenarios in separate threads to be able to kill the running
Shake session quickly even if a scenario has an infinite loop or
something like that (there is a timeout but it’s quite long). This
could result in one of those left-over threads trying to issue a
request while we are already trying to shut down.
To fix that, we wait for the concurrency semaphore to be empty before
shutting down.
2. Just waiting for scenario executions is not quite sufficient as
`runAction` does not wait for all rules to finish (we could just use
runActionSync in `damlc test` but I’d rather make this work
properly). While we do wait for all scenario executions to finish
there is one gRPC request in offInterest that we do not wait for:
gcCtxs.
To fix this, I’ve now routed all gRPC requests through the semaphore
which means that we will also wait for these requests to finish (or
prevent them from spawning).
This makes more sense anyway as scenario executions are mostly fairly
cheap requests while things like setting up the context are expensive
so we want to limit their concurrency.
We should make the concurrency limit configurable but I’ll leave that
for a separate PR.
SS.scenarioServiceClient does not just read the actual client from
some IORef, it registers the available gRPC methods. Apparently we
never knew about this or at least I didn’t.
By only doing this once, we should speed things up a bit and this
fixes once of the assertion failures that we have been seing on
shutdown (pthread_mutex_lock(&mu->mutex) == 0 in sync_posix.cc) which
was caused by trying to register a method from another thread after
destroying the channel.
* Show function names in stack trace on failing scenario
So far, we've only shown the location of the function but not its name.
Now, we add the name of the function as well.
I assume the plan was to implement stack traces. I intend to do that as well
but the message type does not fit my approach. Thus, let's remove it first.
* Print stack traces in the scenario on failure
Currently, we only print the last source location, which is not
particularly helpful for debugging. Now, we put all source locations we
encounter during execution on the continuation stack and print them when
a scenario fails. This PR does not print the names of entered functions
or choices. We leave this for a future PR.
* Address Moritz' comments
The module was only used to format decimal numbers. The only impact the
formatting could have had was to add/strip trailing zeros from the
presentation of the number. However, manual testing shows this did not
happen. Thus, the whole module is now useless.
While experimenting with the pure Haskell gRPC implementation I
noticed that there is a fair amount of duplication in PrettyScenario
so this PR cleans up some of that.
* check that submitter is in maintainers when looking up keys
Fixes#1866. Note that this limitation applies both for `lookupByKey`
and `fetchByKey` -- anything involving retrieving a key is affected.
* add UNTIL-LF to run tests up to a certain version of DAML-LF
* name targets for DAML tests better
* add notes about DAML-LF changes
* commit Test.daml with DAML-LF 1.5 rather than compiling it on the fly
* add scenario tests for #1866
* add warnings about future key behavior in docs
* use flag rather than version when executing
Given that we already made the max message size configurable it only
seems reasonable to also make the timeout configurable and on very
large projects, we do sometimes hit this.
Previously we sometimes ended up declaring the expected exit of the
scenario service as an unexpected exit. This PR addresses this by
adding a new variable that we set to True before we ask the scenario
service to exit by closing stdin.
* Allow controlling the gRPC message limit via daml.yaml
We have had to raise that in the past since it caused issues on large
projects so it makes sense to make it configurable.
* Update unreleased.rst
Co-Authored-By: Beth Aitman <bethaitman@users.noreply.github.com>
* specify the new fields for interning package IDs
* percolating intern tables
* crawl packages for package IDs as a pre-phase: generic prep
* stub case for interned_id in Scala packageref reader
* HasPackageRefs instances for the rest of the ast
* make intern table and use when encoding PackageRefs in v1
* don't need where
* stub out decode for interned package IDs
* no benefit to using uint32 instead of uint64
* percolate in encode one step
* interned case for decoding PackageRefs
* naming details
* intern table decoder
* finish propagating the intern table in encoder
* encode the package ID table
* document the vital assumption of encodeInternedPackageIds
* propagate the intern table through the LF decoder
- done by stacking ReaderT on top of Decode internally,
as discussed with @hurryabit
* daml-lf-proto requires mtl
* stub out interned case in Scala LF decoder
* stub interface decoder function
* get the interned table to most places in InterfaceReader
* support for interned package IDs in Scala decoder
* use ImmArraySeq instead of Vector for Scala intern decode table
* adding that ghc extension didn't make sense
* implement interned ID decoding for InterfaceReader
* scenario service won't have interned package IDs
* test the interned ID resolution in Scala by examining the proto -> AST in detail
* proper precondition for the dev phase of interned IDs testing
* better error reporting for malformed DALFs in intern test
* just import Data.Int
- suggested by @neil-da; thanks
* pass around the lookup function instead of the vector in decoder
- suggested by @neil-da; thanks
* remove derivations for types deleted in e63b012d2d
* rename VersionAware to EncodeCtx
- suggested by @hurryabit; thanks
* rename MDecode to MonadDecode
- suggested by @hurryabit; thanks
* pass a function through the encoder instead of a set
- based on suggestions by @hurryabit and @neil-da; thanks
* daml-ghc test that interned IDs are generated
- suggested by @hurryabit; thanks
* adapt to 5b480c99ec#1844
* add scenario Module decoding to Decode.OfPackage
* use purely data-driven decoding in scenario service in Scala
- decouples scenario service from LF decoder implementation
* make DecodeV1 companion private
* make extension to LFv2 more obvious
Previously, we always reverted back to the table view when the
scenario results changed. This PR changes this to preserve the
selected view.
I’ve also tested that this does not break backwards compatibility: If
you use the newer extension with an older SDK, you will get the
previous behavior of reverting to the table view and you can still
switch by clicking on the button.
Fixes#1675
This is a temporary fix to unblock users that have run into this
limit. The long-term solution here is to use compression and/or make
the limit configurable.
Ironically `managed` didn't turn out to make our code more manageable
and we ended up mostly using it in very isolated places only to then
immediately convert it back to bracket-style functions using `with`.
This PR also removes the use of `managed` from the GcpLogger which was
the only other place where we are using it and it finally kills the
rather silly logic that starting up the scenario service was tied to
having an event logger.
* Rename Value's ContractId to VContractId
* daml-lf: a bit more about PartyId
* daml-lf: Concatenable MatchingStringModule
* daml-lf make clear type used for Scenarios only
* daml-lf create ContractId, LedgerId, TransactionId
* sandbox-sql conversion util
* LedgerName -> LedgerString
* futher type cleanup in the sandbox
* daml-lf add test for LedgerString
* fixing tests
* a bit more safety in the DB
* Address Stephen's comments
* fix rebase
* More fixes for StringModule
* change length of LedgerString (256 -> 255)
* Replace all occurrences of Tagged in DAML-LF AST with newtypes
This must end as it makes the code bases harder to understand without
adding any benefits.
* Add two more usages of mkVal
* Remove Orphans.Lib_hashable
* Remove tagged from package dependencies
* Derive stock classes as such
The current naming suggests, the protobuf encoding and the CLI use different
textual representations of minor versions, which is not true. They only
differ in their types. We push this type conversion to the call sites
of both functions instead.
Up to now, the engine blindly assumed that contract ids pointed to
contracts of the right type. However, this assumption is faulty:
contract ids coming from the Ledger API cannot be type checked
in command translation since we need access to the contract itself
to do so.
This caused some seriously surprising / broken behavior: one could
send an exercise command with the wrong template id and still go
through, or break internal invariants about the type of choices.
This commit fixes this by checking that the type of the contract
instances we fetch is correct at runtime.
cc @hurryabit @dajmaki @remyhaemmerle-da @S11001001 @meiersi-da
* Use DA.Prelude a lot less
* Make LANGUAGE pragmas have the right case
* Don't reexport DA.Prelude
* Remove more DA.Prelude imports
* Remove the nastiest renaming DA.Prelude
* Final removal of DA.Prelude
* Kill DA.Prelude
* Add missing dependencies
* More missing package imports
* More missing package imports
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
* clean up v3 contract key case in TransactionVersions
* clean up duplicate cases in ValueVersions.assignVersion
* make LanguageMinorVersion a proper ADT, aliased LanguageVersion.Minor
* port DecodeV1 to LanguageMinorVersion ADT via implicit conversion
- @remyhaemmerle-da might prefer the explicit application of Stable/Dev
* make releasePrecedes private
* add dev versions to the timeline
* copyright header for LanguageMinorVersion
* build the package database for 1.dev, support 1.dev as --target
* test presence and new-ness of "dev" in timeline
* report dev versions in engine info; note 1.dev target in release notes
* spec and governance rules on 1.dev and minor dev in general
* make the governance rule about dev more direct
* 1.x
* missed plural; thanks @leo-da
* further definition of 1.dev in LFv1 spec; thanks @remyhaemmerle-da
* Move from prettyDiagnostic to prettyDiagnostics
* Remove as much pretty print stuff as we can
* Try moving duplicate named functions with similar semantics and identical types to different names
* Change to returning pretty printed outputs from Diagnostics
* Remove a redundant import
* removing major LF dev version from Haskell proto codecs
* removing major LF dev version from scenario service client
* missed import
* remove Scala support for dev major version; remove --allow-dev option from sandbox cli
* Version.minorFromCliOption function
* don't build daml-stdlib artifacts for dev major
* remove damlc CLI --target dev
* release note about removed dev major LF version
* governance now discusses minor dev, no more major dev
* don't build from daml_lf_dev.proto anymore
* remove daml_lf_dev.proto
* raise deprecated release
* reserve 9999 in the ArchivePayload sum, as suggested by @bitonic
* use reserved proto keyword, as suggested by @bitonic
- `reserved` cannot occur within `oneof` block
* remove --allow-dev test
* dev removal release note followed the previous release; move it back to HEAD
* Move to using proto3-wire from upstream
* Move to upstream proto3-suite, with some custom patches in my fork
* Delete the BUILD.bazel for hte proto3 stuff, not used and the test was failing
* Delete the old proto3-wire and proto3-suite forks
* Delete proto3-wire
* Prettify BUILD.bazel files, sort the deps
* Remove some special cases from the license checker
* Delete unused Nix files from grpc-haskell
* Switch to upstream proto3-suite
* Make old-time work on Windows
* Formatting
* Patch rules_haskell to use a response file for -optP to avoid overflowing argument size limits on Windows
* Update 3rdparty/haskell/BUILD.old-time
Co-Authored-By: neil-da <35463327+neil-da@users.noreply.github.com>
* Update the comments in old-time
* Use the revised location of proto3-suite
* GenTransaction.WithTxValue alias
* undo parts of parent dealing explicitly with serialization
* use GenNode.WithTxValue more often
* use Node*.WithTxValue more often
