* Always return error on duplicate submissions
* Remove unnecessary submission information
Now that duplicate submissions always return an error,
we don't need to store the original submission result.
CHANGELOG_BEGIN
CHANGELOG_END
* Rename ttl to deduplicationTime/deduplicateUntil
* Store absolute deduplicateUntil in domain commands
* Fix my own initials
* Remove CommandDeduplicationEntry
Instead, use CommandDeduplicationResult everywhere,
removing the extra layer.
This reduces the number of GHCs to 2 on Linux (regular and DWARF) and
1 on macOS. Given that each derivation is > 1 GB this should hopefully
help a bit.
changelog_begin
changelog_end
* Add TTL field to protobuf
* Add command deduplication to index service
* Wire command deduplication to DAO
* Implement in-memory command deduplication
* Remove Deduplicator
* Implement JDBC command deduplication
* Add TTL field to domain commands
* Deduplicate commands in the submission service
CHANGELOG_BEGIN
- [Sandbox] Implement a new command submission deduplication mechanism
based on a time-to-live (TTL) for commands.
See https://github.com/digital-asset/daml/issues/4193
CHANGELOG_END
* Remove unused command service parameter
* fixup protobuf
* Add configuration for TTL
* Fix Haskell bindings
* Rename SQL table
* Add command deduplication test
* Redesign command deduplication queries
* Address review comment
* Address review comment
* Address review comments
* Make command deduplication test optional
* Disable more tests
* Address review comments
* Address review comments
* Refine test
* Address review comments
* scalafmt
* Truncate new table on reset
* Store original command result
* Rename table columns
... to be consistent with other upcoming tables
* Rename migrations to solve conflicts
Fixes#4193.
This is one of the tests that keeps breaking every time we change the
number of packages `damlc build` outputs by default which is quite
annoying.
The actual change is trivial, we just read the number of packages from
the manifest. The diff is mostly just propagating this through everything.
changelog_begin
changelog_end
* Add type-level strings in DAML.
This PR adds a `PromotedText` stable package, with `PromotedText` type, which is used to encode type-level strings from DAML into DAML-LF. The reason for this is to preserve the `HasField` instance argument. This PR adds a test that `HasField` is succesfully reconstructed incontexts, during data-dependencies, which wasn't possible before.
changelog_begin
changelog_end
* adresss comments
* fix overly specific tests
Shouldn’t really make a difference due to laziness but at least it
makes it explicit if we need to decode the archive to the AST or just
need to get the package id.
changelog_begin
changelog_end
* Translate unsupported kinds to a special Erased type
This should simplify `data-dependencies` and avoid issues like #4470
since we can match on the type instead of having to guess which types
can and which cannot be translated back to DAML.
changelog_begin
changelog_end
* Move most of the remaining serializable types to stable LF packages
The only serializable types left in DAML stdlib after this PR are the
following:
- DA.Upgrade:MetaEquiv
- DA.Random:Minstd
- DA.Next.Set:Set
- DA.Next.Map:Map
- DA.Generics:MetaSel0
- DA.Generics:MetaData0
- DA.Generics:DecidedStrictness
- DA.Generics:SourceStrictness
- DA.Generics:SourceUnpackedness
- DA.Generics:Associativity
- DA.Generics:Infix0
- DA.Generics:Fixity
- DA.Generics:K1
- DA.Generics:Par1
- DA.Generics:U1
- DA.Internal.Prelude:Optional
Ignoring the Generics stuff which isn’t very urgent imho and the
Upgrade stuff which is probably going to change significantly anyway,
this leaves us with the weird Random module, the wrappers around
TextMap which will go away anyway and DA.Internal.Prelude:Optional
which shouldn’t exist in the first place (I’ll address that in a
separate PR).
CHANGELOG_BEGIN
- [DAML Compiler] Move more types from daml-stdlib to standalone LF
packages. The module names for the types have also changed
slightly. This only matters over the Ledger API when you specify the
module name explicitly. In DAML you should continue to use the
existing module names.
- The types from ``DA.Semigroup` are now in a separate package under
``DA.Semigroup.Types``
- The types from ``DA.Monoid` are now in a separate package under
``DA.Monoid.Types``
- The types from ``DA.Time` are now in a separate package under
``DA.Time.Types``
- The types from ``DA.Validation` are now in a separate package
under ``DA.Validation.Types``
- The types from ``DA.Logic` are now in a separate package under
``DA.Logic.Types``
- The types from `DA.Date` are now in a separate package under
`DA.Date.Types`.
- The `Down` type from `DA.Internal.Prelude` is now in a separate
package under `DA.Internal.Down`.
CHANGELOG_end
* Fix serializability of RelTime
* fix daml-docs
* Fix tests
* Move Any wrappers and Archive to stable packages
There are no actual API or functionality changes in this PR but the
logic for locating the stable packages has slightly changed since the
Any wrappers package only makes sense for LF 1.7. To address this, we
simply filter out stable packages for newer LF versions since it
doesn’t make sense to depend on those anyway.
CHANGELOG_BEGIN
- [DAML Compiler] Move ``Archive`` type to a separate DALF.
CHANGELOG_END
* More comments
* Fix java codegen tests
* fix more tests
* Force cache reset on Windows
* Revert "Force cache reset on Windows"
This reverts commit 9f2b7d70b2.
* Add package_entries table
* Change PublicPackageUpload event to cover list of packages.
Add PublicPackageUploadRejected.
* Produce new package update events in KeyValueConsumption
* Update signature of uploadPackages
* Cleanup InMemoryKVParticipantState. Add submissionId to uploadPackages.
* Fix up InMemoryKVParticipantStateIT
* Initial ledger dao changes for package entries
Drop the participant_id as we never expect to see
entries of other participants. This should be done
for party_entries as well.
* Drop UploadPackagesResult
* Implement getPackageEntries and refactor callers
* Add maxRecordTime to uploadPackages
* First cut at updating ApiPackageManagementService
* Update tests, wire through the packageEntries
* Don't extend IndexPackagesService in InMemoryPackageStore
It does not implement the full interface and it isn't used
directly as one anyway.
* Drop maximum_record_time from package_management_service
Adding maximum record times touches the whole stack. Leaving
this change to another PR.
* Wire through the removal of maximum_record_time.
And remove dead code from InMemoryKVParticipantState
* Remove checking for duplicate package uploads
This aligns with the behaviour of WriteService.
* Reformat
* Fix PackageManagementService after adding of submission_id to the service
* Move all datatypes out of daml-prim
This moves the remaining two modules DA.Types and GHC.Tuple to
separate LF packages with stable identifiers.
The only data types remaining are the ones for typeclasses which will
disappear once we move this to type synonyms.
CHANGELOG_BEGIN
- [DAML Compiler] The modules DA.Types and GHC.Tuple from daml-prim
have been moved to separate packages.
CHANGELOG_END
* Fix codegen tests
* Fix DarReader test
* Fix kvutils tests
* Fix jdbcdao tests
* Fix hs ledger bindings tests
* daml assistant expected auth token in Bearer format
* Daml assistant does no validation of the auth token before passing in on to the ledger.
* clarify code with newtype Token
This is a first step towards making sure that the package ids for
types defined in daml-prim and daml-stdlib don’t change. This PR
mostly adds all the necessary infrastructure for that and moves
GHC.Types and GHC.Prim to make sure it works.
Until data-dependencies are really solid and we have verified that we
no longer have performance issues with an increasing number of Haskell
packages, we still include the source files in daml-prim and then just
rewrite the references.
We will also need to add tests that these packages really have stable
ids but I’ll leave that for separate PRs since this doesn’t make that
much sense anyway until all of the types have moved to stable
packages.
CHANGELOG_BEGIN
- [DAML Compiler] The modules GHC.Prim and GHC.Types from daml-prim
have been moved to separate packages.
CHANGELOG_END
* support --access-token-file in SDK
* rename: HostAndPort -> LedgerArgs; HostAndPortFlags -> LedgerFlags
* integration test for --access-token-file
* doc & more help
* address comments: use FilePath
* adapt to renamed arg: --auth-jwt-hs256-unsafe
* avoid use of single quotes on command line
The shared-secret-based authentication is there exclusively for testing,
this PR makes sure that this is correctly reported through the CLI and
hides this until further notice.
Makes sure that the flag clearly says that this option is not meant for
production use cases.
A warning is already printed when this feature is used, both at startup
and every time this verifier is used.
CHANGELOG_BEGIN
- [Sandbox] The ``--auth-jwt-hs256`` is renamed to
``--auth-jwt-hs256-unsafe``: you are advised to _not_ use this JWT token
signing way in a production environment.
CHANGELOG_END
* support authenticating ledgers in haskell ledger bindings
* address comments
CHANGELOG
- [Haskell Ledger Bindings] - support for passing `JWT` tokens to authenticating ledgers.
When buidling simple project that has our favourite large project as a
dependency, this decreased
- total allocations from 63GB to 57GB
- run time from 34.0s to 31.5s
* stack setup for HLB and Nim
* build_packages.sh and daml-ledger.cabal
* update README
* export-package for daml-ledger
* export-package, run here
* find tmp daml-ledger
* update README for review comments
* ref daml instructions
* further improve the process and documentation in response to comments
* have -with-rtsopts match the bazel build
* fixes for Gary
* add Numeric.java
* ledger-api: rename `decimal` field to `numeric` in value protobuf
* Address Gerolf's comment
* ledger-api: add missing renammings
* ledger-api: relax syntax of numbers that can be sent as numerics
* extractor: fix
* leger-api: change format of number though ledger api
* daml-lf: fix numeric regexp
* ledger: fix tests
* hlb, remove dependency on DAML-LF libs
* Update language-support/hs/bindings/src/DA/Ledger/Services/PackageService.hs
newtype instead of data
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Refactor gRPC request cancellation support in gRPC-haskell
Previously, we had a separate constructor that took an additional
callback that had access to the client call so that we could use that
for cancellation. This PR removes the separate constructor and instead
changes the callback accepted by ClientReaderRequest (and for
consistency also the one accepted by ClientBiDiRequest) to pass the
client call.
This seems like a simpler solution so I’m more hopeful that we will be
able to upstream this change (this is the only API breaking change we
made afaik). There are two caveats here:
1. This will break existing consumers that use ClientReaderRequest. At
this stage in gRPC-haskell this is very reasonable and upstream
doesn’t seem to try very hard to avoid those.
2. The callback is called slightly later than the custom callback we
added before so you have to wait a bit longer until you can cancel
things. At least on our test suite that doesn’t seem to make a
difference.
* Go back to building grpc-haskell-core using c2hs
This should hopefully avoid issues like the CSize vs CULong issue we
had a while back and might fix some of the issues we have been seeing
on CI.
I’m marking the Haskell ledger bindings as non-flaky for now so we can
see if the issues reappear.
* Fix path
* Fix c2hs runfiles
* s/basedir/dirname/
* Fix varname
* Remove fixme \o/
* Mark hs ledger bindings flaky again
The only tests that relied on this, were the ones for package
management and party management but we can fairly easily remove that
dependency by only checking the diff in packages and thereby folding
the listKnown* and uploadParty/uploadDar file into a combined test.
I’ve also bumped one timeout that I’ve seen fail on CI and managed to
get to fail locally under load.
I am unable to get the tests to fail now locally so I’ve marked them
as non-flaky on everything but Windows (we’ve seen weird segfaults
there).
* Use per-test parties in HS ledger bindings tests
This should hopefully solve the flakiness issues caused by the reset
service. For now, there is a very small number of tests that we run on
an isolated sandbox namely the tests for party management and package
management.
* shut up hlint
* Disable reset tests
* Supporting producing sdist tarballs for the HS ledger bindings
The README.md has an explanation for how you can use this.
This should hopefully allow others to experiment with the bindings.
