* make every PublicPackageUploaded update contain exactly one package
* implement update subscriptions that start with offsets with the same major number
* address review comments
* Do not convert the transaction id to Long in LedgerBackedIndexService
The InMemoryLedger does this, because it can make certain assumptions
(like transactionId == offset), but the sql based ledger cannot do that,
because the transactionId may be produced by another backend and thus
have a different format.
* Move tests that use contract keys into their own integration test file
This allows us to run the other tests via the ledger api test tool with
the postgres indexer while InMemoryKVParticipantState doesn't support
contract keys yet.
* Remove a few occurrences of LedgerBegin from TransactionServiceIT.
* Rename CommandTransactionChecksNo1866IT to ContractKeysNo1866IT
* Enable TransactionServiceTests,DivulgenceIT,TransactionBackpressureIT for postgres index|er
* format all the things
* Address a few review comments.
See https://github.com/digital-asset/daml/pull/2032#pullrequestreview-258771927
* Extract reusable base class for ContractKeys tests
This is needed because Bazel's scala_test_suite isolates the tests from
another, and one test cannot extend another one.
* Classify the reference-v2 conformance tests with a long timeout
* DamlOnXSubmissionService shut down of package load executor service
* ReferenceIndexServer specifies viable PackageId
Using dash (-) instead of slash (/)
Previously we rewrote references to `PRSelf` to `PRImport` references
every time we call `initWorld`. However, we call `initWorld` quite
often (e.g. every time we run a scenario) while the packages that need
to be rewritten stay constant. Since rewriting the package references
requires traversing the whole package, this can be quite expensive.
This PR moves the rewriting of package references out of `initWorld`
and caches it as part of `GeneratePackageMap`.
On a large project where I tested this, this caused a drop in the
runtime of `daml test` from 250s to 200s. In the IDE this can also
make a pretty big difference since we call this everytime we run a
scenario which we do on every file change.
* support rpc: GetFlatTransactionByEventId
* support rpc: GetFlatTransactionById
* test: GetTransactions
* support rpc: GetTransactionTrees, and types: TransactionTree,TreeEvent
* reorder TransactionService.hs to match .proto
* rpcs: GetTransactionByEventId, GetTransactionById
* connenct LedgerAPI party and package management with KVUtils
* formatting
* address review comments
* add participantId parameter to sandbox and indexer
* annotate TODO's with github issue numbers
* Bazel: 0.24.0 -> 0.27.0
* Update rules_haskell for Bazel 0.27 compatibility
* Update bazel-deps and bazel-watcher
* Windows escape JVM flags
* load commands at top of .bzl file
Bazel 0.27 no longer allows load commands that are not at the beginning
of the file.
* Update Bazel rules
* subpackage boundary
* native is not defined in BUILD files
* yarn: @bazel/hide-bazel-files
Seems to be required since latest rules_nodejs version. Otherwise, yarn
fails with errors about existing BUILD or BUILD.bazel files.
* grpc-java plugin visibility
* Update fat_cc_library
* Nix Python3 toolchain
* Iteration over depset
* dev_env_package: Create symlinks one level deeper
To prevent symlinking the BUILD file as well. The nested BUILD file
confuses Bazel as of 0.27 and rules_nodejs cannot find the node
executable anymore.
* Update rules_nodejs
* Add managed_directories for node_modules
* hie-bios: Extract bazel-genfiles from bazel info
Bazel 0.27 changed the genfiles location which breaks the hie-core test
on macOS.
* update cc_wrapper to Bazel 0.27
* bazel info -> bazel info bazel-genfiles
* Fix typo in BUILD
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Refactor generatePackageMap
1. I’m too stupid for 4-tuples so I replaced it by a record.
2. GlobalPkgMap was simply unnecessary given that we have a rule for
getting the package map.
3. getDalfDependencies threw away the bytestring that we store in the
package db only to then read the file again.
This also brings a measurable performance improvement:
On the skeleton project, the runtime of `daml build` drops from 1.4s
to 0.9s and total memory use drops from 67MB to 44MB.
* Address review comments
* check that submitter is in maintainers when looking up keys
Fixes#1866. Note that this limitation applies both for `lookupByKey`
and `fetchByKey` -- anything involving retrieving a key is affected.
* add UNTIL-LF to run tests up to a certain version of DAML-LF
* name targets for DAML tests better
* add notes about DAML-LF changes
* commit Test.daml with DAML-LF 1.5 rather than compiling it on the fly
* add scenario tests for #1866
* add warnings about future key behavior in docs
* use flag rather than version when executing
1) Change to SuiteResourceManagementAroundAll, transforming the test to
not assume a new/empty ledger for each test case
2) Use distinct workflow ids so the individual test cases don't clash
with each other
3) Use a new UUID for each randomized command id, instead of using the
uuid used per test run.
Most of them were caused by off-by-one errors in goto definition.
There was also one test that was marked as an expected failure but the
actual bug has been fixed for some time and the only reason it was
failing is that the error message was different than the test
expected.
I’ve also renamed daml-ghc-shake-test-ci to damlc-shake-tests which is
something that I might actually be able to remember :)
Previously, we only checked two diagnostics messages. However, the
second diagnostics messages does not actually need to correspond to
the latest change but can just be a message that has been in the queue
from one of the 1000 changes before. Now we make sure to actually
check all (or at least every second) diagnostic.
This was technically also an issue before but since we only emitted
diagnostics when they changed and we alternate between two states it
worked fine. The way we implement debouncing means that we can now
also end up emitting two consecutive diagnostics changes for the same
set of diagnostics which made this test flaky.
Given that we already made the max message size configurable it only
seems reasonable to also make the timeout configurable and on very
large projects, we do sometimes hit this.
Semantic tests will be skipped if the specified Ledger API endpoint does not
implement TimeService.
This teaches MultiLedgerFixture to deal with cancelled tests.
This improves Ledger API Test Tool reporter to deal with cancellations better.
Previously, we emitted diagnostics notifications as soon as we got
them. This resulted in a lot of flickering due to diagnostics getting
cleared briefly when typing only to immediately reappear.
Now, we buffer them for 0.1s so that a new event restoring the
same diagnostics for a slightly modified file will overwrite the
initial clear of diagnostics for the new document version.
* Split off addRelativeImport from modifying the session
* Switch located imports to adding their own relative imports
* Delete the unused getGhcDynFlags
* Make runGhcEnv no longer need to consult import paths
* Call runGhcEnv directly
* Move getSrcSpanInfos out to Spans
* Remove a redundant import
* Make findImports in Either rather than ExceptT
* Move getImports over to the right place
* Switch to liftEither
* Add external offset column to parameters
We only ever need to use the external offset when restarting the api
server from a persistent data store to resume the ReadService state
update stream from the last processed offset.
In the PostgresIndexer we then need to store the external offset while
adding a new ledger entry.
* Allow disabling the jmx reporter in metricsmanager
* Implement PostgresIndexer component
* Introduce read-only Ledger and Dao
* SandboxEventIdFormatter treats transactionId as opaque string
* Introduce LedgerBackedIndexService
This can be reused both in the Sandbox and the ApiServer
* Implement PostgresIndex component
* Reference Service based on kvutils-v2 and postgres(index|indexer)
* Disable conformance test for ReferenceServer
* Get rid of orphan usage in Convert
* Get rid of orphan usage in AtPoint
* Delete one entirely unused orphan
* Remove a comment that is no longer true
* Add a .gitignore just for the hie-core piece
* Add standalone pieces to make hie-core work with a global Cabal install
* Add more things to .gitignore, PR suggestion
* Add copyright header
