So far, these two flags only worked in sandbox classic because I wasn't
aware the code path consuming the options is not shared between both
implementation. Now, sandbox next is on par with sandbox classic.
CHANGELOG_BEGIN
CHANGELOG_END
For a while now we've had errors along the line of
```
FATAL: could not create shared memory segment: No space left on device
DETAIL: Failed system call was shmget(key=5432001, size=56, 03600).
HINT: This error does *not* mean that you have run out of disk space.
It occurs either if all available shared memory IDs have been taken, in
which case you need to raise the SHMMNI parameter in your kernel, or
because the system's overall limit for shared memory has been reached.
The PostgreSQL documentation contains more information about
shared memory configuration.
child process exited with exit code 1
```
on macOS CI nodes, which we were not able to reproduce locally. Today I
managed to, sort of by accident, and that allowed me to dig a bit
further.
The root cause seems to be that PostgreSQL, as run by Bazel, does not
always seem to properly unlink the shared memory segment it uses to
communicate with itself. On my machine, running:
```
bazel test -t- --runs_per_test=100 //ledger/sandbox:conformance-test-wall-clock-postgresql
```
and eyealling the results of
```
watch ipcs -mcopt
```
I would say about one in three runs leaks its memory segment. After much
googling and some head scratching trying to figure out the C APIs for
managing shared memory segments on macOS, I kind of stumbled on a
reference to `pcirm` in a comment to some low-ranking StackOverflow
answer. It looks like it's working very well on my machine, even if I
run it while a test (and therefore an instance of pg) is running. I
believe this is because the command does not actually remove the shared
memory segments, but simply marks them for removal once the last process
stops using it. (At least that's what the manpage describes.)
CHANGELOG_BEGIN
CHANGELOG_END
* add simplifier tests
* add some lambda constant lifting tests
changelog_begin
changelog_end
* update copyright
* clarify that the subexpression is a lambda
* Explicit export list in tests module
* don't use lambda in test names :-(
Our build is intermittently failing on CI on the target,
`//ledger/participant-state/kvutils:reference-ledger-dump`.
Creating the reference ledger dump exercises ledger-on-memory with the
Ledger API Test Tool as a side effect. The real goal is to run any kind
of kvutils ledger and generate a load of data. With that in mind, the
idea is to be somewhat stable, not fast.
This decreases the number of concurrent tests running at once to
generate the reference dump from the number of CPUs (presumably 8 on CI,
though I'm not sure) to 4.
It does not change the number of concurrent tests when running
`//ledger/ledger-on-memory/conformance-test`. This will still default to
the number of CPUs.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump Flyway version to 6.5
Prevents incurring into https://github.com/flyway/flyway/issues/2759 (which was apparently solved in 6.4.0)
changelog_begin
changelog_end
* Comply with changed method signature
* sandbox: Add a command line flag to disable DAML stack traces
The sandbox now accepts a `--stack-traces no` flag which will turn off
the location tracking in DAML Engine required to produce stack traces
for failing DAML code.
Benchmarks suggest that DAML Engine spends about 10% of its time with
tracking locations. Thus, this flag will give us roughly a 1.1x
speedup when stack traces are not needed.
This flag is still hidden because we would like to validate its
usefulness before we commit to supporting it.
CHANGELOG_BEGIN
CHANGELOG_END
* Make it more obvious where we're overriding methods
CHANGELOG_BEGIN
CHANGELOG_END
* Improve help text
* sandbox: Create proper `ResourceOwner` implementations.
This allows us to use the resource acquisition execution context, rather
than using DirectExecutionContext.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Simplify the construction of SqlLedger.
* sandbox: Inject the dispatcher into the BaseLedger.
* sandbox: Make sure the SqlLedger objects are closed.
Tiny regression. No one noticed. It's OK.
* sandbox: Simplify ReadOnlySqlLedger.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* sandbox: Pull out functions to make SqlLedger.Owner easier to read.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-api-common: Factor out wrapping Dispatcher in a ResourceOwner.
* sandbox: Move the PersistenceQueue into a ResourceOwner.
* ledger-api-common: Add a comma.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Copy code button
Added the sphinx copy code extension (https://sphinx-copybutton.readthedocs.io/en/latest/) to the docs. Tested with and it worked.
CHANGELOG_START
CHANGELOG_END
* Remove vendored copy of sphinx_copybutton
changelog_begin
changelog_end
* remove debugging output
changelog_begin
changelog_end
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
switch CI nodes from n1-standard-8 to c2-*
A while back (#4520), I did a bunch of performance tests when trying to
size up the requirements for the hosted macOS nodes we needed to buy. As
part of that testing, it looked like `c2-standard-8` nodes were faster
(full build down from ~95 to ~75 minutes) and marginally cheaper
($0.4176 vs $0.4280) than the `n1-standard-8` we are currently using.
Then I got distracted, and I forgot to upgrade our existing machines.
CHANGELOG_BEGIN
CHANGELOG_END
* Address https://github.com/digital-asset/daml/pull/6507#discussion_r446113575
* drop blindinginfo.proto
changelog_begin
changelog_end
* drop BlindingCoder
* Remove blindinginfo Protobuf definition JAR
changelog_begin
[DAML-LF] The blindinginfo Protobuf definition JAR, which was previously unused, has been pulled from the artifacts released on Maven
changelog_end
Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
Added a new section in the cheat-sheet for basic DAML concepts (there's anotehr PR for it). This PR links the cheat-sheet to the GSG. This was a requirement that came out of the user tests where testers said that they would like to have a quick overview of the most important DAML concepts explained (they found jumping through the glossary and the rest of the docs too complicated).
CHANGELOG_BEGIN
CHANGELOG_END
When trying to understand how disclosure/divulgence information is
computed for the scneario service, I stumbled on some dead code:
* The `EnrichState.divulgeContracts` function is never used. Let's
remove it.
* The `EnrichState.localDivulgendes` field is initialized with an empty
map and never touched again. Thus, it's always going to stay an empty
map. This means that the
`EnrichedTransaction.localImplicitDisclosure` is also always going to
be an empty map. Both fields can hence be removed.
I'm not sure if the `BlindingInfo.localDivulgence` field can be removed
as well. In my understanding, this data structure could be loaded from
transactions serialized with the field being non-empty in the past.
Thus, I've refrained from removing the field and set it to an empty map
when constructing a `BlindingInfo` from an `EnrichedTransaction`.
CHANGELOG_BEGIN
CHANGELOG_END
* 'Thumbs up/down' and 'Feedback' button fix
Changed the position of the 'Thumbs up/down' button to be in the bottom right corner above the 'Feedback' button. Added SASS to remove both buttons for mobile (as we were getting a lot of gibberish feedback
CHANGELOG_START
CHANGELOG_END
* 'Thumbs up/down' and 'Feedback' button fix
Changed the position of the 'Thumbs up/down' button to be in the bottom right corner above the 'Feedback' button. Added SASS to remove both buttons for mobile (as we were getting a lot of gibberish feedback
CHANGELOG_START
CHANGELOG_END
* Putting some of the blocks where they were
I broke the tests for Sandbox < 1.2.0 when I switched to explicit
party allocation yesterday since I didn’t realize this only landed in
SDK 1.2.0 (sorry about that).
changelog_begin
changelog_end
This introduces the same trick for sharing the postgres instance that
we use in the main workspace to the compatibility workspace. The bash
code for that is currently duplicated. Didn’t seem worth trying to
make it sufficiently generic that we can share it (it hasn’t changed
since we added it) but if someone disagrees, I’m happy to reconsider
that.
changelog_begin
changelog_end
* Move top level case classes to package object
changelog_begin
changelog_end
* Move Server Message classes to separate file
* Server in Server.scala and ServiceMain in ServiceMain.scala
* Copyright headers
This upgrades styled-components to the latest version and adds peer
dependencies as yarn told me to. I did test this a bit side-by-side
with Navigator from 1.2.0 to see if I could notice any changes both in
Firefox and Chrome and it looks exactly the same.
The changes are all fairly mechanical following type errors.
changelog_begin
changelog_end
CHANGELOG_BEGIN
Update spark version to update jetty to address security vulnerabilities
CHANGELOG_END
Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
* Allocate parties in test-tool compat tests
Compatibility is really something you care about in a production
setup. Implicit party allocation is a development feature and not
available on most ledgers. Therefore, this PR switches the tests over
to allocate parties explicitly. For sandbox-next we also disable the
implicit party allocation feature completely which allows us to
include the ClosedWorldIT test. Sadbonx-classic does not allow
disabling it atm.
In principle, we could run in both configurations but I don’t think
the additional costs (we have hundreds of those tests and this would
double the number) are worth it due to the same reasons that I’ve
given at the beginning for why explicit allocation is more important.
changelog_begin
changelog_end
* Fix postgresql tests
changelog_begin
changelog_end
The Server object and especially the apply method is extremely big and
hard to follow. There are a great many functions nested within the apply
method. This PR attempt to organize things, mostly by moving local
functions into methods of the Server class. I think this makes things
easier to follow, and I think it's more conventional.
Note that with this change I added some implicit parameters to the
Server constructor, which I think makes sense for these kinds of values
(contexts, sequencer factory, etc.) I also moved the Message trait to
the top level, but we will probably it its own file, following this PR.
There are a few other shallow tweaks that I made along the way.
Open to feedback on the high level structuring, as I'm not super
familiar with idiomatic design in Scala.
changelog_begin
changelog_end
* DAML REPL --static-time regression test
* damlc repl --static-time|--wall-clock-time flags
CHANGELOG_BEGIN
- [DAML REPL] The time mode can now be specified using the
``--static-time`` and ``--wall-clock-time`` flags.
CHANGELOG_END
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Verify the effect of setTime using getTime
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
It is pretty easy to hit this, e.g., when your templates haven’t been
uploaded to the ledger. Just printing the response doesn’t actually
include the response body which means that you don’t see the actual
error so it’s pretty useless. This PR changes that by printing the
status code and the response body.
All the rest is just test setup to be able to submit a script with a
template that has not been uploaded to the ledger.
changelog_begin
changelog_end
Following some discussions on Slack, I’ve decided to spend a bit of
time trying to see which deps can be bumped fairly easily. This PR
bumps react and react-dom to the latest versions. The upgrade doesn’t
seem to require any code changes.
I did test Navigator locally in quickstart-java (looking around,
creating contracrs, exercising a few choices) and everything looks as
expected.
changelog_begin
changelog_end
* add -Xsource:2.13, -Ypartial-unification to common_scalacopts
* add now-referenced scalaz-core where needed
* work around bad type signatures in scalatest Aggregating, Containing
* unused Any suppression
* work around bad partial-unification wrought by type alias
* remove unused Conversions import
- not required in 4f68cfc480 either, so unsure how it's survived this long
* work around Future.traverse; remove unused show import
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused bounds
* remove -Ypartial-unification and -Xsource:2.13 where they were explicitly passed
* longer comment on what the options do
- suggested by @stefanobaghino-da; thanks
* forget Future.traverse, just use scalaz, it knows how to do this
This PR updates the lower bound of the api test tool required to use the
granular test selection feature to the first snapshot that includes it
(as opposed to the floating 0.0.0).
CHANGELOG_BEGIN
CHANGELOG_END
* Move database initialization to Server apply method
* Reorder dao/server creation
* Read packages from database on startup
* Test starting a trigger after a shutdown
changelog_begin
changelog_end
* bindings-rxjava: Return the `Disposable` from `Bot.wire`.
And in `BotTest`, dispose of the connection.
We are seeing some spurious errors in runs of BotTest due to connections
shutting down in the wrong order. By explicitly disposing of the wiring
before shutting down the ledger client, we can hopefully suppress these
errors.
They're not causing test failures, but they do often obscure the real
failure.
CHANGELOG_BEGIN
- [RxJava Bindings] `Bot.wire` and `Bot.wireSimple` now return a
`Disposable`, which can be used to shut down the flows. You are
encouraged to call `.dispose()` before terminating the client.
CHANGELOG_END
* bindings-rxjava: Don't run `awaitTermination` in a loop.
When shutting down the ledger client, instead of calling
`channel.awaitTermination` in a loop, we just call it once with an
approximately-infinite timeout.
* bindings-rxjava: Use the loan pattern for disposing in BotTest.
* bindings-rxjava: Simplify BotTest a little further.
This PR adds an extra post-release job to CI that will run the
[`compatiblity/update-versions.sh`][0] script and open a PR with the
result.
[0]: cb82a8d6be/compatibility/update-versions.sh
CHANGELOG_BEGIN
CHANGELOG_END
* Display why a party knows about a contract in table view
When displaying scenario results in table view in DAML Studio, we now
indicate _why_ a party knows about the existence of a contract:
- `S` means the party is a signatory.
- `O` means the party is an observer.
- `D` means the party has learned about the contract via disclosure or
divulgence.
With the information we get from the scenario service there is no
straightforward way of distinguishing between disclosed (you
witnessed the `create`) and divulged (you witnessed a `fetch`)
contracts. Fortunately, both words start with a "D" and we're fine. :)
This addresses the first point of
https://github.com/digital-asset/daml/issues/6412 for the table view.
CHANGELOG_BEGIN
* [DAML Studio]
When displaying scenario results in table view in DAML Studio, we now
indicate _why_ a party knows about the existence of a contract:
- `S` means the party is a signatory.
- `O` means the party is an observer.
- `D` means the party has learned about the contract via disclosure or
divulgence.
CHANGELOG_END
* Add tooltips
CHANGELOG_BEGIN
CHANGELOG_END
* Add test
CHANGELOG_BEGIN
CHANGELOG_END
* Remove tooltip for invisible contracts
CHANGELOG_BEGIN
CHANGELOG_END
* Move parties to then end
CHANGELOG_BEGIN
CHANGELOG_END
On closing the SingleThreadExecutionSequencerPool, all sequencers in the
pool are set to null (presumably to encourage garbage collection).
However, you can still ask for the next executor, which will then return
`null`, causing all manner of problems down the line.
This change forces the failure early, making it easier to debug, by
checking whether the pool is closed and if so, throwing an
`IllegalStateException`.
We see these null pointer exceptions in the `BotTest` test logs from
time to time.
CHANGELOG_BEGIN
- [RxJava Bindings] We now fail faster, with a more meaningful error,
when RxJava flows continue running after shutting down the client.
CHANGELOG_END
* Support multiple auth tokens in DAML Script
This piggy backs on top of the already existing --participant-config
feature. While you can argue that it might be slightly confusing that
you have to specify the same participant twice to specify different
auth tokens, I think this actually makes sense: In an ideal
world (ignoring any performance issues) you have one participant per
party anyway and one connection per participant specified in the
config file still seems like a very reasonable model.
changelog_begin
- [DAML Script] You can now use DAML Script with multiple auth
tokens. This is particularly useful if you are working with the JSON
API where you can only have one party per token or with an IAM that
only provides single-party tokens. The tokens are specified in the
participant configuration passed via `--participant-config` in a new
``access_token`` field. The existing `--acess-token-file` flag is still supported if you want to use the same token for all connections. See
https://docs.daml.com/daml-script/index.html#running-daml-script-against-authenticated-ledgers
for more details.
changelog_end
* I will never understand rst
changelog_begin
changelog_end
The //:migration-stable test-cases had issues with test results not
being invalidated after an update of the HEAD SDK. This may be due to
the SDK distribution files being undeclared dependencies of the `daml`
binary. This change adds the SDK distribution files to the `data`
attribute of the generated `daml` `cc_binary` to ensure that changes are
noticed by Bazel and cached test-cases invalidated.
The corresponding runfiles tree includes a symlink per file of the SDK
release. At the time of commit this amounts to ~9MiB of symlinks for one
SDK version (0.0.0 in particular).
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This addresses a security vulnerability. Unfortunately, we need to
force a newer version of resize-img ignoring our
dependencies. However, that seems to work fine based on my
testing (running navigator on quickstart-java and looking at
favicons).
changelog_begin
changelog_end
* Fix contains check in ConcurrentCompiledPackages
`contains` searches for a _value_ not a _key_. We have a package id
here in both cases which is clearly a key.
I am not exactly clear on what exactly happens if you hit this bug. I
believe it’s just a performance issue so probably hard to write a test
for.
I did take a brief look at whether we can make this
typesafe (`contains` accept an `Object` which is how this typechecks)
and apparently calling `asScala` and then using
`scala.collection.concurrent.Map` should work but I don’t know if this
has performance implications or if there is another reason why we
didn’t do this. Happy to make the change if someone tells me this is
the right thing to do.
changelog_begin
changelog_end
* Switch to Scala’s concurrent map which doesn’t pretend types are bad
changelog_begin
changelog_end
* upgrade elliptic version to address vulnerability
* Revert "upgrade elliptic version to address vulnerability"
This reverts commit dbf19c32
* upgrade elliptic version to address vulnerability
CHANGELOG_BEGIN
CHANGELOG_END
Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
* Use range for elliptic rather than specific version
Noobs had a problem running yarn install v1.12.3
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.40s. second time around when they^ve changed the code. Simply put they would way too often (almost alqways) forget to include when calling yarn install v1.12.3
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.34s.. This would be a very simple workaround that woould lead to them being aware from the very first step that there's the parameter that needs to be called.
CHANGELOG_BEGIN
CHANGELOG_END
