Commit Graph

24 Commits

Author SHA1 Message Date
Nicu Reut
25f1254120
Move the metrics API to the observability module [PLEN-35] (#15831) 2022-12-12 09:35:55 +00:00
Nicu Reut
644009feac
Fix async gauges usages by returning only the value [PLEN-61] (#15793) 2022-12-06 09:26:05 +01:00
Sergey Kisel
1c6f6a748b
IdentityProviderId support for the persistence layer of user/party management [DPP-1332] (#15716) 2022-12-02 16:12:04 +01:00
Sergey Kisel
7760ec9fb3
IdentityProviderConfig persistence layer [DPP-1331] (#15634) 2022-11-24 13:32:42 +01:00
Nicu Reut
e8c7e29ba2
Add labels through the MetricsContext in the metrics API [PLEN-5] (#15349) 2022-10-27 09:26:03 +02:00
atriantafyllos-da
ac8618aaed
Removed unused cache metrics [DPP-1246]. (#15149)
Removed unused cache metrics [DPP-1246].

CHANGELOG_BEGIN
The following metrics were removed from the metrics relevant to caches:
  - load_successes
  - load_failures
  - load_total_time
CHANGELOG_END
2022-10-11 16:39:56 +02:00
atriantafyllos-da
fd38b652ec
Update caffeine to 3.0.5. (#15176)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-06 17:12:39 +02:00
tudor-da
5169e7d4dc
[LLP] Implement ParticipantInMemoryState and updater [DPP-1043] (#14151)
* Implement ParticipantInMemoryState and updater

changelog_begin
changelog_end

* Addressed Marton's review comments

* Addressed last two comments
2022-06-30 09:58:23 +00:00
tudor-da
2614646a35
[LLP] Mutable contract state cache uses Offset as cache_index (#13733)
changelog_begin
changelog_end
2022-05-09 11:23:34 +02:00
Stefano Baghino
f1cd4b1c7c
Remove dependencies on compatibility libraries (#12548)
Continues the work started in https://github.com/digital-asset/daml/pull/12543

These libraries were only needed to transition from Scala 2.12 to 2.13
and are no longer useful as all the necessary items are now available
in Scala 2.13.

changelog_begin
changelog_end
2022-01-24 18:04:07 +00:00
pbatko-da
f7e2faf5aa
[User management] Addressing outstanding code review comments from https://github.com/digital-asset/daml/pull/12344# (#12479)
CHANGELOG_BEGIN
CHANGELOG_END
2022-01-19 15:23:59 +01:00
pbatko-da
4fe6e53b9d
[User management] Persistence with caching (#12344)
Adding
 - `PersistentUserManagementStore` and `CachedUserManagementStore`,
 - `UserManagementStorageBackendTemplate` and sql migrations,
 - CLI flags: `--user-management-max-cache-size` and `--user-management-cache-expiry`;
 And wiring `PersistentUserManagementStore` where before we had `InMemoryUserManagementStore`.
2022-01-18 15:59:22 +01:00
Gary Verhaegen
d2e2c21684
update copyright headers (#12240)
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.

I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-03 16:36:51 +00:00
Gerolf Seitz
2555dbb30b
Use soft references for values in the caches (#10715)
Soft references allow the garbage collector to collect the values referenced
by soft references in case of memory pressure. Since these caches are not the
source of truth for the data, it's better to remove values from the cache
instead of having the process die due to OOM (not enough heap or excessive GC).

Fixes DPP-561.

CHANGELOG_BEGIN
CHANGELOG_END
2021-08-31 13:51:14 +00:00
Samir Talwar
4b8b67a1b5
Upgrade Scalatest to v3.2.9. (#10576)
* Upgrade Scalatest to v3.2.9.

Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.

The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.

As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.

CHANGELOG_BEGIN
CHANGELOG_END

* http-json-oracle: Fix a Scalatest dependency.

* ledger-api-client: Fix a Scalatest dependency.
2021-08-12 23:19:35 +00:00
Miklos
29fcebe224
Fix recording of cache metrics [KVL-888] (#9382)
* Fix recording of cache metrics.
CHANGELOG_BEGIN
CHANGELOG_END

* Removed unused imports.

* Code tidying.
2021-04-13 10:47:08 +00:00
Moritz Kiefer
6b2ac0e0c6
Port parts of //ledger/... to Scala 2.13 (#8470)
* Port parts of //ledger/... to Scala 2.13

Fairly random choice of directories, I just went through them in
alphabetical order. The one thing that I had to disable for now are
the conformance tests since the ledger API test tool has a dependency
not compatible with Scala 2.13.

changelog_begin
changelog_end

* Remove accidentally included //ledger/ledger-api-client/...

doesn’t actually work yet

changelog_begin
changelog_end
2021-01-12 10:55:00 +01:00
Moritz Kiefer
22ce940954
Upgrade scalafmt and enable trailing commas (#8437)
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.

As announced, this will be merged on Saturday to avoid too many conflicts.

changelog_begin
changelog_end
2021-01-09 11:37:37 +01:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
Samir Talwar
70af09db51
kvutils: Generalize caching. [KVL-747] (#8262)
* kvutils: Remove QueryableReadSet, and simplify caching a little.

* kvutils: Generalize the caching ledger state reader.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Make StateReader even more general (no Option).

* kvutils: Convert LedgerStateReaderWithFingerprints to a type alias.

And DamlLedgerStateReaderWithFingerprints too.

* kvutils: Replace LedgerStateReaderWithFingerprintsFromValues.

With LedgerStateOperationsReader and a `.map`.

* kvutils: Replace RawToDamlLedgerStateReaderWithFingerprintsAdapter.

With a `.comapKeys`.

* kvutils: Delete duplicate caching code.

* caching: Move `Cache#get` into its own subtrait.

This method is used by the submission validator, but not by the
caching state readers.

* caching: Move cache mapping from CachingStateReader into Cache.

* caching: Remove an over-eager comment.

* kvutils: Rename `StateReader#comapKeys` to `contramapKeys`.

I got contravariant functors confused with cofunctors.

* caching: Add Scaladoc and improve parameter names in MappedCache.

* kvutils: Adapters everywhere!

* caching: Rename the base test class.
2020-12-14 20:13:34 +00:00
Moritz Kiefer
7e05dc7932
Upgrade rules-scala and scalatest (#8187)
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.

This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.

Apologies for the giant PR, I don’t see a way to keep it smaller.

changelog_begin
changelog_end
2020-12-08 06:59:23 +01:00
Samir Talwar
a749928172
caching: Wait for the cache to evict some values in tests. (#6438)
* caching: Wait for the cache to evict some values in tests.

Windows, especially, might not do it the first time. Not sure why;
caching is weird.

CHANGELOG_BEGIN
CHANGELOG_END

* caching: Pull out duplicate tests into CacheEvictionSpecBase.
2020-06-19 19:30:58 +02:00
Samir Talwar
6496ddd903
caching | kvutils | sandbox: Use a size-based cache for LF value translation. (#6432)
* caching: Split caches into new files.

* caching: Rename `Cache.from` to `WeightedCache.from`.

* caching: Move `Configuration` inside `WeightedCache`.

* caching: Add test cases.

* caching: Allow for Caffeine builders to be covariant.

* caching: When instrumenting the Caffeine cache, compose, don't inherit.

* caching: Add a size-based cache.

* caching: Extract out common test cases into base classes.

* caching: Use the size-based cache for LF value translation.

CHANGELOG_BEGIN
CHANGELOG_END

* caching: Simplify the eviction tests.

* caching: Increase the encapsulation in CaffeineCache.

* caching: Commas are important.

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-06-19 14:42:40 +00:00
Stefano Baghino
6f1e051648
Integrate metrics for cache (#6043)
* Integrate metrics for cache

Centralizes the creation of instrumented caches in a single point and adds
metrics coming from Caffeine into the mix.

changelog_begin
[Ledger API Server] if --max-state-value-cache-size is greater than zero, the
following additional metrics will be recorded under the daml.kvutils.submission.validator.state_value_cache namespace:
hits, misses, load_successes, load_failures, load_total_time, evictions and evicted_weight
changelog_end

* Fix Bazel build file formatting

* Address https://github.com/digital-asset/daml/pull/6043#discussion_r427902339

* Address https://github.com/digital-asset/daml/pull/6043#discussion_r427904794

* Review DropwizardStatsCounter

- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905074
- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905184
- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905307
- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905387
- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905650

* Address https://github.com/digital-asset/daml/pull/6043#discussion_r427906243

* Fix implicit numeric widening fatal warning

* Address https://github.com/digital-asset/daml/pull/6043#discussion_r427960762

* Fix infinite loop in metrics
2020-05-20 15:51:45 +02:00