Commit Graph

9193 Commits

Author SHA1 Message Date
pbatko-da
f2b7902c68
[User management] Add CreateUserResponse and GetUserResponse gRPC response wrappers [DPP-854] (#12682)
changelog_begin
Ledger API Specification: CreateUser and GetUser endpoints of UserManagementService now return the CreateUserResponse or GetUserResponse messages, whereas previously they were returning the User message).
changelog_end
2022-02-08 12:29:57 +01:00
Moritz Kiefer
7547c20507
Fix path to JSON API logback file (#12809)
specifying a non-existent logback file is apparently not an error and
just silently produces garbage.

Probably should add a test that checks all logback files for existent
but that seems somewhat convoluted to setup so I’p prefer to skip that
for now.

changelog_begin
changelog_end
2022-02-08 10:46:56 +00:00
azure-pipelines[bot]
bfa42be998
update NOTICES file (#12806)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2022-02-08 09:50:04 +01:00
Marton Nagy
89c9c5d3e2
Switch to read-through maximum ledger time in MutableCacheBackedContractStore (#12779)
changelog_begin
changelog_end
2022-02-07 21:32:22 +01:00
Moritz Kiefer
22165730bf
Migrate Haskell tests to Sandbox on X (#12761)
* Migrate Haskell tests to Sandbox on X

The diff here looks extremely confusing. sandbox-classic is now
sandbox on x not sandbox classic so this isn’t a typo. It is really
moving to sandbox on x.

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2022-02-07 21:18:16 +01:00
tudor-da
c9dcd1c493
Update kvutils Config with missing flags options from SandboxConfig [DPP-864] (#12629)
* Update kvutils Config and SandboxConfig
* Remove `eager-package-loading` and `scenario` from SandboxConfig
* Add `profile-dir` and `stackTraces` to kvutils Config
* Configurable authService in kvutils

changelog_begin
changelog_end

* Addressed Samir's review comments

* Add back Maximum supported deduplication duration enforcement of 1 hour
2022-02-07 20:13:04 +00:00
Remy
ee7e0c1cd8
Engine: fix compilation exception catch (#12787)
During refactoring of #9993, the "unsafe" compilation of the seedy command
has been moved out of the scope of `Engine.runSafely`, whose only purpose
was to catch exceptions thrown by the Speedy compiler.

This fixes the issue and renames the function `runSafely` to
`runCompilkerSafely` to make its purpose more obvious.

This is not a critical bug, as the command preprocessing should ensure
that (1) the command are properly typed and (2) all the dependencies
have been loaded.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-07 20:07:53 +00:00
Stephen Compall
f63165aa76
chunked HTTP response for /v1/users endpoint (#12777)
Enriches #12766 with a chunked, lazy internal approach to
aggregating pages from the participant server, rather than
aggregating all users directly in memory before returning them.

CHANGELOG_BEGIN
- [JSON API] If ``/v1/users`` fails while retrieving users from the participant,
  it will be reported as a warning, with a partial set of users, rather than
  failing.
  See `issue #12777 <https://github.com/digital-asset/daml/pull/12777>`__.
CHANGELOG_END
2022-02-07 18:20:19 +00:00
Gary Verhaegen
85977d13dc
Update RELEASE.md (#12794)
I suppose whoever fixed this forgot to update these instructions, but as of `2.0.0-snapshot.20220204.9165.0.225c58f4` I observed well-formed "Bob" and "Alice" values in the messaging dropdowns.
2022-02-07 17:08:16 +00:00
Remy
e6fed38c1a
LF: reduce number of fieds in test-scenario-stable-many-fields (#12793)
from 130 to 125, reason the test is currently flaky (It fails around 20% of the time).

Surprisingly, #12678 seems to have increase the stack usage for the type checker.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-07 17:47:42 +01:00
Moritz Kiefer
01d827b7c4
Enable java_conversions in scalapb (#12784)
* Enable java_conversions in scalapb

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2022-02-07 14:57:57 +00:00
pbatko-da
fa0062303d
[User management] Use JVM timestamps [DPP-798] (#12662)
I.e. calculate number of microseconds since Unix epoch in JVM rather than separately in each supported database.

changelog_begin
changelog_end
2022-02-07 15:22:28 +01:00
tudor-da
d63651be58
[Sandbox-on-X] Duplicate party allocations explicit rejection (#12767)
* Added test case in PartyManagementServiceIT

changelog_begin
changelog_end

run-full-compat: true
2022-02-07 14:14:19 +01:00
Marton Nagy
cbff4b124a
LAPI code cleanup (pure refactoring) (#12768)
removing dead code

changelog_begin
changelog_end
2022-02-07 12:43:06 +00:00
Marton Nagy
79fd012508
Setup logging for sandbox-on-x conformance tests (#12786)
changelog_begin
changelog_end
2022-02-07 11:25:42 +00:00
Moisés Ackerman
f0dd2de506
Remove scenarios from daml-studio docs (#12750)
* remove scenario from daml-studio Abort example

* remove scenario from daml-studio CreateAuthFailure example

* remove scenario from daml-studio ExerciseAuthFailure example

* remove scenario from daml-studio NotVisibleFailure example

* replace references to 'scenario' with 'script'

* update Iou copyright notice

* update daml-studio docs pictures

changelog_begin
changelog_end
2022-02-07 11:53:37 +01:00
pbatko-da
1865ccc309
[User management] Return empty users' next page token early [DPP-840] (#12773)
If UserManagementStore retrieved fewer users than requested we can assume this is the last page and thus set the next_page_token to the empty string.

changelog_begin
changelog_end
2022-02-07 09:54:56 +01:00
Moritz Kiefer
a6151f4c22
Cleanup daml_ledger compat tests (#12776)
Somehow left this in accidentally.
Perhaps surprisingly it is harmless since we call this before we
enable `set -e` and the command fails but it is extremely confusing.

changelog_begin
changelog_end
2022-02-07 09:41:32 +01:00
tudor-da
14a0459f0a
SandboxNextFixture replaced by SOX-based SandboxFixture (#12654)
* SandboxNextFixture replaced by Sandbox-on-X based SandboxFixture

changelog_begin
changelog_end

* Some fixed tests

* No direct dependencies on //ledger/sandbox:sandbox and //ledger/sandbox:sandbox-scala-tests-lib

* Fix after rebase

* Rename SandboxFixture and add a missing dep

* Generate valid party names if hint is empty

* Smaller maxInboundMessageSize

* Added test for empty display name

* SandboxServer is a ResourceOwner
* Uses execution context passed as an input for resource management
* Fixes flaky FlywayMigrations issue with null Thread.currentThread.currentClassLoader
* SandboxServer simplification returns Port instead of ApiServer

* Dedicated PMAllocateWithoutDisplayName for non-Canton ledgers
* Created since Canton does not return empty display names
2022-02-07 09:25:22 +01:00
azure-pipelines[bot]
484a2d736d
update NOTICES file (#12782)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2022-02-07 08:50:00 +01:00
matevarga-da
83b9d5ab0b
Fixing repetition by mistake in the docs of JSON API > Production Setup > Data Continuity (#12780)
CHANGELOG_BEGIN
CHANGELOG_END
2022-02-07 08:36:55 +01:00
azure-pipelines[bot]
7c9ecdabf2
update NOTICES file (#12778)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2022-02-05 12:13:15 +00:00
azure-pipelines[bot]
4012358f85
update NOTICES file (#12775)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2022-02-04 22:46:26 +00:00
Moritz Kiefer
1df6e95162
Fix out of bounds in run_jar.cc (#12774)
changelog_begin
changelog_end
2022-02-04 18:18:33 +00:00
Stefano Baghino
9c35fa286e
Return all users from GET /v1/users/, paginating 1k at a time (#12766)
* Return all users from `GET /v1/users/`, paginating 1k at a time

The test runs overall for ~30 seconds, mostly due to the time required to
create 20k users.

changelog_begin
[HTTP-JSON API] Previously, a 10k limit was in place on the number of users returned by
`GET /v1/users/`. This limit has been removed and users are retrieved in chunks from the ledger
changelog_end

As of now, this builds the response in-memory, which is likely not what we want to
support long-term.

* Relax test condition due to interference with other test setups
2022-02-04 17:34:20 +00:00
Gary Verhaegen
b0d81e79f1
split: release (#12769)
Mostly to get a version with `info.json`, but I guess this could also be
used to get the ball rolling on that testing we want to do on Monday.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-04 17:30:20 +00:00
Stephen Compall
6bdb901127
user tokens in oauth2-middleware (#12563)
* operate tokenProvidesClaims on user management service output

* ignore everything but the applicationId claim for StandardJWTPayload

* let the client tester work with non-custom tokens

* test claims check on non-custom token

* add changelog

CHANGELOG_BEGIN
- [Auth Middleware] Supports standard auth tokens for participant user management.
  See `issue #12563 <https://github.com/digital-asset/daml/pull/12563>`__.
CHANGELOG_END
2022-02-04 11:29:10 -05:00
Adriaan Moors
e3bae961cc
VersionClient supports getApiFeatures (#12724)
* VersionClient supports getApiFeatures

This exposes the `features` part of the version service's
`GetLedgerApiVersionResponse` protobuf.

Map the protobuf to user-friendly subclasses of `Feature`,
defined in `c.d.ledger.api.domain`.

VersionClient does not expose experimental features, as they
are meant for internal testing only.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Simon Meier <meiersi-da@users.noreply.github.com>

Co-authored-by: Simon Meier <meiersi-da@users.noreply.github.com>
2022-02-04 16:02:36 +00:00
Marton Nagy
225c58f4b6
Tidy up ingestion Schema framework (pure refactoring) (#12759)
By removing dead code

changelog_begin
changelog_end
2022-02-04 16:11:36 +01:00
azure-pipelines[bot]
27c930a478
update NOTICES file (#12763)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2022-02-04 14:59:35 +00:00
Marton Nagy
5a2dffd79e
Add multi-db sandbox-on-x conformance tests [DPP-802] (#12585)
* Enhances SandboxOnXRunner with manipulateConfig
* Adds Postgres and Oracle conformance test runner binaries
* Adapts BUILD.bazel

changelog_begin
changelog_end
2022-02-04 15:28:20 +01:00
Gary Verhaegen
70c13f2d7e
split: add provenance info (#12760)
This is somewhat redundant with the version number for snapshots, but
will be useful for stable releases.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-04 14:44:16 +01:00
Gary Verhaegen
8423732bdd
daily snapshot: pick better time (#12757)
CI nodes are reset at 4, which means starting at 3 doesn't leave us with
enough time to complete.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-04 12:45:52 +00:00
pbatko-da
732c3bbc03
[User management] Change from uint32 to int32 for max_rights_per_user in UserManagementFeature (#12732)
changelog_begin
changelog_end

Breaks-protobuf: true
2022-02-04 12:46:16 +01:00
pbatko-da
97676bdb2c
Increase time-out in OngoingStreamAuthIT (#12747) 2022-02-04 12:44:40 +01:00
Sergey Kisel
ce5030fc9a
Align Daml Script intro section of the docs with picture (#12742)
CHANGELOG_BEGIN
CHANGELOG_END
2022-02-04 12:36:52 +01:00
Marton Nagy
69ca42fe72
Fixing idempotent insert for Oracle (second attempt) [DPP-562] (#12753)
The original solution turned out to reveal a bug in Oracle DB:
the batch insert with IGNORE_ROW_ON_DUPKEY_INDEX with rows containing
significant amount of varying payload can run on Oracle JDBC
driver exceptions.

* Fixes with switching to single insert instead of multi-insert for package upload
for Oracle
* Enhances unit test to uncover the aforementioned issue

changelog_begin
changelog_end
2022-02-04 12:22:12 +01:00
nickchapman-da
56140569f8
Adapt daml-script to paginated listUsers. (#12710)
* Adapt daml-script to paginated listUsers.

changelog_begin
changelog_end

* adapt testUserListPagination to rename listUsers -> listAllUsers
2022-02-04 11:05:48 +00:00
azure-pipelines[bot]
2a2d656bc3
update NOTICES file (#12755)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2022-02-04 10:45:54 +01:00
pbatko-da
6926e9adb4
Deprecate user identity service [DPP-862] (#12694)
changelog_begin
Ledger API Specification: Ledger Identity service is now deprecated.
changelog_end
2022-02-04 10:32:09 +01:00
pbatko-da
2be158d719
Fix flakiness from user management service IT (#12756)
changelog_begin
changelog_end
2022-02-04 10:18:44 +01:00
Sergey Kisel
15caea42c0
Use fetchAndPersistBracket instead of fetchAndPersist (#12672)
* Use fetchAndPersistBracket instead of fetchAndPersist

* add changelog

CHANGELOG_BEGIN
- [JSON API] Under rare conditions, a POST-style multi-template query backed by
  database could have different template IDs appear to be queried at different
  ledger times, if updated concurrently.  This conditions is now checked and
  accounted for.  See `issue #12672
  <https://github.com/digital-asset/daml/pull/12672>`__.
CHANGELOG_END

* add a searchQuery metric event for each fetch trial in sync search

* revert the contract ID and contract key fetches, and explain why

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2022-02-03 20:16:47 +00:00
Marton Nagy
3ad546aaf0
Fixing idempotent insert for Oracle (and H2) [DPP-562] (#12737)
The idempotent insert clauses are racy for Oracle and H2, this is
only used for inserting packages idempotently.

* Fixes H2 by manually retrying maximum 10 times the execution in question
* Fixes Oracle by applying a different approach not requireing retrying:
using IGNORE_ROW_ON_DUPKEY_INDEX
* Adding unit tests which ensures racy idempotent insert is not an issue
(this was failing originally)
* Adding withConnections helper to StorageBackendSpec

changelog_begin
changelog_end
2022-02-03 21:00:06 +01:00
Gary Verhaegen
8033b36509
fix daily yml (#12751)
I was a bit too enthusiastic and merged before running.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-03 20:37:11 +01:00
pbatko-da
7137b46543
[User management] Use pagination for listing users [DPP-840] (#12610)
CHANGELOG_BEGIN
Ledger API Specification: UserManagementService.ListUsers is now using pagination
CHANGELOG_END
2022-02-03 20:32:50 +01:00
Gary Verhaegen
ca976fe3d7
nightly split releases (#12744)
Because there's no reason not to. The only obstacle to automating the
normal release process is that we need an explicit manual validation
step for our audit log when creating a release, but split-releases are
created in the Assemblty repo, so we can have the audit log over there.

The diff is going to be very messy because there's a lot of stuff moving
around. The only thing that is not just moving a job to a separate file
is the `ci/daily-snapshot.yml` file, which is 100% new and is meant as
a new Azure Pipelines entrypoint (which I will create after this gets
approved). I have made a reasonable effort to create individual commits
that simplify reviewing, but I expect it's still going to be kind of a
mess. I'm open to opening separate PRs to ~bump my stats~ move one job
at a time if that makes reviewing (and testing) easier.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-03 19:05:35 +00:00
Robin Krom
9e1fe24f71
cda: use aliases in the first feature section of docs (#12720)
CHANGELOG_BEGIN
CHANGELOG_END
2022-02-03 18:12:56 +01:00
tudor-da
c34f042aa5
Remove deduplication feature descriptor SYNC_ONLY (#12719)
* Remove deduplication feature descriptor SYNC_ONLY
* Not used since participant-side command deduplication

changelog_begin
changelog_end

* Removed SYNC_ONLY

Breaks-Protobuf: true

* Addressed review comments
Breaks-Protobuf: true
2022-02-03 17:06:28 +00:00
Adriaan Moors
fd49e1704b
Update release testing instructions for windows (#12730)
* Update release instructions for windows

CHANGELOG_BEGIN
CHANGELOG_END

* Apply suggestions from code review
2022-02-03 17:37:50 +01:00
Stefano Baghino
014eb36bcc
Add @ray-roestenburg-da to the release testing rotation (#12746)
changelog_begin
changelog_end
2022-02-03 17:33:10 +01:00