digital-asset/daml
1
1
Fork 0
mirror of https://github.com/digital-asset/daml.git synced 2024-09-20 09:17:43 +03:00
Code Issues Projects Releases Wiki Activity
644e76d047
daml/daml-lf/archive
History
Moritz Kiefer 644e76d047
Build a zip containing all protobuf files (#5412) 
* Build a zip containing all protobuf files

This builds a fat zip that contains the DAML-LF protobuf files and
the Ledger API protobuf files for a given release.

I’ll tackle the Azure config for uploading this to github releases afterwards.

To ease reviewing this is how the resulting zip looks like:

```
Archive:  bazel-bin/release/protobufs.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
     3593  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/package_service.proto
     1355  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/ledger_identity_service.proto
     6262  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/event.proto
     2282  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/testing/reset_service.proto
     1908  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/testing/time_service.proto
      886  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/ledger_offset.proto
     1327  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/transaction_filter.proto
     2380  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/command_submission_service.proto
     6208  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/transaction_service.proto
     3800  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/transaction.proto
     1506  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/completion.proto
     1893  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/ledger_configuration_service.proto
     5109  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/admin/party_management_service.proto
     3562  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/admin/package_management_service.proto
     3559  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/admin/config_management_service.proto
     5542  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/commands.proto
     4286  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/command_completion_service.proto
     2989  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/active_contracts_service.proto
     3393  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/command_service.proto
     5868  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/value.proto
     1810  2010-01-01 00:00   protos-0.0.0/com/digitalasset/ledger/api/v1/trace_context.proto
      558  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_6/daml_lf_0.proto
     1766  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_6/daml_lf.proto
    27265  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_6/daml_lf_1.proto
      558  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_7/daml_lf_0.proto
     1766  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_7/daml_lf.proto
    35657  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_7/daml_lf_1.proto
      558  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_8/daml_lf_0.proto
     1766  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_8/daml_lf.proto
    37168  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_8/daml_lf_1.proto
      596  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_dev/daml_lf_0.proto
     1804  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_dev/daml_lf.proto
    39864  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_dev/daml_lf_1.proto
```

changelog_begin
changelog_end

* Try to get tar from PATH

* Address review comments
2020-04-03 14:16:15 +02:00
..
src Reduce memory necessary to run the zipbomb detection test (#5345) 2020-04-01 15:35:54 +00:00
archive.bzl replace DAML Authors with DA in copyright headers (#5228) 2020-03-27 01:26:10 +01:00
BUILD.bazel Build a zip containing all protobuf files (#5412) 2020-04-03 14:16:15 +02:00
README.md daml-lf proto: archive proto for daml-lf 1.6 (#3142) 2019-10-15 18:35:20 +00:00

README.md

DAML-LF archive

This component contains the .proto definitions specifying the format in which DAML-LF packages are stored -- the DAML-LF archive. All the proto definitions are kept in the directory src/protorotobuf/com/digitalasset/daml_lf_dev/

The entry point definition is Archive in src/protorotobuf/com/digitalasset/daml_lf_dev/daml_lf.proto. Archive contains some metadata about the actual archive (currently the hashing function and the hash), and then a binary blob containing the archive. The binary blob must be an ArchivePayload -- we keep it in binary form to facilitate hashing, signing, etc. The encoding and decoding of the payload is handled by Haskell and Java libraries in daml-core-package, so that consumers and producers do not really need to worry about it.

ArchivePayload is a sum type containing the various DAML-LF versions supported by the DAML-LF archive. Currently we have two major versions:

Snapshot versions

The component contains also an arbitrary number of snapshots of the protobuf definitions as they were as the time a particular version of DAML-LF was frozen. Those snapshots are kept in the directories src/protorotobuf/com/digitalasset/daml_lf_x_y/, where x.y is a already frozen DAML-LF version. A snapshot for version x.y can be used to read any DAML-LF version from 1.0 to x.y without suffering breaking changes (at the generated code level) often introduced in the current version.

Building

It produces several libraries containing code to encode / decode such definition, a Haskell one, and several Java ones:

$ bazel build //daml-lf/archive:daml_lf_archive_haskell_proto
$ bazel build //daml-lf/archive:daml_lf_dev_archive_java_proto
$ bazel build //daml-lf/archive:daml_lf_1_6_archive_java_proto

Editing the .proto definitions

When editing the proto definitions, you must make sure to not change them in a backwards-incompatible way. To make sure this doesn't happen:

  • DO NOT delete message fields;
  • DO NOT change the number of a message field or an enum value;
  • DO NOT change the type of a message field;

Note that "fields" include oneof fields. Also note that the "don't delete fields" rule is there not because they introduce a backwards incompatible change, but rather because after a field has been deleted another commiter might redefine it with a different type without realizing.

What is OK is renaming message fields while keeping the number and semantics unchanged. For example, if you have

message Foo {
  bytes blah = 1;
}

it's OK to change it to

message Foo {
  // this field is deprecated -- use baz instead!
  bytes blah_deprecated = 1;
  string baz = 2;
}

Conversion from the .proto to AST

The .proto definitions contain the serialized format for DAML-LF packages, however the code to convert from the .proto definitions to the actual AST lives elsewhere.