git.iko.soy/rowtype-yoga/purescript-protobuf
on v0.9.1 help sign in
rowtype-yoga/purescript-protobuf/README.md
190 lines 7.2 kB history raw
1# purescript-protobuf
3[![Test](https://github.com/xc-jp/purescript-protobuf/workflows/Test/badge.svg?branch=master)](https://github.com/xc-jp/purescript-protobuf/actions)
4[![Pursuit](http://pursuit.purescript.org/packages/purescript-protobuf/badge)](http://pursuit.purescript.org/packages/purescript-protobuf/)
6Purescript library and code generator for
7[Google Protocol Buffers Version 3](https://developers.google.com/protocol-buffers/docs/reference/proto3-spec).
9This library operates on
10[`ArrayBuffer`](https://pursuit.purescript.org/packages/purescript-arraybuffer-types/docs/Data.ArrayBuffer.Types#t:ArrayBuffer), so it will run both
11[in Node](https://pursuit.purescript.org/packages/purescript-node-buffer/docs/Node.Buffer.Class)
12and in browser environments.
14## Code Generation
16The `shell.nix` environment provides
18* The Purescript toolchain
19* The [`protoc`](https://github.com/protocolbuffers/protobuf/blob/master/src/README.md) compiler
20* The `protoc-gen-purescript` executable plugin for `protoc` on the `PATH` so that
21 [`protoc` can find it](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.compiler.plugin).
23```
24$ nix-shell
26Purescript Protobuf development environment.
27To build purescript-protobuf, run:
29 npm install
30 spago build
32To test purescript-protobuf, run:
34 protoc --purescript_out=./test/generated test/*.proto
35 spago -x test.dhall build
36 spago -x test.dhall test
38To generate Purescript .purs files from .proto files, run:
40 protoc --purescript_out=path_to_output *.proto
42[nix-shell]$
43```
45## Writing programs with the generated code
47None of the modules in this package should be imported directly in our program.
48Rather, we'll import the message modules from the generated `.purs` files,
49as well as modules for reading and writing `ArrayBuffer`s.
51For example, a message in a `.proto` file declared as
53```
54message MyMessage {
55 sint32 my_field = 1;
57```
59will export these four names in the generated `.purs` modules.
611. A message record type
62 * `type MyMessageR = { my_field :: Maybe Int }`.
632. A message data type
64 * `newtype MyMessage = MyMessage MyMessageR`.
653. A message encoder which works with
66 [__purescript-arraybuffer-builder__](http://pursuit.purescript.org/packages/purescript-arraybuffer-builder/)
67 * `putMyMessage :: forall m. MonadEffect m => MyMessage -> PutM m Unit`
684. A message decoder which works with
69 [__purescript-parsing-dataview__](http://pursuit.purescript.org/packages/purescript-parsing-dataview/)
70 * `parseMyMessage :: forall m. MonadEffect m => Int -> ParserT DataView m MyMessage`
72Then, in our program, our imports will look something like this.
75```purescript
76import Generated.Module (MyMessage(..), putMyMessage, parseMyMessage)
77import Text.Parsing.Parser (runParserT)
78import Data.ArrayBuffer.Builder (execPutM)
79```
81The generated code modules will import modules from this
82package.
84The generated code depends on packages
86```
87 , "parsing"
88 , "parsing-dataview"
89 , "arraybuffer-types"
90 , "arraybuffer"
91 , "arraybuffer-builder"
92 , "uint"
93 , "long"
94 , "text-encoding"
95```
97which are in
98[__package-sets__](https://github.com/purescript/package-sets),
99except for
100[__purescript-longs__](https://pursuit.purescript.org/packages/purescript-longs)
101(see `spago.dhall` in this package for the particulars).
103It also depends on the Javascript package
104[__long__](https://www.npmjs.com/package/long).
106### Generated message instances
108We cannot easily derive common instances like `Eq` for the
109generated message types because
1101. The types [might be recursive](https://github.com/purescript/documentation/blob/master/errors/CycleInDeclaration.md).
1112. The types might contain fields of type
112 [`ArrayBuffer`](https://pursuit.purescript.org/packages/purescript-arraybuffer-types/docs/Data.ArrayBuffer.Types#t:ArrayBuffer)
113 which doesn't have those instances.
115All of the generated message types have an instance of
116[`Generic`](https://pursuit.purescript.org/packages/purescript-generics-rep/docs/Data.Generic.Rep#t:Generic).
117This allows us to sometimes use
118[`genericEq`](https://pursuit.purescript.org/packages/purescript-generics-rep/docs/Data.Generic.Rep.Eq#v:genericEq)
119and
120[`genericShow`](https://pursuit.purescript.org/packages/purescript-generics-rep/docs/Data.Generic.Rep.Show#v:genericShow)
121on the generated messages, if neither of two conditions above apply to
122our particular message types.
124All of the generated message types have an instance of
125[`NewType`](https://pursuit.purescript.org/packages/purescript-newtype/docs/Data.Newtype#t:Newtype).
127### Interpreting invalid encoding parse errors
129When the decode parser encounters an invalid encoding in the protobuf input
130stream then it will fail to parse.
132When
133[`Text.Parsing.Parser.ParserT`](https://pursuit.purescript.org/packages/purescript-parsing/docs/Text.Parsing.Parser#t:ParserT)
134fails it will return a `ParseError String (Position {line::Int,column::Int})`.
135The byte offset at which the invalid encoding occured is given by the
136formula `column - 1`.
138## Features
140We aim to support
141[__proto3__](https://developers.google.com/protocol-buffers/docs/reference/proto3-spec).
142Many __proto2__-syntax descriptor files will
143also work, as long as they don't have __proto2__ features.
145We don't support
146[extensions](https://developers.google.com/protocol-buffers/docs/proto?hl=en#extensions).
148The generated optional record fields will use `Nothing` instead of the
149[default values](https://developers.google.com/protocol-buffers/docs/proto3?hl=en#default).
151We do not preserve
152[unknown fields](https://developers.google.com/protocol-buffers/docs/proto3?hl=en#unknowns).
154We do not support
155[services](https://developers.google.com/protocol-buffers/docs/proto3?hl=en#services).
157### Imports
159The code generator will use the `package` statement in the `.proto` file
160and the base file name as the Purescript module name for that file.
162The Protobuf
163[`import`](https://developers.google.com/protocol-buffers/docs/proto3#importing_definitions)
164statement allows Protobuf messages to have fields
165consisting of Protobuf messages imported from another file, and qualified
166by the package name in that file. In order to generate
167the correct Purescript module name qualifier on the types of imported message
168fields, the code generator must be able to lookup the package name
169statement in the imported file.
171For that reason, we can only use top-level
172(not [nested](https://developers.google.com/protocol-buffers/docs/proto3#nested))
173`message` and `enum` types from an `import`.
175The generated Purescript code will usually have module imports which cause
176the `purs` compiler to emit warnings. Sorry.
178## Performance
180The implementation is simple and straightforward. We haven't done
181any special optimizations. For example, when encoding a protobuf varint, we
182allocate a list of new one-byte `ArrayBuffers`s and then copy them all into
183position in the final `ArrayBuffer`. For another example, when decoding a
184packed field of numbers, we build a list of the numbers, and then copy them
185all into the final `Array`. Also, this whole library is very stack-unsafe.
186This may all be improved in later versions.
188## Contributing
190Pull requests welcome.