git.iko.soy/urbit.org
on master help sign in
Merge pull request #1091 from urbit/ames-encryption
05aaac6poprox
arvo/ames/cryptography.mdnew file+145
compare with master
unifiedsplit
@@ -0,0 +1,145 @@
1++++
2+title = "Cryptography"
3+weight = 5
4+template = "doc.html"
5++++
6+
7+Here we give a technical overview of how Ames implements cryptography.
8+
9+## Summary
10+
11+By default, all [packets are encrypted](#packets) with 256-bit
12+[AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) symmetric key
13+encryption, whose key is obtained by [Diffie-Hellman key exchange](#key-exchange),
14+with public/private keys generated using elliptic curve
15+[Curve25519](https://en.wikipedia.org/wiki/Curve25519). The only exception to
16+this are comet [self-attestation packets](#comets), which are unencrypted.
17+
18+All packets are also [signed](#packets) using [Elliptic Curve Digital Signature
19+Algorithm](https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm).
20+
21+The `$ames-state` includes a [`+acru:ames`](#crypto-core) core, an interface
22+core into which one of the cryptosuites in `zuse` may be implemented. Some, but
23+not all, of the calls to cryptography-related tasks (encryption, signing,
24+decrypting, and authenticating) are peformed utilizing the `+acru` core.
25+
26+## Packet encryption and authentication {#packets}
27+
28+Each Urbit ship possesses two networking keypairs: one for encryption, and one
29+for authentication. We often refer to these two keypairs as though they were a
30+single keypair because they are stored as a single atom. [Elliptic Curve
31+Diffie-Hellman](https://en.wikipedia.org/wiki/Elliptic-curve_Diffie%E2%80%93Hellman)
32+is used for encryption, while [Elliptic Curve Digital Signature
33+Algorithm](https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm)
34+is used for authentication
35+
36+The encrypted payload of each packet is a `$shut-packet`, which is the `+jam` of
37+a cell with the `bone`, message number, and message fragment or ack (see
38+[Ames](@/docs/ames/ames.md) for more information on packet structure). The
39+message fragment is signed using the authentication key. It is encrypted using
40+[`+en:crub:crypto`](@/docs/arvo/reference/cryptography.md#en) found in
41+`sys/zuse.hoon`, which utilizes the 256-bit AES-SIV algorithm.
42+
43+## Diffie-Hellman key exchange {#key-exchange}
44+
45+For each foreign ship a given ship has communicated with, `$ames-state` contains a
46+`$peer-state`, inside which the `$symmetric-key` (an atom which nests under
47+`@uw`) is utilized for encrypting all Ames packets shared between the two ships.
48+The `symmetric-key` is derived using [`+shar:ed:crypto`](@/docs/arvo/reference/cryptography.md#shar) found in `sys/zuse.hoon`,
49+which is an arm utilized for generating the symmetric key for elliptic curve
50+[Diffie-Hellman key
51+exchange](https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange)
52+with [curve25519](https://en.wikipedia.org/wiki/Curve25519). Briefly, each ship
53+in a two-way conversation computes the shared symmetric key for that
54+conversation by computing the product of their own private key and the public
55+key of the other party.
56+
57+## Comet self-attestation {#comets}
58+
59+Recall that the `@p` of a comet is the hash of their 128-bit public key cast as
60+a `@p`. Since the public key of a comet is not stored on Azimuth, a comet proves
61+its identity with an "attestation packet". This is an unencrypted packet whose
62+payload is the comet's signature created with its private key. This is the only
63+circumstance under which a ship will send an unencrypted packet. The signature
64+is generated with
65+[`+sign:as:crub`](@/docs/arvo/reference/cryptography.md#sign-as) found in
66+`sys/zuse.hoon`.
67+
68+Upon hearing an attestation packet, the receiving ship will generate a symmetric
69+key for communications with the comet, according to the [key
70+exchange](#key-exchange) protocol.
71+
72+The fact that the first packet exchanged between a comet and another ship must
73+be an attestation packet is why comets are unable to initiate communication with
74+one another, and also why comets must be the first to initiate communication
75+with a non-comet. This is a technical limitation with a planned workaround.
76+
77+## `+acru:ames` {#crypto-core}
78+
79+The `+crypto-core` in `$ames-state` is an `+acru:ames` core, a
80+[lead](@/docs/hoon/reference/advanced.md#dry-polymorphism-and-core-nesting-rules)
81+interface core for asymmetric cryptosuites found in `sys/lull.hoon` which
82+handles encryption, decryption, signing, and verifying. In practice, the only
83+cryptosuite in use is [`+crub:crypto`](#crub), which implements [Suite B
84+Cryptography](https://en.wikipedia.org/wiki/NSA_Suite_B_Cryptography).
85+
86+```hoon
87+ ++ acru $_ ^? :: asym cryptosuite
88+ |% :: opaque object
89+ ++ as ^? :: asym ops
90+ |% ++ seal |~([a=pass b=@] *@) :: encrypt to a
91+ ++ sign |~(a=@ *@) :: certify as us
92+ ++ sure |~(a=@ *(unit @)) :: authenticate from us
93+ ++ tear |~([a=pass b=@] *(unit @)) :: accept from a
94+ -- ::as ::
95+ ++ de |~([a=@ b=@] *(unit @)) :: symmetric de, soft
96+ ++ dy |~([a=@ b=@] *@) :: symmetric de, hard
97+ ++ en |~([a=@ b=@] *@) :: symmetric en
98+ ++ ex ^? :: export
99+ |% ++ fig *@uvH :: fingerprint
100+ ++ pac *@uvG :: default passcode
101+ ++ pub *pass :: public key
102+ ++ sec *ring :: private key
103+ -- ::ex ::
104+ ++ nu ^? :: reconstructors
105+ |% ++ pit |~([a=@ b=@] ^?(..nu)) :: from [width seed]
106+ ++ nol |~(a=ring ^?(..nu)) :: from ring
107+ ++ com |~(a=pass ^?(..nu)) :: from pass
108+ -- ::nu ::
109+ -- ::acru ::
110+```
111+
112+As the `+acru` core is merely an interface, the details on how it is implemented
113+may vary according to the cryptosuite. We summarize what each
114+core is utilized for here, but see
115+[`crub:crypto`](@/docs/arvo/reference/cryptography.md#crub) for more details on
116+how the specific cryptosuite utilized by Ames is implemented.
117+
118+#### `+as:acru`
119+
120+This core is used for the standard asymmetric cryptographic operations: encrypting
121+(`+seal`), signing (`+sign`), authenticating (`+sure`), and decrypting (`+tear`).
122+
123+#### `+ex:acru`
124+
125+This core is used to extract keys and their fingerprints. `+sec` is the secret key (which
126+may be empty), `+pub` is the public key associated to the secret key, `+pac` is
127+the fingerprint associated to the secret key, and `+fig` is the fingerprint
128+associated to the public key. We note that when the core contains both
129+encryption and authentication keys, they are typically concatenated to be
130+returned as a single atom.
131+
132+#### `+nu:acru`
133+
134+This core contains constructors for the `+acru` core. `+pit:nu` is used to
135+construct an `+acru` core with both a private and public key (i.e. both
136+`+sec:ex` and `+pub:ex` are set) from a bitlength and seed. `+nol:nu` can then be
137+called from an `+acru` core created with `+pit:nu` to get an `+acru` core with
138+only the secret key, while `+com:nu` can be called to get an `+acru` core with
139+only the public key.
140+
141+#### `+en:acru`, `+de:acru`, and `+dy:acru`
142+
143+These arms are for symmetric encryption (`+en`) and decryption (`+de` and
144+`+dy`). The difference between the decryption arms is that `+de` returns null
145+and `+dy` crashes upon failure.
arvo/reference/_index.mdnew file+12
@@ -0,0 +1,12 @@
1++++
2+title = "Reference"
3+weight = 1
4+sort_by = "weight"
5+template = "sections/docs/chapters.html"
6++++
7+
8+[Cryptography](@/docs/arvo/reference/cryptography.md)
9+
10+An overview of where Arvo uses cryptography, what algorithms are used, and how
11+they are implemented.
12+
arvo/reference/cryptography.mdnew file+220
@@ -0,0 +1,220 @@
1++++
2+title = "Cryptography"
3+weight = 10
4+template = "doc.html"
5++++
6+
7+This is a summary of the cryptography functions found in `sys/zuse.hoon`, Arvo's
8+standard library. This page currently only documents cryptographic functions
9+directly utilized by [Ames](@/docs/arvo/ames/ames.md). `zuse` also contains
10+cryptographic functions relevant to Ethereum such as the `+keccak:crypto` core,
11+but they are currently undocumented.
12+
13+Documentation for [Insecure Hashing](@/docs/hoon/reference/stdlib/2e.md) and the
14+[SHA Hash Family](@/docs/reference/stdlib/3d.md) is found in the Hoon standard
15+library reference.
16+
17+## Summary
18+
19+`zuse` contains several cryptosuites. The ones utilized by Ames are
20+[`+ed:crypto`](#ed), [`+aes:crypto`](#aes), and [`+crub:crypto`](#crub), with
21+the latter being the only one which is implemented as an [`+acru:ames`](@/docs/arvo/ames/cryptography/#crypto-core)-shaped core.
22+
23+## `+crub:crypto` {#crub}
24+
25+`+crub:crypto` implements an
26+[`+acru:ames`](@/docs/arvo/ames/cryptography/#crypto-core) core that implements
27+[Suite B Cryptography](https://en.wikipedia.org/wiki/NSA_Suite_B_Cryptography).
28+
29+It utilizes AES symmetric key encryption and decryption from [`+aes:crypto`](#aes)
30+implemented using the Diffie-Hellman key exchange protocol, elliptic curve
31+digital signature algorithm (ECDSA) signing and verification with [`+ed:crypto`](#ed),
32+and generates public/private key pairs using elliptic curve cryptography with
33+`+ed:crypto`.
34+
35+A `+crub:crypto` core's payload contains public encryption and authentication
36+keys and optional secret encryption and authentication keys.
37+```hoon
38+ ++ crub !:
39+ ^- acru
40+ =| [pub=[cry=@ sgn=@] sek=(unit [cry=@ sgn=@])]
41+ |%
42+ ...
43+```
44+`+crub` cores (because they follow the `+acru` interface) are typically created using one of the constructors in [`+nu:crub`](#nu).
45+
46+### `+seal:as`
47+
48+```hoon
49+ ++ seal ::
50+ |= [bpk=pass msg=@]
51+ ...
52+```
53+
54+Forms a symmetric key using Diffie-Hellman key exchange with the secret key
55+stored at `sgn.u.sek` and a public key `bpk`. Then `+sign`s `msg`, encrypts the
56+signed message using `+en:siva:aes` with the symmetric key, and then `+jam`s it.
57+
58+Crashes if `sek` is null.
59+
60+### `+sign:as` {#sign:as}
61+
62+```hoon
63+ ++ sign ::
64+ |= msg=@
65+ ...
66+```
67+
68+Signs message `msg=@` using the secret authentication key `sgn.u.sek`, then forms a
69+cell `[signed-message msg]` and `+jam`s it.
70+
71+Crashes if `sek` is null.
72+
73+### `+sure:as`
74+
75+```hoon
76+ ++ sure ::
77+ |= txt=@
78+ ...
79+```
80+
81+`+cue`s `txt` to get a signature `sig=@` and message `msg=@`. Verifies that
82+`sig` was `msg` signed using the secret key associated to the public key stored
83+at `sgn.pub`. Returns `(unit msg)` if so, null otherwise.
84+
85+### `+tear:as`
86+
87+```hoon
88+ ++ tear ::
89+ |= [bpk=pass txt=@]
90+ ...
91+```
92+
93+Forms a secret symmetric key using Diffie-Hellman key exchange using the secret
94+key `cry.u.sek` and encryption key part of the public key `bpk` (which here is a
95+concatenation of both the encryption and authentication public keys). `+cue`s
96+`txt` and decrypts it using `+de:siva:aes` with the symmetric key. If decryption
97+is successful, verifies the decrypted message using authentication key part of
98+`bpk`, and returns it if so. Returns null otherwise.
99+
100+Crashes if `sek` is null.
101+
102+### `+de`
103+
104+```hoon
105+ ++ de :: decrypt
106+ |= [key=@J txt=@]
107+ ...
108+```
109+
110+`+cue`s `txt` then decrypts with the symmetric key `key` using `+de:sivc:aes`.
111+Returns null in case of failure.
112+
113+### `+dy`
114+
115+```hoon
116+ ++ dy :: need decrypt
117+ |= [key=@J cph=@]
118+ ...
119+```
120+
121+Same as `+dy`, but crashes in case of failure.
122+
123+### `+en` {#en}
124+
125+```hoon
126+ ++ en :: encrypt
127+ |= [key=@J msg=@]
128+```
129+
130+Encrypts `msg` with the symmetric key `key` using `en:sivc:aes`, then `+jam`s
131+it.
132+
133+### `+fig:ex`
134+
135+Returns the fingerprint (SHA-256) of `+pub:ex`.
136+
137+### `+pac:ex`
138+
139+Returns the fingerprint (SHA-256) of `+sec:ex`. Crashes if `sek` is null.
140+
141+### `+pub:ex`
142+
143+Returns the concatenation of `sgn.pub` and `cry.pub`.
144+
145+### `+sec:ex`
146+
147+Returns the concatenation of `sgn.u.sek` and `cry.u.sek`.
148+
149+### `+pit:nu` {#nu}
150+
151+```hoon
152+ ++ pit :: create keypair
153+ |= [w=@ seed=@]
154+ ...
155+```
156+
157+Creates a `+crub` core with encryption and authentication public/private keypairs
158+generated from a bitwidth `w` and `seed`. The private keys are generated with
159+SHA-512, while `+puck:ed:crypto` is used to derive the public keys from the
160+private keys.
161+
162+This is how one typically generates a brand new `+crub` core for signing and
163+encrypting your own messages.
164+
165+### `+nol:nu`
166+
167+```hoon
168+ ++ nol :: activate secret
169+ |= a=ring
170+ ...
171+```
172+
173+Takes in a `ring` from a `+sec:ex:crub` and generates a new `+crub` core with
174+`sek` taken from `+sec:ex` and `pub` generated with `+puck:ed:crypto`. Crashes
175+if `+sec:ex` is not a `+crub` secret key.
176+
177+### `+com:nu`
178+
179+```hoon
180+ ++ com :: activate public
181+ |= a=pass
182+ ...
183+```
184+
185+Takes in a `pass` from a `+pub:ex:crub` and generates a new `+crub` core with
186+`pub` taken from `+pub:ex` and null `sek`.
187+
188+## `+ed:crypto` {#ed}
189+
190+This core contains cryptographic primitives and helper functions for elliptic
191+curve cryptography with [Curve25519](https://en.wikipedia.org/wiki/Curve25519).
192+
193+`+ed:crypto` is primarily used to generate public/private keypairs from a seed
194+for use with [elliptic curve
195+Diffie-Hellman](https://en.wikipedia.org/wiki/Elliptic-curve_Diffie%E2%80%93Hellman)
196+key agreements as well as [Elliptic Curve Digital Signature
197+Algorithm](https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm).
198+These functionalities are ultimately utilized by [`+crub:crypto`](#crub), the
199+only cryptosuite which [Ames](@/docs/arvo/ames/ames.md) makes use of.
200+
201+Most gates in `+ed:crypto` are [jetted](@/docs/vere/jetting.md), meaning that an
202+external reference library is utilized whenever these functions are called,
203+rather than running in Nock natively. See the [Vere
204+documentation](@/docs/vere/cryptography/#ed) for more information about the
205+library utilized by jets.
206+
207+## `+aes:crypto` {#aes}
208+
209+This core contains cryptographic primitives and helper functions for
210+[AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) symmetric key
211+encryption and decryption. As is the case with `ed:crypto`, these
212+functionalities are utilized by [`+crub:crypto`](#crub), and most gates are
213+jetted. See also the Vere documentation on [AES
214+SIV](@/docs/vere/cryptography/#aes) for more information about the library
215+utilized by jets.
216+
217+This core contains several doors, each one used for a different variation of AES
218+according to key size and mode. The only ones currently in use are
219+`+siva:aes:crypto` and `+sivc:aes:crypto`, which are 128-bit and
220+256-bit modes of [`AES-SIV`](https://www.aes-siv.com) respectively.
system-overview/cryptography.md+10−2
@@ -99,8 +99,8 @@ of private networking keys and retrieval of public networking keys utilized by
99Ames. The Jael vane of planets, stars, and galaxies are responsible for
99Ames. The Jael vane of planets, stars, and galaxies are responsible for
100distributing the public keys of their moons (ultimately via Ames).
100distributing the public keys of their moons (ultimately via Ames).
101
101
102−`zuse` is the Hoon standard library. It contains cryptographic functions which
102+`zuse` is part of the standard library. It contains cryptographic functions
103−are utilized by Ames. All cryptographic primitives are
103+which are utilized by Ames. All cryptographic primitives are
104[jetted](@/docs/vere/jetting.md) in Vere with standard vetted implementations of
104[jetted](@/docs/vere/jetting.md) in Vere with standard vetted implementations of
105cryptographic libraries.
105cryptographic libraries.
106
106
@@ -114,3 +114,11 @@ infrastructure utilized by Urbit. `azimuth-tracker` obtains networking public
114keys for planets, stars, and galaxies from this store, which are then stored in
114keys for planets, stars, and galaxies from this store, which are then stored in
115Jael and utilized by Ames for end-to-end encrypted communication.
115Jael and utilized by Ames for end-to-end encrypted communication.
116
116
117+### Additional documentation
118+
119+The following pages contained more detailed information about the cryptography
120+utilized by each of the system components.
121+
122+ - [Ames](@/docs/arvo/ames/cryptography.md)
123+ - [Zuse](@/docs/arvo/reference/cryptography.md)
124+ - [Vere](@/docs/vere/cryptography.md)
vere/_index.md+4
@@ -35,3 +35,7 @@ A walkthrough of each of the `u3` modules.
35## [How to write a jet](@/docs/vere/jetting.md)
35## [How to write a jet](@/docs/vere/jetting.md)
36
36
37A jetting guide by for new Urbit developers.
37A jetting guide by for new Urbit developers.
38+
39+## [Cryptography](@/docs/vere/cryptography.md)
40+
41+References on the cryptography libraries utilized by jets.
vere/cryptography.mdnew file+44
@@ -0,0 +1,44 @@
1++++
2+title = "Cryptography"
3+weight = 6
4+template = "doc.html"
5++++
6+
7+All cryptographic primitives utilized by Arvo are
8+[jetted](@/docs/vere/jetting.md). This is done for performance-related reasons
9+in other parts of the system, but for cryptography this is also extremely
10+important because it allows us to utilize standard reference implementations for
11+the primitives written in C.
12+
13+All jets related to encryption may be found in `pkg/urbit/jets/e/`.
14+
15+In this section we review what specific implementations are utilized. At the
16+moment, only libraries directly related to Ames are documented here, though we
17+note that there are jets for other cryptographic functions such as the [SHA
18+Hash Family](@/docs/reference/stdlib/3d.md) as well.
19+
20+### Ed25519 {#ed}
21+
22+Urbit implements [Ed25519](http://ed25519.cr.yp.to/) based on the SUPERCOP
23+"ref10" implementation. Additionally there is key exchanging and scalar addition
24+included to further aid building a PKI using Ed25519. All code is licensed under
25+the permissive zlib license.
26+
27+All code is pure ANSI C without any dependencies, except for the random seed
28+generation which uses standard OS cryptography APIs (CryptGenRandom on Windows,
29+`/dev/urandom` on nix).
30+
31+### AES-SIV {#aes}
32+
33+The library we utilize for AES-SIV is an
34+[RFC5297](https://tools.ietf.org/html/rfc5297)-compliant C implementation of
35+AES-SIV written by Daniel Franke on behalf of [Akamai
36+Technologies](https://www.akamai.com). It is published under the [Apache License
37+(v2.0)](https://www.apache.org/licenses/LICENSE-2.0). It uses OpenSSL for the
38+underlying [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) and
39+[CMAC](https://en.wikipedia.org/wiki/One-key_MAC) implementations and follows a
40+similar interface style.
41+
42+While the jets are found in `pkg/urbit/jets/e`, the statically-linked package is
43+found at `pkg/libas_siv/`.
44+