@@ -6,11 +6,15 @@ template = "doc.html"
6insert_anchor_links = "right"
6insert_anchor_links = "right"
10Let's go through the type definitions of some of the most used types when working with Graph Store.
9Let's go through the type definitions of some of the most used types when working with Graph Store.
13Here's `sur/post.hoon`
13Here's `sur/post.hoon`
16+ <summary> Code Listing (Click to Expand)</summary>
15+$ index (list atom)
19+$ index (list atom)
16+$ uid [=resource =index]
20+$ uid [=resource =index]
@@ -41,6 +45,7 @@ Here's `sur/post.hoon`
41+$ reference
45+$ reference
42 $% [%graph group=resource =uid]
46 $% [%graph group=resource =uid]
43 [%group group=resource]
47 [%group group=resource]
48+ [%app =ship =desk =path]
@@ -50,11 +55,12 @@ Here's `sur/post.hoon`
50 [%code expression=cord output=(list tank)]
55 [%code expression=cord output=(list tank)]
51 [%reference =reference]
56 [%reference =reference]
59+$ index (list atom)
65+$ index (list atom)
60+$ uid [=resource =index]
66+$ uid [=resource =index]
@@ -67,6 +73,7 @@ Here's `sur/post.hoon`
67An index fragment is not an explicitly defined type, but since an index is `(list atom)`, it follows that the type of an index fragment is `atom`. This is what gives the developer the flexibility to use more than just numbers in an index: any type that is representable as an atom can be used in the index.
73An index fragment is not an explicitly defined type, but since an index is `(list atom)`, it follows that the type of an index fragment is `atom`. This is what gives the developer the flexibility to use more than just numbers in an index: any type that is representable as an atom can be used in the index.
69### Hashing (Part 1)
75### Hashing (Part 1)
71:: +sham (half sha-256) hash of +validated-portion
78:: +sham (half sha-256) hash of +validated-portion
72+$ hash @ux
79+$ hash @ux
@@ -78,10 +85,12 @@ An index fragment is not an explicitly defined type, but since an index is `(lis
78These types are used to cryptographically sign a given post, so that the host of some content cannot act as an imposter, and post content impersonating as someone else. `signature` represents a triple of signed message of hash, author, and author's life at time of posting. These can be used to cryptographically attest to a message. The implementation is a form of asymmetric/public-key encryption, where `q` and `r` are data necessary to look up a ship's public key on azimuth, which can be used to verify the validity of the message.
85These types are used to cryptographically sign a given post, so that the host of some content cannot act as an imposter, and post content impersonating as someone else. `signature` represents a triple of signed message of hash, author, and author's life at time of posting. These can be used to cryptographically attest to a message. The implementation is a form of asymmetric/public-key encryption, where `q` and `r` are data necessary to look up a ship's public key on azimuth, which can be used to verify the validity of the message.
80### Content Types
87### Content Types
82+$ reference
90+$ reference
83 $% [%graph group=resource =uid]
91 $% [%graph group=resource =uid]
84 [%group group=resource]
92 [%group group=resource]
93+ [%app =ship =desk =path]
@@ -94,7 +103,11 @@ These types are used to cryptographically sign a given post, so that the host of
97−`reference` is a type that models a reference to a graph object. Currently, the valid types of references are either a reference to a specific node in a graph, or a specific group.
106+`reference` is a type that models a reference to a graph object.
108+- `%graph` is a reference to a specific node in a given graph identified by its resource
109+- `%group` is a reference to a given resource itself
110+- `%app` is a reference to an application, hosted on `ship`, located at `desk`, with a `path` that the receiving application can use to link to some content (this is just `/` when unused).
99`content` enumerates all the possible content types that a post can have. The possible content types can be:
112`content` enumerates all the possible content types that a post can have. The possible content types can be:
@@ -107,6 +120,7 @@ These types are used to cryptographically sign a given post, so that the host of
107Currently, these are the only content types supported by Graph Store.
120Currently, these are the only content types supported by Graph Store.
112 $: author=ship
126 $: author=ship
@@ -125,6 +139,7 @@ Post is a fundamental type that represents what we normally think of as a post o
125An `indexed-post` is a post with an associated index fragment that can be used to validate a post's index with an index fragment that is expected at the end of the index list.
139An `indexed-post` is a post with an associated index fragment that can be used to validate a post's index with an index fragment that is expected at the end of the index list.
127### Hashing (Part 2)
141### Hashing (Part 2)
129+$ validated-portion
144+$ validated-portion
130 $: parent-hash=(unit hash)
145 $: parent-hash=(unit hash)
@@ -137,7 +152,6 @@ An `indexed-post` is a post with an associated index fragment that can be used t
138The parts of a `post` that are actually hashed to obtain a value of type the earlier type `hash`.
153The parts of a `post` that are actually hashed to obtain a value of type the earlier type `hash`.
141## Graph Store
155## Graph Store
143Here's `sur/graph-store.hoon`
157Here's `sur/graph-store.hoon`
@@ -149,15 +163,15 @@ Here's `sur/graph-store.hoon`
149+$ graph ((mop atom node) gth)
163+$ graph ((mop atom node) gth)
150+$ marked-graph [p=graph q=(unit mark)]
164+$ marked-graph [p=graph q=(unit mark)]
166++$ maybe-post (each post hash)
152−+$ node [=post children=internal-graph]
167++$ node [post=maybe-post children=internal-graph]
153+$ graphs (map resource marked-graph)
168+$ graphs (map resource marked-graph)
155−+$ tag-queries (jug term resource)
170++$ tag-queries (jug term uid)
157+$ update-log ((mop time logged-update) gth)
172+$ update-log ((mop time logged-update) gth)
158+$ update-logs (map resource update-log)
173+$ update-logs (map resource update-log)
161+$ internal-graph
175+$ internal-graph
162 $~ [%empty ~]
176 $~ [%empty ~]
163 $% [%graph p=graph]
177 $% [%graph p=graph]
@@ -169,18 +183,18 @@ Here's `sur/graph-store.hoon`
169 =tag-queries
183 =tag-queries
170 =update-logs
184 =update-logs
171 archive=graphs
185 archive=graphs
172− validators=(set mark)
186+ ~
175+$ update [p=time q=action]
189+$ update [p=time q=action]
177+$ logged-update [p=time q=logged-action]
191+$ logged-update [p=time q=logged-action]
180+$ logged-action
194+$ logged-action
181 $% [%add-graph =resource =graph mark=(unit mark) overwrite=?]
195 $% [%add-graph =resource =graph mark=(unit mark) overwrite=?]
182 [%add-nodes =resource nodes=(map index node)]
196 [%add-nodes =resource nodes=(map index node)]
183− [%remove-nodes =resource indices=(set index)]
197+ [%remove-posts =resource indices=(set index)]
184 [%add-signatures =uid =signatures]
198 [%add-signatures =uid =signatures]
185 [%remove-signatures =uid =signatures]
199 [%remove-signatures =uid =signatures]
@@ -189,8 +203,8 @@ Here's `sur/graph-store.hoon`
189 $% logged-action
203 $% logged-action
190 [%remove-graph =resource]
204 [%remove-graph =resource]
192− [%add-tag =term =resource]
206+ [%add-tag =term =uid]
193− [%remove-tag =term =resource]
207+ [%remove-tag =term =uid]
195 [%archive-graph =resource]
209 [%archive-graph =resource]
196 [%unarchive-graph =resource]
210 [%unarchive-graph =resource]
@@ -202,19 +216,18 @@ Here's `sur/graph-store.hoon`
202 [%tags tags=(set term)]
216 [%tags tags=(set term)]
203 [%tag-queries =tag-queries]
217 [%tag-queries =tag-queries]
206−+$ permissions
220++$ permissions
207 [admin=permission-level writer=permission-level reader=permission-level]
221 [admin=permission-level writer=permission-level reader=permission-level]
209:: $permission-level: levels of permissions in increasing order
223:: $permission-level: levels of permissions in increasing order
211:: %no: May not add/remove node
225:: %no: May not add/remove node
212:: %self: May only nodes beneath nodes that were added by
226:: %self: May only nodes beneath nodes that were added by
213:: the same pilot, may remove nodes that the pilot 'owns'
227:: the same pilot, may remove nodes that the pilot 'owns'
214:: %yes: May add a node or remove node
228:: %yes: May add a node or remove node
215+$ permission-level
229+$ permission-level
216 ?(%no %self %yes)
230 ?(%no %self %yes)
220</details>
233</details>
@@ -244,18 +257,20 @@ Here's `sur/graph-store.hoon`
247A graph is a loosely interconnected set of data which can reference each other and be arbitrarily nested and interconnected. This concept is based off of the mathematical concept of a graph: a collection of nodes, and connections between nodes called edges.
261A graph is a loosely interconnected set of data which can reference each other and be arbitrarily nested and interconnected. This concept is based off of the mathematical concept of a graph: a collection of nodes, and connections between nodes called edges.
249`graph` is a `mop`, a map which maintains ordering on its keys based on a sorting function also known as an ordered map. A graph's keys are `atom`s representing a node's index fragment and whose values are `node`s, where entries are sorted by largest valued keys first, using `gth` as the sorting function. This is the fundamental data structure used in Graph Store.
263`graph` is a `mop`, a map which maintains ordering on its keys based on a sorting function also known as an ordered map. A graph's keys are `atom`s representing a node's index fragment and whose values are `node`s, where entries are sorted by largest valued keys first, using `gth` as the sorting function. This is the fundamental data structure used in Graph Store.
251Here are some helpful Wikipedia pages for more info on what this data type represents:
265Here are some helpful Wikipedia pages for more info on what this data type represents:
252−- [Graph (abstract data type)](https://en.wikipedia.org/wiki/Graph_(abstract_data_type))
267+- [Graph (abstract data type)](<https://en.wikipedia.org/wiki/Graph_(abstract_data_type)>)
253- [Graph database](https://en.wikipedia.org/wiki/Graph_database#Background)
268- [Graph database](https://en.wikipedia.org/wiki/Graph_database#Background)
254−- [Graph (discrete mathematics)](https://en.wikipedia.org/wiki/Graph_(discrete_mathematics))
269+- [Graph (discrete mathematics)](<https://en.wikipedia.org/wiki/Graph_(discrete_mathematics)>)
256`node` represents a pair of a `post` and all of its children, which is an `internal-graph`.
271`node` represents a pair of a `post` and all of its children, which is an `internal-graph`.
258−`internal-graph` is a tagged union representing the state that children can be in. Either a `node` has children in the form of a `graph`, or does not have any and is labeled as `%empty`.
273+`internal-graph` is a tagged union representing the state that children can be in. Either a `node` has children in the form of a `graph`, or does not have any and is labeled as `%empty`.
260`marked-graph` is the pair of a `graph` and an optionally present `mark`, which is used by Graph Store to validate a graph against the provided validator.
275`marked-graph` is the pair of a `graph` and an optionally present `mark`, which is used by Graph Store to validate a graph against the provided validator.
@@ -263,8 +278,6 @@ Here are some helpful Wikipedia pages for more info on what this data type repre
264`network` is the highest level data structure used by Graph Store to represent all the information that the agent is aware of.
279`network` is the highest level data structure used by Graph Store to represent all the information that the agent is aware of.
268### Tag Queries
281### Tag Queries
@@ -274,13 +287,14 @@ Here are some helpful Wikipedia pages for more info on what this data type repre
274`tag-queries` is a mapping where the keys are `term`s and the values are a `set` of `resource`s (this pattern is called a `jug`). It is a simple tagging system that allows for various ad-hoc collections, similar to filesystem tags being used to sort different files/folders. While the type's name is `tag-queries`, there is no complex querying system as of now. Currently, you can add term/resources pairs into the tag queries, get a list of all terms in tag-queries, and get the whole `jug` out of Graph Store.
287`tag-queries` is a mapping where the keys are `term`s and the values are a `set` of `resource`s (this pattern is called a `jug`). It is a simple tagging system that allows for various ad-hoc collections, similar to filesystem tags being used to sort different files/folders. While the type's name is `tag-queries`, there is no complex querying system as of now. Currently, you can add term/resources pairs into the tag queries, get a list of all terms in tag-queries, and get the whole `jug` out of Graph Store.
276### Update (Part 1)
289### Update (Part 1)
278+$ update [p=time q=action]
292+$ update [p=time q=action]
280+$ logged-action
294+$ logged-action
281 $% [%add-graph =resource =graph mark=(unit mark) overwrite=?]
295 $% [%add-graph =resource =graph mark=(unit mark) overwrite=?]
282 [%add-nodes =resource nodes=(map index node)]
296 [%add-nodes =resource nodes=(map index node)]
283− [%remove-nodes =resource indices=(set index)]
297+ [%remove-posts =resource indices=(set index)]
284 [%add-signatures =uid =signatures]
298 [%add-signatures =uid =signatures]
285 [%remove-signatures =uid =signatures]
299 [%remove-signatures =uid =signatures]
@@ -289,8 +303,8 @@ Here are some helpful Wikipedia pages for more info on what this data type repre
289 $% logged-action
303 $% logged-action
290 [%remove-graph =resource]
304 [%remove-graph =resource]
292− [%add-tag =term =resource]
306+ [%add-tag =term =uid]
293− [%remove-tag =term =resource]
307+ [%remove-tag =term =uid]
295 [%archive-graph =resource]
309 [%archive-graph =resource]
296 [%unarchive-graph =resource]
310 [%unarchive-graph =resource]
@@ -307,9 +321,10 @@ Here are some helpful Wikipedia pages for more info on what this data type repre
308The `update` type is what is used to interact with Graph Store. It is used both to update subscribers with data (outgoing data) and to write to Graph Store itself (incoming data). All actions are meant to be sent as pokes to Graph Store in the form of a `action` (encapsulated by a `graph-update`), except for the last three actions ,`%keys`, `%tags`, and `%tag-queries`, which must be sent scry requests. All actions defined here allow you to create/read/update/delete various objects in a running Graph Store gall agent. An `action` encapsulates all `logged-action`; any `logged-action` is an `action`, but not necessarily the other way around.
322The `update` type is what is used to interact with Graph Store. It is used both to update subscribers with data (outgoing data) and to write to Graph Store itself (incoming data). All actions are meant to be sent as pokes to Graph Store in the form of a `action` (encapsulated by a `graph-update`), except for the last three actions ,`%keys`, `%tags`, and `%tag-queries`, which must be sent scry requests. All actions defined here allow you to create/read/update/delete various objects in a running Graph Store gall agent. An `action` encapsulates all `logged-action`; any `logged-action` is an `action`, but not necessarily the other way around.
310−If you want to check out a relevant code listing to see how graph store handles these pokes, check out the `graph-update` arm in the poke handler at [`app/graph-store.hoon`](https://github.com/urbit/urbit/blob/e2ad6e3e9219c8bfad62f27f05c7cac94c9effa8/pkg/arvo/app/graph-store.hoon#L221-L227)
324+<!-- todo link to reference explaining pokes -->
312### Update (Part 2)
326### Update (Part 2)
314+$ update-log ((mop time logged-update) gth)
329+$ update-log ((mop time logged-update) gth)
315+$ update-logs (map resource update-log)
330+$ update-logs (map resource update-log)
@@ -326,6 +341,7 @@ It is important to note that the source of truth is actually the `update-logs`,
326The reason for having the main CRUD actions being `logged-update`s is so that Graph Store knows which order to process the log entries in when it is rebuilding its current state. The time associated with the `logged-update` is a way of specifying the canonical order to process operations. All other actions that aren't part of logged-update stand on their own and don't need a timestamp in order to properly apply them.
341The reason for having the main CRUD actions being `logged-update`s is so that Graph Store knows which order to process the log entries in when it is rebuilding its current state. The time associated with the `logged-update` is a way of specifying the canonical order to process operations. All other actions that aren't part of logged-update stand on their own and don't need a timestamp in order to properly apply them.
328### Permissions
343### Permissions
330+$ permissions
346+$ permissions
331 [admin=permission-level writer=permission-level reader=permission-level]
347 [admin=permission-level writer=permission-level reader=permission-level]
@@ -341,4 +357,3 @@ The reason for having the main CRUD actions being `logged-update`s is so that Gr
343These are the types from the permissioning system explained earlier. `permissions` is just a length-3 cell of `permissions-level`s for Admins, Writers, and Readers respectively.
359These are the types from the permissioning system explained earlier. `permissions` is just a length-3 cell of `permissions-level`s for Admins, Writers, and Readers respectively.