git.iko.soy/urbit.org
on master help sign in
Merge pull request #973 from urbit/dk-graph
c5a8381David Kerschner
tutorials/concepts/arch.mdnew file+21
compare with master
unifiedsplit
@@ -0,0 +1,21 @@
1++++
2+title = "Landscape Architecture"
3+weight = 4
4+template = "doc.html"
5++++
6+
7+# Landscape Architecture
8+
9+Landscape is designed using a particular userspace architecture mode consisting of stores, hooks, and threads.
10+
11+## Stores
12+
13+Stores are the first of the two gall models to form this architecture. A store is intended to maintain all the state required for an application or set of applications. Stores are intended to be only accessible by the ship it's running on. A store may not send pokes, peeks, or watches. A store should be treated and thought of as a database. While this is not enforced, it is the convention. The primary example of a store is `graph-store` which can be found at `app/graph-store`
14+
15+## Hooks
16+
17+Hooks are the second gall application used in this architecture and are not always present. Hooks are used for inter-app communication. This allows a separation of access control concerns and a division between those things which are able to be done by a store itself and those done by another gall application. `graph-store` in particular has two hooks `graph-pull-hook` for outgoing requests, and `graph-push-hook` for incoming requests.
18+
19+## Thread
20+
21+Threads are used when you want to have a series of transactions on a store where the order of those transactions matters. No guarantees are made about the order in which individual requests to a gall applications are processed; threads were introduced to address this. Threads do not maintain their own state while inactive and are instead run only upon request. A good example is `ted/graph/create`
tutorials/hoon/graph.mdnew file+158
@@ -0,0 +1,158 @@
1++++
2+title = "Graph Store"
3+weight = 3
4+template = "doc.html"
5++++
6+
7+# What is Graph Store?
8+
9+Graph Store is a gall implementation of a tree with additional virtual edges by allowing nodes to reference other nodes. It is most comparable to modern NoSQL databases in structure, though it allows for schemas to be written in the form of marks. It is used as the backend for an increasing number of Landscape applications, and is considered to be best used for social data.
10+
11+## Data types
12+
13+`graphs` is a map of `resource` to `marked-graph`
14+
15+`resource` is a pair of ship and term which is where the graph is hosted and the name of the resource. Every graph is it's own resource.
16+
17+`marked-graph` is a pair of `graph` and `(unit mark)`. The semantics of a graph are defined by the mark. This is just for validating the structure of the graph.
18+
19+`graph` is a `mop`, a mold builder for order map, of an atom and a `node`.
20+
21+`node` is a pair of `post` and `children`.
22+
23+`post` is defined in `sur/post`. It contains:
24+
25+```hoon
26++$ post
27+ $: author=ship
28+ =index
29+ time-sent=time
30+ contents=(list content)
31+ hash=(unit hash)
32+ =signatures
33+ ==
34+```
35+
36+
37+Most of these elements are self explanatory.
38+
39+`contents` is a `list` of `content` where `content` is a tagged union of text, url, code, or reference. This is where the contents of a publish post or an entry from links would be stored.
40+
41+`hash` is a `sham` hash of `validated-portion` which includes the parent hash, author, time sent, and contents.
42+
43+`signatures` is a set of `signature` which is used to cryptographically sign graph-store messages.
44+
45+When interacting with `graph-store` you will need to create an `update`
46+
47+`update` is a tagged union that currently only has one option
48+
49+`[%0 p=time q=update-0]`
50+
51+The reason this is a union is to serve as future proofing for version changes.
52+
53+An `update-0` can either be logged in the case of adding/removing nodes or signatures, or non-logged as in the case of adding/removing a graph. This logging refers to if this action adds an entry to the update log that is used to synchronize state between ships.
54+
55+Below is an excerpt of `sur/graph-store.hoon` at the time of writing, showing the definition of `logged-update-0` and `update-0` which should now be readable.
56+
57+```hoon
58++$ logged-update-0
59+ $% [%add-nodes =resource nodes=(map index node)]
60+ [%remove-nodes =resource indices=(set index)]
61+ [%add-signatures =uid =signatures]
62+ [%remove-signatures =uid =signatures]
63+ ==
64+::
65++$ update-0
66+ $% logged-update-0
67+ [%add-graph =resource =graph mark=(unit mark)]
68+ [%remove-graph =resource]
69+ ::
70+ [%add-tag =term =resource]
71+ [%remove-tag =term =resource]
72+ ::
73+ [%archive-graph =resource]
74+ [%unarchive-graph =resource]
75+ [%run-updates =resource =update-log]
76+ ::
77+ :: NOTE: cannot be sent as pokes
78+ ::
79+ [%keys =resources]
80+ [%tags tags=(set term)]
81+ [%tag-queries =tag-queries]
82+ ==
83+```
84+
85+
86+## Pokes
87+
88+Graph Store can be poked with a `graph-update`, which can:
89+ - add and remove a graph
90+
91+ `[%add-graph =resource =graph mark=(unit mark)]`
92+
93+ `[%remove-graph =resource]`
94+
95+ - add and remove a node to a graph
96+
97+ `[%add-nodes =resource nodes=(map index node)]`
98+
99+ `[%remove-nodes =resource indices=(set index)]`
100+
101+ - add and remove signatures
102+
103+ `[%add-signatures =uid =signatures]`
104+
105+ `[%remove-signatures =uid =signatures]`
106+
107+ - add and remove tags
108+
109+ `[%add-tag =term =resource]`
110+
111+ `[%remove-tag =term =resource]`
112+
113+ - archive and unarchive a graph
114+
115+ `[%archive-graph =resource]`
116+
117+ `[%unarchive-graph =resource]`
118+
119+## Scries
120+
121+What follows is a summary of the scrys available in graph-store
122+
123+- Fetch keys
124+ `[%x %keys ~]`
125+
126+- Fetch tags
127+ `[%x %tags ~]`
128+
129+- Fetch tag queries
130+ `[%x %tag-queries ~]`
131+
132+- Fetch a specific graph by resource
133+ `[%x %graph @ @ ~]`
134+
135+- Archive a specific graph by resource
136+ `[%x %archive @ @ ~]`
137+
138+- Fetch a subset of the graph by resource and start and end indices
139+ `[%x %graph-subset @ @ @ @ ~]`
140+
141+- Select a node from a graph by resource and index
142+ `[%x %node @ @ @ *]`
143+
144+- Fetch a subset of the children of a node by resource, index, and start and end indices
145+ `[%x %node-children-subset @ @ @ @ @ *]`
146+
147+- Fetch a subset of the update log by resource and start and end indices
148+ `[%x %update-log-subset @ @ @ @ ~]`
149+
150+- Fetch the update log by resource
151+ `[%x %update-log @ @ ~]`
152+
153+- Fetch the time of the last entry of the update log
154+ `[%x %peek-update-log @ @ ~]`
155+
156+## Marks
157+
158+There are marks in `mar/graph/validator` that are worth reviewing. They are each short enough to read quickly. They are used to validate the shapes of various nouns to be stored in the graph-store.