@@ -0,0 +1,331 @@
1++++
2+title = "Educational Guide: TodoMVC, Urbit Edition"
3+date = 2021-01-05
4+[taxonomies]
5+grant_type = ["bounties"]
6+grant_category = ["Documentation"]
7+[extra]
8+image = ""
9+description = "The goal of this bounty is to create a series of guides that can instruct a newcomer to Urbit as to how to go about creating a full-stack application from scratch."
10+reward = 10
11+assignee = "~rabsef-bicrym"
12+id = "1372696624"
13+completed = false
14+link = ""
15++++
16+
17+## Overview
18+
19+The goal of this bounty is to create a series of guides that can instruct a
20+newcomer to Urbit as to how to go about creating a full-stack application from
21+scratch. Together, they'll form a an introductory course to development on
22+Urbit.
23+
24+The application we'll demonstrate is the classic [TodoMVC](http://todomvc.com/examples/react/#/) app, which many
25+developers will be immediately familiar with. Even if a newcomer isn't familiar
26+with TodoMVC, it's quite easy to get a handle on it because it's a todo
27+list--surely *everyone* at this point has experience with todo lists.
28+
29+The classic TodoMVC example is a simple, functional and aesthetically pleasing
30+web-based todo list. What it lacks, however, is any form of persistence or
31+collaborative qualities; both of which are complex undertakings with traditional
32+web development. Our guide will demonstrate just how simple it is to give an
33+application these capabilities on Urbit, leading to a production-ready application.
34+
35+## Part 1: Serving Static Content
36+
37+Urbit ships with a web server called `eyre` which can do most of the things that
38+any other web server can, including serving static content, like a single page
39+application.
40+
41+By default, your Urbit is configured to serve Landscape at the `/` and
42+`/~landscape` URIs. Landscape is a single-page application that performs its own
43+client-side routing, but it's also possible to serve other applications as well.
44+For reference, see the
45+[debug](https://github.com/urbit/urbit/blob/959b585b384f66e54421f70f95d6d152077907bb/pkg/arvo/app/dbug.hoon)
46+application, which is a standalone application served from the `/~debug` URI.
47+
48+Our goal is to serve a pre-built `TodoMVC` application at the route `/~todo`.
49+It's important to note that this process could be applied to serve *any* HTML
50+file, whether it's a single-page application or not. With very little
51+imagination, it becomes easy to see how one could serve their personal blog at
52+`/~blog`, or a static "About Me" page at `/~about`.
53+
54+Urbit as a personal *file* server is useful in and of itself, but it will
55+require a basic understanding of Hoon, Clay, Gall, and the development
56+environment more generally. We'll break this guide into two sections: one
57+focused on getting a development environment set up, and another on actually
58+using it.
59+
60+### Part 1a: Setting up your Development Environment
61+
62+This section is fairly straightforward, as it has been covered many times
63+before. ~timluc-miptev's gall guide has a [great
64+overview](https://github.com/timlucmiptev/gall-guide/blob/master/workflow.md#three-workflow-options)
65+of how to work with fake ships efficiently, and the existing documentation has
66+plenty of material material.
67+
68+After the basic process of getting started, we'll want to cover how to develop
69+against a separate project repo, why you'd want to do that, how files are synced
70+with development ships (this is essentially the "deployment process"), and how
71+to use some basic generators.
72+
73+This guide should satisfy the following user stories, each of which start with
74+*As a new developer reading this guide, ...*:
75+
76+- I can install the Urbit binary.
77+- I can boot a fake ship for development.
78+- I can mount a pier.
79+- I can create new fake ships quickly.
80+- I have set up a project repository for my code.
81+- I understand the relationship between my project repository and my ship.
82+- I can copy files from my project repository to my ship.
83+- I can use a generator to inspect the contents of copied files.
84+
85+### Interlude: Hoon, Clay and Generators
86+
87+At this point it's probably worth digging a little deeper into some of the
88+things that were skimmed over in the last part. For starters, we did a good
89+amount of work with `clay` in the last lesson without really digging into what
90+it is or how it works.
91+
92+To start, let's talk a bit about what a pier is and its relation to the unix
93+filesystem, the way files are represented and how to navigate our pier from
94+within Urbit (*e.g.* `+ls %`, `+cat %/gen/cat`), and what `|mount` and `|commit`
95+are doing (at a high level).
96+
97+That might be a good segue into some basic Hoon via a super simple
98+generator--just enough to explain important syntax rules (*e.g.* ace v. gap),
99+the concept of runes, and provide links to places where you can learn more about
100+all of these things.
101+
102+### Part 1b: Serving the TodoMVC Application
103+
104+This section covers the process of serving static content to a URI within your
105+Urbit.
106+
107+First, you'll need to select a version of TodoMVC to use. For a modern and
108+widely understood version, I'd recommend the
109+[react-hooks](https://github.com/tastejs/todomvc/tree/master/examples/react-hooks)
110+version.
111+
112+The meat of this section is really in creating a simple gall agent that can be
113+used to serve the TodoMVC app directory. There are two ways you could go about
114+this: a) use the `file-server` agent, or b) use `eyre` directly, as is done with
115+the
116+[`dbug`](https://github.com/urbit/urbit/blob/959b585b384f66e54421f70f95d6d152077907bb/pkg/arvo/app/dbug.hoon)
117+application. Regardless of which option is chosen, a demonstration of the
118+conversion from one to the other would be an informative bonus lesson.
119+
120+By the end of this lesson:
121+
122+- I understand what Gall is at a high level.
123+- I know what an *agent* is.
124+- I know how to start a gall agent.
125+- I can serve a static file (or directory of files) from a URI on my ship.
126+
127+## Part 2: Data Modeling
128+
129+Now that we can serve a TodoMVC client, our next goal is to get it to actually
130+persist its data. In order to accomplish that we'll first have to spend some
131+time thinking about what our data actually looks like.
132+
133+There are two broad areas of our data model that we'll want to think about:
134+
135+1. What kind of data will represent our state, and
136+2. What kinds of actions we'll take to modify our state.
137+
138+After working through what we'll want to store and how we'll make changes to the
139+items in that store, we should then segue into how to represent our data as Hoon
140+types. The natural language description of our data model should lend itself to
141+a corresponding representation in Hoon, allowing the reader to mmediately ground
142+the Hoon in an understood domain.
143+
144+Through the course of discussing how to represent todos and the actions to take
145+upon them, the concepts of molds, standard library types and Hoon's type system
146+can be gently introduced.
147+
148+By the end of this lesson:
149+
150+- I know how to define my own structures.
151+- I know how to interpret the contents of my `/sur` directory.
152+- I have a basic understanding of Hoon's built-in types and structures.
153+- I know how to work with types in the Dojo.
154+
155+## Part 3: Taking action
156+
157+In the last lesson we defined our data model and the actions that will be taken
158+upon it. In this lesson we'll extend our original gall agent to be aware of our
159+data model.
160+
161+First, we'll cover the concept of a `poke` by handling the actions we defined in
162+the previous lesson. Along the way we'll need to set up our agent's state using
163+the types we defined previously. This portion will be an ideal time to introduce
164+the way a running agent can be poked from the dojo.
165+
166+After using `~&` to print our state upon various actions for a while, we'll
167+demonstrate the purpose of `scry` and extend our agent's `on-peek` arm to expose
168+ways of reading our state through direct, explicitly written scrys.
169+
170+By the end of this lesson:
171+
172+- I know how my application store its state.
173+- I know how to modify my application's state.
174+- I know how what a `poke` is, what it's for, and how to respond to one.
175+- I can poke my agent from the dojo.
176+- I know what `scry` is.
177+- I know how to expose scry paths in `on-peek`.
178+- I can `scry` my agent.
179+
180+## Part 4: Client Integration
181+
182+There's a lot to cover in this section, so it'll probably be best to break it up
183+into multiple sections. When integrating with a client we need to take care of
184+writing to our urbit and receiving data from it when things change, which means
185+we need to understand a) how to work with JSON data coming and going from our
186+ship to the browser, b) the architecture of how Urbit interacts with the outside
187+world over HTTP, and c) how to combine this knowledge into a React application.
188+
189+### Part 4a: Working with JSON in Hoon
190+
191+Most of the outside world speaks in JSON, which means that our Urbit, at least
192+for now, also needs to understand how to parse and serialize its data into this
193+structure.
194+
195+Parsing JSON in JavaScript is fairly straightforward, because JS is dynamically
196+typed; Hoon however, is statically typed, which makes our task a bit more
197+challenging.
198+
199+This guide should cover the process of both parsing JSON into Hoon data
200+structures and going the other way; turning Hoon data structures back into JSON
201+for sending to a client.
202+
203+By the end of this lesson:
204+
205+- I understand how to parse JSON in Hoon.
206+- I understand how to serialize Hoon into JSON.
207+- I understand how Hoon's type system relates to JSON parsing.
208+
209+### Part 4b: Client-server architecture & subscriptions
210+
211+Urbit makes heavy use of [Command Query Responsibility
212+Segregation](https://www.martinfowler.com/bliki/CQRS.html), or CQRS, which means
213+that our application can either issue *commands* or *queries*, but not both
214+simultaneously. This is contrasted with the more common Create Read Update
215+Delete (CRUD) model, in which our client is performing more direct manipulation
216+of records in a datastore. In Urbit, CQRS is paired with an [Event
217+Sourcing](https://www.martinfowler.com/eaaDev/EventSourcing.html) approach,
218+which means that our model for receiving information on the client is as a
219+stream of events.
220+
221+In a CRUD-style model, a Todo client would work as follows:
222+
223+1. Client queries (Read) for all todos to load into the page.
224+2. Client creates (Create) a new todo, which produces a response with the new
225+ todo.
226+3. The returned todo is added to the list.
227+
228+To contrast that with a CQRS + Event Sourcing model, we'd have the following:
229+
230+1. Client queries for all todos to load into the page.
231+2. Client sets up a subscription to receive any new todos that are created as
232+ events.
233+3. Client issues a command to create a new todo, which produces *no response,*
234+ since these responsibilities are separated.
235+4. After the new todo is created by the server, it emits an event which is
236+ picked up by the subscription in step (2), which is then added to the list.
237+
238+In practice, step (1) above is actually unnecessary, because the subscription in
239+step (2) can be configured to produce all todos as an initial event upon
240+instantiation.
241+
242+This pattern is implemented in Urbit with `poke`, `scry`, (covered in the last
243+section) and `watch`. A `poke` is our terminology for a command, a `scry` is for
244+querying, and `watch` is for subscribing to updates to our data when they
245+happen. We've already seen commands (`poke` and queries (`scry`), but
246+subscriptions (`watch`) are new, and much more commonly used than queries for
247+the same reason as our query above could be eliminated.
248+
249+Most web developers are much more familiar with the CRUD approach than they are
250+with the model that Urbit uses, so it's worth explaining why this more complex
251+approach is employed.
252+
253+The answer becomes clear when you consider Urbit's primary purpose, which is for
254+building distributed systems. In a distributed system, each agent is interacting
255+with many others through message passing. Imagine a chat application, in which
256+many agents are sending messages at once: creating a message and then adding the
257+returned message into your UI would work for messages *you* send, but would miss
258+any messages sent to you by other agents. In Urbit's approach, messages sent by
259+other agents are no different than those sent by your agent, meaning the same
260+implementation that handles one agent also handles *n* other agents.
261+
262+After explaining the rationale behind the approach, we'll want to demonstrate
263+how to extend `on-watch` to expose our todos, how to publish events to
264+subscribers, and how to get those events to a client. This lesson should focus
265+on making the mechanics of getting information from client to server and back
266+again clear--actually wiring up the TodoMVC frontend can wait until the next
267+lesson.
268+
269+By the end of this section:
270+
271+- I understand why a client sets up subscriptions instead of receiving direct
272+ data responses.
273+- I know how to expose data to subscribers via `on-watch`.
274+- I know how to publish state changes to subscribers.
275+- I understand how data travels from my client to my ship.
276+- I understand how data travels from my ship to my client.
277+
278+## Part 5: Collaboration
279+
280+The final phase of this guide should demonstrate the ways in which applications built on Urbit can communicate with one another. TodoMVC has no built-in interface mechanic for this, so you'll have to be a bit creative and think through what "multiplayer mode" looks like for TodoMVC. It's okay for this to be fairly simple, as our goal is to demonstrate how to extend an agent to work with others—not to build a great collaborative todo list.
281+
282+It's likely (and desirable) that the addition of this feature introduces the need to update our data model to handle todo lists belonging to multiple different ships (since before we were only dealing with our own todos). That makes this an ideal time to demonstrate how to upgrade our application's state to a new version, as one would do in the real-world with a major feature addition.
283+
284+By the end of this section:
285+
286+- I understand how to set up a subscription on a foreign ship.
287+- I understand how to tear down subscriptions.
288+- I understand how to store and represent data belonging to multiple ships.
289+- I understand how to upgrade my application's state.
290+
291+
292+## Deliverables
293+Your solution to this bounty should include:
294+
295+* A standalone repository with gall agent(s) that demonstrate each working part.
296+* A pull request against the [urbit/docs](https://github.com/urbit/docs) repository that adds guided tutorials for each section.
297+
298+## Expectations
299+
300+You will work closely with the director of urbit.org on this project. This means that regular check-in calls will be held to discuss the project's progress. Additionally, the you are expected to provide regular status updates on the project to the Urbit community via twice-monthly updates on this bounty.
301+
302+The director will ensure that you have access to the necessary resources to complete this project. All prior work done on this project will be explained and made available to you, and should questions arise that require the expertise of engineers at Tlon, time will be made (schedules permitting) for your questions to be answered either in writing or over a call.
303+
304+## Milestones
305+
306+
307+### Part 1: Serving Static Content
308+2 stars
309+Sample code and guide are completed and ready for merge against urbit/docs.
310+
311+
312+### Part 2: Data Modeling
313+1 stars
314+Sample code and guide are completed and ready for merge against urbit/docs.
315+
316+
317+### Part 3: Taking Action
318+1 stars
319+Sample code and guide are completed and ready for merge against urbit/docs.
320+
321+
322+### Part 4: Client Integration
323+3 stars
324+Sample code and guide are completed and ready for merge against urbit/docs.
325+
326+
327+### Part 5: Collaboration
328+3 stars
329+Sample code and guide are completed and ready for merge against urbit/docs.
330+
331+