on master help sign in
Merge branch 'urbit:master' into master
commit87be46de56e819df79717a22895e70b75ac9668c authorthelifeandtimes <tlatfcp@protonmail.com> dateSat 16 Apr 2022, 23:59 (5 years ago) parent7ba7d50f257522
32 files changed+3038−30
content/blog/immunology-for-the-internet-age.md+521 content/docs/arvo/clay/marks/_index.md+1−1 content/docs/arvo/clay/marks/marks.md+1−1 content/docs/hoon/hoon-school/egg-timer.md+1−1 content/docs/userspace/dist/dist.md+1−1 content/docs/userspace/full-stack/1-intro.md+89 content/docs/userspace/full-stack/10-final.md+118 content/docs/userspace/full-stack/2-types.md+215 content/docs/userspace/full-stack/3-agent.md+338 content/docs/userspace/full-stack/4-json.md+357 content/docs/userspace/full-stack/5-marks.md+97 content/docs/userspace/full-stack/6-eyre.md+79 content/docs/userspace/full-stack/7-react-setup.md+214 content/docs/userspace/full-stack/8-app-logic.md+406 content/docs/userspace/full-stack/9-desk.md+226 content/docs/userspace/full-stack/_index.md+52 content/docs/userspace/gall-guide/10-scry.md+5−4 content/docs/userspace/gall-guide/12-next-steps.md+11−3 content/docs/userspace/gall-guide/7-sur-and-marks.md+9−9 content/docs/userspace/threads/gall/poke-thread.md+1−1 content/docs/userspace/threads/gall/start-thread.md+2−2 content/docs/userspace/threads/gall/stop-thread.md+1−1 content/docs/userspace/threads/gall/take-facts.md+1−1 content/docs/userspace/threads/gall/take-result.md+1−1 content/events/2022-04-21-developer-call-uqbar-and-urbits-first-blockchain.md+24 content/events/2022-04-26-urbit-foundation-meetup-kingsnight-amsterdam.md+24 content/grants/app-engram.md+117 content/grants/apprenticeship-studio-email.md+1−1 content/grants/digital-editor-MROB.md+1−1 content/grants/native-planet-2.md+89 content/grants/native-urbit.md+1−1 pages/index.js+34−1
content/blog/immunology-for-the-internet-age.mdnew file+521
@@ -0,0 +1,521 @@
1++++
2+title = "Immunology for the Internet Age"
3+date = "2022-04-07"
4+description = "A consideration of the history of the Internet motivates introspection on the nature and causes of social dysfunction in a globally shared space. Centralized solutions fail to yield satisfactory outcomes for human freedom and thriving. Decentralized autonomous organizations and their technological apparatus together represent the evolution of an immune system against a corporatized Internet."
5+[extra]
6+author = ""
7+ship = "~lagrev-nocfep"
8+image = "https://live.staticflickr.com/4322/35891780782_4179f476f9_k.jpg"
9++++
10+
11+_A version of the following was presented by ~lagrev-nocfep at the [MTAConf 2022
12+conference on “The Decentralization of Power”](https://www.mtaconf.org/)._
13+
14+**✿**
15+
16+IN THE BEGINNING, *God created the prokaryotes. And the
17+prokaryotes were without nucleus and organelle; and structural
18+simplicity was upon the face of the deep. And chemotaxis moved it upon
19+the face of the waters.*
20+
21+A single-celled organism possesses many advantages, notably simplicity
22+of form. Being largely indistinguishable, they have no differentiation
23+or specialization. Prokaryotes often form large colonies, such as the
24+cyanobacteria. Some prokaryotes discovered early on that it paid better
25+to consume resources that had already been processed into usable form by
26+another cell, rather than to process those resources oneself. Thus
27+predator and prey were born, and they have always exemplified divergent
28+competitive evolutionary strategies. The resulting arms race meant that
29+predators were continually adapting to better extract resources from
30+prey, while prey were evolving to evade, resist, fight, or poison
31+predators. Selection pressures applied at individual, species, and clade
32+levels, lead to speciation across deep time.
33+
34+More complex forms seemed better able to evade predators, and thus
35+simple symbiotic alliances between differentiated prokaryotes evolved
36+into the eukaryotes (yielding specialized features such as the nucleus
37+and mitochondria) and multicellular life forms. Simpler forms also
38+abounded, including the liminal virus. Viruses are forms of near-life so
39+simple that they simply exploit cells to reproduce. They are entirely
40+predatory or at best symbiotic with other life. Evasion of pathogenic
41+disease, or disequilibrated colonization by simpler forms of life, has
42+been a---if not the---primary driver towards increasing complexity in
43+multicellular life.[^1]
44+
45+**✿**
46+
47+The Internet began in the late 1960s, born of scientific alliance
48+between the Cold War Department of Defense and the academic research
49+community.[^2] This community had a signal advantage over many
50+subsequent communities: it was dominated by strong norms of discourse
51+which shaped the professionalism of the community. This is not to
52+discount the significance of courtly intrigue, of course, which deftly
53+exploited tensions between competing systems.[^3] [^4] [^5]
54+Throughout this era, though, the Internet generally functioned as a
55+real-name high-trust environment---if not exactly of peers, at least of
56+colleagues.
57+
58+The Internet evolved as a coordination mechanism---literally a nervous
59+system incarnate in copper and silicon---particularly through the Usenet
60+days of the 1980s. In our days of heightened online tension and mediated
61+preference cascades, it is rather quaint to go back and look at what
62+qualified as email flame wars at the time (e.g. Torvalds/Tanenbaum). One
63+hallmark of the Usenet era, however, is a gradually eroding social
64+trust: spam was invented[^6] and Internet-based scams started to spin
65+up. Pseudonymity gained in popularity as a countermeasure to both of
66+these as well as restrictions on discourse. The revelation that
67+persistent digital records could cost someone's job[^7] further fueled a
68+cyberpunk-like obsession with identity control and hacking.
69+
70+Computer viruses could propagate for the first time: it never made much
71+sense to build a computer virus until systems were connected. Although
72+John von Neumann and others early explored the possibility of
73+mathematical or code structures which could self-replicate, it was not
74+until the Creeper virus in the 1970s that self-replicating code was
75+observed in the wild. And like a bacillus in an immune system-free host,
76+code-based computer viruses began to spread from the 1980s onwards.
77+
78+The crisis arrived in the form of the World Wide Web. Founded initially
79+on a belief in the democratization of information, the Web rapidly
80+became dominated by a handful of large corporations (henceforth
81+"megacorps"). User experience became the predominant driver of adoption,
82+leading to rapid preference cascades as new platforms became available.
83+Any child of the 90s or earlier can recite the lived litany: Ebay,
84+MySpace, Friendster. Initially these served as proofs-of-concept, then
85+became the only significant player (until the next killer app came
86+along). Search engines drove discovery, then focused attention
87+asymmetrically. The "walled garden" approach of AOL, then Apple and
88+Facebook, led to concentrations of users which tipped the balance from
89+single webpage hosts presenting their own idiosyncratic views to
90+corporate-approved genericity and ultimately censorship.[^8]
91+
92+**✿**
93+
94+When was the first time you did something in the physical world because
95+of something you read or encountered online? For generations born after
96+the advent of the Web, they may not recall such a time when meatspace
97+existence wasn't driven by life online. As Internet data sources have
98+overwhelmed traditional media, publishing, and distribution services,
99+the resulting tsunami of information and informatization has swept into
100+every corner of lived human experience in technological societies. The
101+psychological (memetic) virus problem is much greater than actual code
102+viruses. The ongoing universal exposure to alien and internally
103+incoherent value systems has undergirded the well-documented crisis of
104+credibility (collapse of meaning) that all Western institutions are
105+experiencing today. The World Wide Web and its technological
106+predecessors acted as a cultural petri dish of clean agar, which grew
107+every spore and germ which fell upon it. We live in the midst of a
108+memetic epidemic because of the World Wide Web, and the dominant strain
109+is corporate, approved by a marketing department and run through a
110+search-engine optimization algorithm.
111+
112+The turn towards a low-trust then zero-trust environment was inevitable
113+if anything like an analogy with prokaryotic evolution holds true. The
114+megacorp platforms are in a way worse than a purely zero-trust
115+environment, because they don't advertise that the world is zero-trust.
116+Thereby they lure the unwary into transgressing arbitrary global norms
117+and courting the wrath of social media mobs and the concomitant loss of
118+access to their data held in "trust." To mix a metaphor, megacorps are
119+apex predators---at best, sheepdogs with a strong strain of wolfishness;
120+at worst, tar pits manned by various bad-faith actors. The pre-Web
121+cypherpunks were right: "Privacy is the power to selectively reveal
122+oneself to the world."[^9] If anything, they were too narrow; not only
123+their valued "open society" (polyvalent crypto-anarchy) but *any*
124+non-megacorp society at all requires true privacy.
125+
126+What cyberspace (and human culture writ broadly) needed was an immune
127+system---and in good Darwinian form, such an immune system has been
128+evolving before our eyes. An immune system formally consists of layers
129+with increasing specificity: first surface barriers, then the innate
130+immune system, and finally the adaptive immune system. The objective of
131+the immune system is to protect the organism: in this case, human
132+sociality and agency.
133+
134+First, what is human sociality? Human culture has been the means of
135+participating in the Great Discussion which ranges from the earliest
136+days of oral culture to the birth of literacy and libraries down to our
137+day. High culture and low culture have both articulated and preserved
138+valuable insights and forms of human flourishing. However, throughout
139+the 20^th^ century and particularly by the means of the consumer
140+revolution, our lives have been thoroughly colonized by capital and
141+commercial instruments. So-called "economies of scale" have disrupted
142+smallholders, artisans, and shopkeepers to the extent that these have
143+been virtually eradicated from the modern landscape in favor of the
144+plastic and polluting "dark Satanic mills." Although small-scale
145+"provident living" has supported gardening, canning, and home
146+production, none of this has survived across a large enough cultural
147+segment to matter macroscopically, and it has unfortunately been tarred
148+with the "prepper" brush that delegitimizes economic downsizing as
149+antisocial because anticapitalist. The siren call of cheap consumer
150+goods has ushered in a generational collapse in self-sufficiency and
151+resulted in the complete colonization of entertainment, food
152+consumption, and self-care by massive corporate brands. Not only have
153+physical goods entered a post-consumer phase, but only certain classes
154+of ideas, attitudes, political stances, regional accents, and
155+sensibilities enjoy the sanction of megacorp sociality. There is
156+essentially no native culture left, only that forged in the enterprise
157+brand management office.[^10]
158+
159+In an immune system, *surface barriers* are designed to make it hard for
160+exogenous agents to enter a cell or organism; in our analogy, the
161+primary surface barrier was the match of physical separation to
162+psychological separation. Surface barriers were rendered finally
163+obsolete by the launch of the World Wide Web and its subsequent
164+infiltration of every aspect of human culture. To the extent they exist
165+today, it is a matter of ignorance, self-discipline, or happy accident.
166+
167+The *innate immune system* responds non-specifically to pathogens. Any
168+immune system must have a way of deciding what is *me* and what is
169+*not-me*. Framed as an identity problem in a zero-trust environment,
170+users have evolved several ways to uniquely identify agents without
171+reference to legacy systems. Chief among these means is the blockchain,
172+introduced by Bitcoin for control and transaction of digital assets and
173+since much exploited beyond its design intent. Ethereum-based ERC tokens
174+represent tweaks of this identity concept, as do many other blockchain
175+platforms.
176+
177+Identity is still fluid and polysemous, but in certain key instances it
178+is indelibly demonstrable.[^11] The individual actor has the capability
179+to express itself with many public faces or none; one of these typically
180+enjoys government sanction and liability, but it is no more *real* than
181+the others, only more legible to regulation and taxation.[^12]
182+
183+Another aspect of the innate immune system is the public visibility of
184+code. Open source has developed an ethic of transparency on
185+epistemological grounds (i.e. you have a right to know what code you are
186+running) and practical grounds. (Linus's law: "given enough eyeballs,
187+all bugs are shallow."[^13])
188+
189+The *adaptive immune system* responds to specific pathogenic incursions.
190+Particular conditions arise, whether through social, regulatory,
191+military, or other requirements, which must be met with specialized
192+forms that are thereafter in evolutionary competition with the external
193+factor. For instance, one megacorp solution to what it perceives as a
194+threat, i.e. unsanctioned memes, is to attempt to mandate the ultimate
195+traceability of any image's provenance.[^14] The Internet immune system
196+thus evolves to further obfuscate legibility and enhance individual
197+control and self-determination by shifting to less-traceable
198+technologies such as the InterPlanetary File System (IPFS). The point is
199+not whether or not a salient technology such as deepfakes, 3D-printed
200+firearms, deviant scientific or political thought[^15], or even the
201+DRM-free sharing of copyrighted material can be contained[^16]; the
202+experience with export-grade cryptography[^17] has already shown that
203+regulatory attempts ultimately fail. The philosophical question is what
204+kind of world will evolve (a *fait accompli* driven by factors beyond
205+any individual or office) and the practical question is how any
206+particular group of actors can preserve their own agency, autonomy, and
207+ambit of action.
208+
209+The first decentralized autonomous organization, "The DAO" (Đ)
210+itself[^18], was created in 2016 as a way of distributing ownership and
211+stake in a joint venture capital fund. Đ was a new form of organization
212+in that it was essentially a mathematical entity, a set of smart
213+contracts on the blockchain to which anyone had access. Đ lived up at
214+least to "decentralized" and "autonomous." No money was held centrally,
215+with token-based authentication yielding the right to vote on projects.
216+Having set the criteria for membership and investment, Đ then operated
217+in truly democratic fashion until an exploit of imperfectly written
218+smart contract code led to theft, crisis, and the reorganization of the
219+Ethereum blockchain itself.
220+
221+While the heirs of Đ serve a role in the current Internet immune system,
222+the original was formed in response to exogenous regulatory concerns,
223+notably ongoing governmental insistence on restricting investment to the
224+already-wealthy. The U.S. Securities and Exchange Commission and other
225+agencies have in place arbitrary investor requirements which would make
226+little sense even if pegged to something like inflation or index rather
227+than the rapidly-devaluing petrodollar. Nominally put in place to
228+protect the naïve from getting scammed, these restrictions have had the
229+pernicious effect of pulling up the ladder behind the *actuel riche*.
230+
231+DAOs have rapidly evolved as a way of distributing ownership and stake
232+in a purely democratic way. (As an aside, the word "democracy" has been
233+much abused, but here I use it in a sense much like the Greek original:
234+the holders of an immutable and indelible token cast their secured votes
235+to govern their society.) DAOs represent a resurgence of the sovereign
236+city-state in new guise. Will this be enough to evade "imperial"
237+aggregates which dominate today's Web?
238+
239+Decentralized autonomous organizations, zero-knowledge cryptographic
240+proofs, cryptocurrencies, proof-of-identity systems, and their
241+as-yet-unborn kin represent the evolution of an immune system on top of
242+the Darwinian churn of the legacy Internet. The stakes are necessarily
243+high: possession is *de facto* ownership, the other end of the stick
244+from "not your keys, not your coins" (the old cryptocurrency chestnut).
245+The focus is currently on ownership of money, non-fungible tokens, and
246+similar assets, but broadly speaking such systems allow the
247+authenticated ownership of all data and assets one holds, from chat
248+messages to market shares to physical real estate. There may be good
249+reasons to retain the legacy system for some classes of physical or
250+legal artifacts, but at minimum these will be in competition with purely
251+autonomous constructs. (As an aside, I am of the philosophical opinion
252+that legal recognition should not be sought generally for new forms of
253+organization, as any legibility to legacy systems opens new attack
254+surfaces in an immunological sense.) "Code is law."[^19]
255+
256+A decentralized autonomous organization seems to be at least the first
257+two: having set the criteria for membership, the organization can act as
258+the (distributed[^20], possibly unknown in the zero-knowledge proof
259+sense) stakeholders direct. This form is still somewhat "yeasty" in that
260+like cyanobacteria it acts as an undifferentiated mass, and we should
261+expect further innovations in DAO-like entities.[^21] The most salient
262+epithet of a DAO is "decentralized"; one can imagine hierarchical,
263+concentric, or role-specialized DAOs as well as other taxonomically
264+related forms, just as one can imagine progressively less organized
265+instances in a chaos-happy Internet.
266+
267+Like the first mutant with a new gene, Đ itself failed. Many lessons can
268+be drawn from that failure, including the criticality of getting smart
269+contract code absolutely correct ("code is law"), and how to administer
270+a blockchain in the presence of massive fraud. And indeed, what has
271+happened to the blockchain world of decentralized finance? Millisecond
272+differentials in responsiveness lead to huge swings in profitability of
273+transactions. Frontrunner bots watch algorithmically for transactions
274+they can complete faster than the original source, thereby making money
275+on asymmetric knowledge.[^22] Zero-day exploits of code abound,
276+heralding a need for more thorough vetting and more expressive and
277+secure smart contract languages.
278+
279+Blockchains (and thus DAOs via Ethereum) operate as proof-of-work,
280+proof-of-stake, etc. This provides node operators with a way of
281+verifying consensus reality and thereby cementing a record of digital
282+transactions. Future innovations hold forth the possibility of secure
283+proof-of-identity blockchains (such as the Uqbar Layer 1 smart contract
284+blockchain on Urbit[^23]), wherein each node which can provide concrete
285+cryptographic evidence of its identity gets a vote on the consensus.
286+
287+We must consider modern systems to necessarily interact at the pinnacle
288+intensity of competence and competition: any differential advantage that
289+can be exploited to outcompete an adversary not only can be exploited
290+but *should* be. Illegibility, redundancy, and apposite complexity are
291+allies in the immune fight to maintain identity and ownership in the
292+modern Internet.[^24] Any system which does not have an operational
293+DAO-like layer is immuno­compromised. All tools for the individual and
294+collective need to be architected around a zero-trust adversarial world.
295+They also need to guard the embers of community and enable new growth.
296+
297+In a nutshell, the early Internet was a cocktail party. By the age of
298+Usenet and into the World Wide Web era, the Internet evolved into a
299+player-versus-player defect--defect equilibrium in which brigading,
300+spamming, and anonymity are rational adaptive behaviors. Indeed,
301+anonymity is barely a protection given the lengths to which governments
302+and megacorps go to prevent truly untraceable usage. This approach is
303+adaptive for agencies which have given up on any other model of reality,
304+fundamentally lacking a vision of humans as other than atomized
305+individual krill for the megacorp apex predators: consumers for the
306+consumers. The legacy megacorp Internet has become a crab bucket, always
307+drawing everything back into itself in a dysfunctional and frankly
308+satanic way.
309+
310+Communities built on decentralized Web3 platforms such as Mastodon,
311+Fediverse, and Urbit[^25] have stepped back from the purely PvP world of
312+the globally-namespaced social media Internet. These today have more of
313+a block party feel, wherein one can wander from house to house or room
314+to room and encounter some of the same faces in the same or new guise.
315+Conversations can drop and resume anew in very different locales with
316+largely the same composition. Marrying the underlying cryptographic
317+identity requirements of DAOs to communication tools and databases
318+yields a strong stable solution to the problem of retaining a foothold
319+of personal identity from which to coordinate one's multifaceted digital
320+life. IPFS, Sovrin Foundation, Tim Berners-Lee's Solid, and Urbit all
321+offer their take on this requirement. Even a Signal chat group running
322+its own server is a kind of token-free DAO.
323+
324+Since the Renaissance, essentially all theology has been humanist in
325+nature, concerning itself with how man should live to flourish in this
326+world and inherit in the next. Even secularized descendants enjoin an
327+ethos of ritual behaviors and practices. The social and cultural "stack"
328+(in software parlance) upon which we build provides for a variety, broad
329+or narrow, of human cultural forms. Lately, the variety of forms has
330+been diminishing both by cutting off older forms from the modern world
331+and by cultivating only certain state-legible apex predators[^26] and
332+their preferred ovine repast.
333+
334+Contra this monoculture, the *telos* of social computing is to recreate
335+the village green, not the penitentiary. Revisiting a thesis from above:
336+"All tools for the individual and collective need to be architected
337+around *using cryptographic proofs to create stable high-trust havens in
338+an otherwise* zero-trust adversarial world." The village is a stable,
339+long-term tribe well below Dunbar's limit which shares values and assets
340+in common, from which a member occasionally ventures out into the
341+broader social web but always has a home. Every village will grow its
342+own culture, perhaps even its own *cultus*, enabled by decentralized
343+technologies to protect their assets and precious peculiarities. The
344+World Wide Web has been an extinction-level event for many forms of
345+life, livelihood, and culture. For others, it has offered strange forms
346+of life support and even thriving. With a new immune system evolved and
347+now operational, human endeavor stands on the cusp of a Cambrian
348+explosion, unconstrainable by a rent-seeking boardroom-and-bureaucracy
349+world that never comprehended the light shining in its darkness.
350+
351+[^1]: Kaufman, Jim. (2010) ["Evolution and
352+ immunity"](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2913256/).
353+ *Immunology 130*, iss. 4, pp. 459--462.
354+
355+[^2]: President Eisenhower's original coinage was likely "the
356+ military-industrial-academic complex."
357+
358+[^3]: E.g., Richard Bellman of the RAND Corporation on the origin of
359+ "dynamic programming," a mathematical discipline: in dealing with a
360+ recalcitrant secretary of defense who hated the word "research,"
361+ Bellman instead chose the word "dynamic" arbitrarily because "it's
362+ impossible to use the word 'dynamic' in a pejorative sense. ...It
363+ was something not even a Congressman could object to. So I used it
364+ as an umbrella for my activities." (Bellman, Richard. (1984) [*Eye
365+ of the
366+ Hurricane*](https://books.google.ch/books?id=4FwGCwAAQBAJ&lpg=PA159&dq=#v=onepage&q=).)
367+
368+[^4]: The nuclear triad doctrine, joining the Air Force's Strategic Air
369+ Command bombers, the Navy's nuclear submarines, and the Army's ICBMs
370+ and other assorted nuclear warheads, was based less on strategic
371+ considerations than on appeasing constituent departments of the
372+ military. Strategists developed a formal justification decades later
373+ in a magnificent demonstration of motivated reasoning. (Woolf, Amy.
374+ (2016) "U.S. Strategic Nuclear Forces: Background, Developments, and
375+ Issues". [CRS
376+ 7-5700](https://digital.library.unt.edu/ark:/67531/metadc847627/m2/1/high_res_d/RL33640_2016Mar10.pdf).)
377+
378+[^5]: The Air Force early commissioned the RAND Corporation on contract
379+ with Douglas Aircraft Company as the first "think tank"; RAND soon
380+ upended the traditional role of passive reception of studies by
381+ making its own recommendations and showing their analytical chops.
382+ Other service branches followed suit, such as the U.S. Navy's Center
383+ for Naval Analyses and the Battelle Memorial Institute for various
384+ federal contracting agencies; compare also the role of the national
385+ laboratory system, e.g. Los Alamos, Sandia, Livermore, etc. for the
386+ Department of Energy and Department of Defense. (Rich, Michael.
387+ (2003) "RAND: How Think Tanks Interact with the Military".
388+ [RP-1050](https://www.rand.org/pubs/reprints/RP1050.html).)
389+
390+[^6]: "On January 18, 1994, the first large-scale deliberate USENET spam
391+ occurred. A message with the subject "Global Alert for All: Jesus is
392+ Coming Soon" was cross-posted to every available newsgroup."
393+ ([Wikipedia, "History of email
394+ spam"](https://en.wikipedia.org/wiki/History_of_email_spam))
395+
396+[^7]: The first instance which seems to have brough this to general
397+ public awareness was the submission of Judge Robert Bork's video
398+ rental history as evidence in his Supreme Court hearings before the
399+ U.S. Senate. The resulting Video Privacy Protection Act was
400+ delightfully naïve. Besides relying on "video rental" narrowly
401+ construed, users have effectively no leverage against
402+ unilaterally-imposed terms of service and end-user license
403+ agreements. (See, e.g., [Electronic Privacy Information Center,
404+ "Video Privacy Protection
405+ Act"](https://archive.epic.org/privacy/vppa/).)
406+
407+[^8]: E.g. Tumblr and OnlyFans and adult content; Twitter and right-wing
408+ content; Youtube and Facebook and unsanctioned medical content, etc.
409+ Note that this is regardless of the viability of a given view or the
410+ necessity of vigorous debate within a field, but focuses on the
411+ palatibility of politics to the megacorp entity providing the
412+ platform, even if the content drives platform usage.
413+
414+[^9]: Hughes, Eric. (1993) [*A Cypherpunk's
415+ Manifesto*](https://nakamotoinstitute.org/cypherpunk-manifesto/).
416+
417+[^10]: The mass shift to remote work from March 2020 onwards due to the
418+ COVID-19 pandemic has accelerated the trend of corporate
419+ colonization of private spaces. At first, this was occasionally
420+ humorous, as when a child ran into the background of a Zoom call, or
421+ underwear or private materials were inadvertently left in view of
422+ the remote audience. In the end, it demanded that individuals
423+ reconfigure their homes to accommodate mass corporate preferences,
424+ thus another invasion of private space by the megacorp panopticon.\
425+ I note as well an invasive doctrine demanding "truth in
426+ communication" under names like ["radical
427+ transparency"](https://www.bridgewater.com/principles-and-culture)
428+ and ["radical
429+ candor"](https://third-bit.com/2018/11/24/afraid-of-change/) put
430+ forward by partisans of power over against vulnerable employees,
431+ clients, and customers, who may be disproportionately affected by a
432+ megacorp demand for one-sided transparency.
433+
434+[^11]: Several have pointed to the failure of Craig S. Wright to
435+ demonstrate cryptographically that he is Satoshi Nakamoto, the
436+ pseudonymous inventor of Bitcoin. (C.f. Khaosan, Venzen. (2016)
437+ ["Craig Wright is Not Satoshi Nakamoto---The Technical
438+ Proof"](https://www.ccn.com/technical-proof-craig-wright-not-satoshi-nakamoto/).)
439+
440+[^12]: Davidson, Dale, and Rees-Mogg, William. (1999) *The Sovereign
441+ Individual: Mastering the Transition to the Information Age*.
442+
443+[^13]: Raymond, Eric. (1999) [*The Cathedral and the
444+ Bazaar*](http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s04.html).
445+
446+[^14]: The [Coalition for Content Provenance and
447+ Authenticity](https://c2pa.org/) "addresses the prevalence of
448+ misleading information online through the development of technical
449+ standards for certifying the source and history (or provenance) of
450+ media content."
451+
452+[^15]: Cf. Walker, Shawn, et al. (2019) ["The disinformation landscape
453+ and the lockdown of social
454+ platforms."](https://www.tandfonline.com/doi/full/10.1080/1369118X.2019.1648536)
455+ *Information, Communication & Society 22*, pp. 1531--43. Walker et
456+ al. lament that "highly ephemeral mis/disinformation campaigns" are
457+ not susceptible of study due to closed platform APIs. "Social media
458+ platforms \[attempt\] to rekindle trust by appearing to reinforce
459+ individual privacy within a newly secured user community, a set of
460+ measures that also locked academic and non-profit researchers out
461+ from studying social platforms while preserving corporate and
462+ business access to social media users' data." They fail to address
463+ the tension between community self-determination and academic
464+ exploitation.
465+
466+[^16]: "If in fact you can't crack \[the encryption\] at all, government
467+ can't get in, then everybody is walking around with a Swiss bank
468+ account in their pocket -- right? So there has to be some concession
469+ to the need to be able to get into that information somehow."
470+ ([Barack Obama in March 2016 at
471+ SXSW](https://www.businessinsider.com/obama-comments-on-encryption-at-sxsw-2016-3))
472+
473+[^17]: Heninger, Nadia. (2016) ["The Legacy of Export-Grade Cryptography
474+ in the 21^st^
475+ Century"](https://publish.illinois.edu/science-of-security-lablet/files/2016/08/10062016-Heninger-Slides.pdf).
476+
477+[^18]: Chohan, Usman. (2017) ["The Decentralized Autonomous Organization
478+ and Governance
479+ Issues"](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3082055).
480+ Discussion Paper Series: *Notes on the 21^st^ Century*.
481+
482+[^19]: Lessig, Lawrence. (1999) *Code and Other Laws of Cyberspace*.
483+
484+[^20]: Hobart, Byrne \~lableg-tadrex. (Nov 17, 2021) ["The Promise and
485+ Paradox of
486+ Decentralization"](https://urbit.org/blog/2021-11-18-the-promise-and-paradox-of-decentralization).
487+ *The Diff*.
488+
489+[^21]: See, e.g., the Governor Alpha smart contract protocols. Solari,
490+ R. (2021) ["The case for Governor
491+ Alpha".](https://medium.com/tally-blog/the-case-for-governor-alpha-15c1362d49eb)
492+
493+[^22]: Robinson, D. (2020) ["Ethereum is a dark
494+ forest"](https://www.paradigm.xyz/2020/08/ethereum-is-a-dark-forest).
495+ Paradigm blog.
496+
497+[^23]: [Uqbar Network](https://uqbar.network/); ["Introducing Uqbar
498+ Network"](https://uqbarnetwork.substack.com/p/introducing-uqbar-network);
499+ \~hocwyn-tipwex/uqbar-event-horizon (Urbit group)
500+
501+[^24]: This is not to deny the benefits of system-legibility, in
502+ particular legal protections. As common law develops around DAOs, I
503+ hope for similar affordances where they must interact with the
504+ legacy legal system.
505+
506+[^25]: Originally this list included Discord, as there has been
507+ substantial growth in stable goal-oriented communities thereon.
508+ However, more recently the company [adopted a user
509+ policy](https://discord.com/blog/addressing-harmful-off-platform-behavior)
510+ which subjects use of their product to judgment calls about
511+ "off-platform behavior"---a clear violation of a cypherpunk ethos
512+ which Discord likely never espoused anyway.
513+
514+[^26]: E.g., Auerbach, David. (Mar 3, 2022) ["The metaverse will steal
515+ your
516+ identity"](https://unherd.com/2022/03/the-metaverse-will-steal-your-identity/).
517+ *UnHerd*. Auerbach correctly recognizes and decries the looming
518+ "monetisation of identity" which will allow any platform controller
519+ to incentivize prosocial or antisocial behaviors at will.
520+
521+_The header image is [Volvox sp. microscopic algae](https://www.flickr.com/photos/picksfromoutthere/35891780782/in/photolist-WFCKYA-XPdGLq-2k9Cew6-ViCgrB-23QUBQR-21rF362-CCSE2n-23AwZ7m-CbKedu-gznqa-2dHUv3k-EE8eoq-gznq9-i5zqqk-WYSwcf-XWVgQk-WabAAG-gLjRj-gDcLp-4VvECm-259cDCS-i5zqRk-25cV246-i5yNvM-gupiMJ-PXLbhr-LUACZ7-ihvxSH-ihvvVg-ihwjbF-ihvxDX-i5yQDd-ihzQwK-fWwpxb-rarKrm-8LWTQf-qVar5j-ihzAPA-JNrgyK-ihzBrC-ca6Ex9-2kCTSrj-ihz67k-qfJsYd-248ePDL-ihztbm-ihzuyS-LQYZQq-2kCTRxL-ihvWPA) by Picturepest on Flickr._
content/docs/arvo/clay/marks/_index.md+1−1
@@ -12,7 +12,7 @@ Overview of `mark`s.
1212
1313[Writing Marks](/docs/arvo/clay/marks/writing-marks)
1414
15−A practical walk-through of writing a `mark` file.
15+A practical walkthrough of writing a `mark` file.
1616
1717[Using Marks](/docs/arvo/clay/marks/using-marks)
1818
content/docs/arvo/clay/marks/marks.md+1−1
@@ -52,7 +52,7 @@ The second case of interest is the `+json` arm for converting to and from a `%js
5252
5353## Sections
5454
55−[Writing Marks](/docs/arvo/clay/marks/writing-marks) - A practical walk-through of writing a `mark` file.
55+[Writing Marks](/docs/arvo/clay/marks/writing-marks) - A practical walkthrough of writing a `mark` file.
5656
5757[Using Marks](/docs/arvo/clay/marks/using-marks) - Details of using `mark` conversion gates and `mark` cores in your own code.
5858
content/docs/hoon/hoon-school/egg-timer.md+1−1
@@ -161,4 +161,4 @@ In this case, we will not produce any card or change application state. Therefor
161161--
162162```
163163
164−Finally, we define the remaining 8 unused but required Gall Agent armsas instances of the default arms provided in `/lib/default-agent.hoon`.
164+Finally, we define the remaining 8 unused but required Gall Agent arms as instances of the default arms provided in `/lib/default-agent.hoon`.
content/docs/userspace/dist/dist.md+1−1
@@ -6,7 +6,7 @@ template = "doc.html"
66
77Urbit allows peer-to-peer distribution and installation of applications. A user can click on a link to an app hosted by another ship to install that app. The homescreen interface lets users manage their installed apps and launch their interfaces in new tabs.
88
9−This document describes the architecture of Urbit's app distribution system. For a walk-through of creating and distributing an app, see the [`Guide`](/docs/userspace/dist/guide) document.
9+This document describes the architecture of Urbit's app distribution system. For a walkthrough of creating and distributing an app, see the [`Guide`](/docs/userspace/dist/guide) document.
1010
1111## Architecture
1212
content/docs/userspace/full-stack/1-intro.mdnew file+89
@@ -0,0 +1,89 @@
1++++
2+title = "1. Introduction"
3+weight = 1
4+template = "doc.html"
5++++
6+
7+This series walks through the writing of a full Gall agent, and then the process
8+of integrating it with a React front-end. This series follows on from the
9+previous [Gall Guide](/docs/userspace/gall-guide/intro). If you haven't
10+completed that, or otherwise aren't familiar with the basics of writing Gall
11+agents, it's strongly recommended to work through that guide first.
12+
13+The app we'll be looking at is a simple journal with an agent called `%journal`.
14+In the browser, users will be able to add plain text journal entries organized
15+by date. Entries may be scrolled through in ascending date order, with more
16+entries loaded each time the bottom of the list is reached. Old entries will be
17+able to be edited and deleted, and users will be able to search through entries
18+by specifying a date range.
19+
20+The `Journal` app we'll be looking at can be installed on a live ship from
21+`~pocwet/journal`, and its source code is available [here](https://github.com/urbit/docs-examples/tree/main/journal-app).
22+
23+![journal ui screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/entries.png)
24+
25+This walkthrough does not contain exercises, nor does it completely cover every
26+aspect of building the app in full depth. Rather, its purpose is to demonstrate
27+the process of creating a full-stack Urbit app, showing how everything fits
28+together, and how concepts you've previously learned are applied in practice.
29+
30+The primary focus of the walkthrough is to show how a Javascript front-end is
31+integrated with a Gall agent and distributed as a complete app. Consequently,
32+the example app is fairly simple and runs on a local ship only, rather than one
33+with more complex inter-ship networking.
34+
35+Each section of this walkthrough will list additional resources and learning
36+material at the bottom of the page, which will cover the concepts discussed in a
37+more comprehensive manner.
38+
39+Here is the basic structure of the app we'll be building:
40+
41+![journal app
42+diagram](https://media.urbit.org/docs/userspace/full-stack-guide/journal-app-diagram.svg)
43+
44+## Sections
45+
46+#### [1. Introduction](/docs/userspace/gall-2/1-intro)
47+
48+An overview of the guide and table of contents.
49+
50+#### [2. Types](/docs/userspace/gall-2/2-types)
51+
52+Creating the `/sur` structure file for our `%journal` agent.
53+
54+#### [3. Agent](/docs/userspace/gall-2/3-agent)
55+
56+Creating the `%journal` agent itself.
57+
58+#### [4. JSON](/docs/userspace/gall-2/5-json)
59+
60+Writing a library to convert between our agent's marks and JSON. This lets our
61+React front-end poke our agent, and our agent send updates back to it.
62+
63+#### [5. Marks](/docs/userspace/gall-2/4-marks)
64+
65+Creating the mark files for the pokes our agent takes and updates it sends out.
66+
67+#### [6. Eyre](/docs/userspace/gall-2/6-eyre)
68+
69+A brief overview of how the webserver vane Eyre works.
70+
71+#### [7. React App Setup](/docs/userspace/gall-2/7-react-setup)
72+
73+Creating a new React app, installing the required packages, and setting up some
74+basic things for our front-end.
75+
76+#### [8. React App Logic](/docs/userspace/gall-2/8-http-api)
77+
78+Analyzing the core logic of our React app, with particular focus on using
79+methods of the `Urbit` class from `@urbit/http-api` to communicate with our
80+agent.
81+
82+#### [9. Desk and Glob](/docs/userspace/gall-2/9-web-scries)
83+
84+Building and "globbing" our front-end, and putting together a desk for
85+distribution.
86+
87+#### [10. Summary](/docs/userspace/gall-2/10-final)
88+
89+Some final comments and additional resources.
content/docs/userspace/full-stack/10-final.mdnew file+118
@@ -0,0 +1,118 @@
1++++
2+title = "10. Summary"
3+weight = 10
4+template = "doc.html"
5++++
6+
7+That's it! We've built our agent and React front-end, put together a desk and
8+published it. We hope this walkthrough has helped you see how all the pieces
9+for together for building and distributing an app in Urbit.
10+
11+The reference material for each section of this walkthrough is listed
12+[below](#reference-material), the source code for our app is available
13+[here](https://github.com/urbit/docs-examples/tree/main/journal-app), and it can
14+be installed from `~pocwet/journal`.
15+
16+In this guide we've built a separate React app for the front-end, but Hoon also
17+has a native domain-specific language for composing HTML structures called Sail.
18+Sail allows you to compose a front-end inside a Gall agent and serve it
19+directly. See the [Sail guide](/docs/hoon/guides/sail) for details.
20+
21+Along with `@urbit/http-api`, there's also the `@urbit/api` NPM package, which
22+contains a large number of helpful functions for dealing with Hoon data types
23+and interacting with a number of agents - particularly those used by the Groups
24+app. Its source code is [available
25+here](https://github.com/urbit/urbit/tree/master/pkg/npm/api).
26+
27+## Reference material
28+
29+Here is the reference material for each section of this walkthrough.
30+
31+#### Types
32+
33+- [Gall Guide /sur section](/docs/userspace/gall-guide/7-sur-and-marks#sur) -
34+ This section of the Gall Guide covers writing a `/sur` structure library for
35+ an agent.
36+
37+- [Ordered map functions in
38+ `zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284-L5688) -
39+ This section of `zuse.hoon` contains all the functions for working with
40+ `mop`s, and is well commented.
41+
42+#### Agent
43+
44+- [The Gall Guide](/docs/userspace/gall-guide/intro) - The Gall Guide covers all
45+ aspects of writing Gall agents in detail.
46+
47+- [Ordered map functions in
48+ `zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284-L5688) -
49+ This section of `zuse.hoon` contains all the functions for working with
50+ `mop`s, and is well commented.
51+
52+- [`/lib/agentio.hoon`](https://github.com/urbit/urbit/blob/master/pkg/base-dev/lib/agentio.hoon) -
53+ The `agentio` library in the `%base` desk contains a large number of useful
54+ functions which making writing Gall agents easier.
55+
56+#### JSON
57+
58+- [The JSON Guide](/docs/hoon/guides/json-guide) - The stand-alone JSON guide
59+ covers JSON encoding/decoding in great detail.
60+- [The Zuse Reference](/docs/hoon/reference/zuse/table-of-contents) - The
61+ `zuse.hoon` reference documents all JSON-related functions in detail.
62+
63+- [`++enjs:format` reference](/docs/hoon/reference/zuse/2d_1-5#enjsformat) -
64+ This section of the `zuse.hoon` documentation covers all JSON encoding
65+ functions.
66+
67+- [`++dejs:format` reference](/docs/hoon/reference/zuse/2d_6) - This section of
68+ the `zuse.hoon` documentation covers all JSON _decoding_ functions.
69+
70+- [Eyre Overview](/docs/arvo/eyre/eyre) - This section of the Eyre vane
71+ documentation goes over the basic features of the Eyre vane.
72+
73+#### Marks
74+
75+- [The Marks section of the Clay documentation](/docs/arvo/clay/marks/marks) -
76+ This section of the Clay vane documentation covers mark files comprehensively.
77+- [The mark file section of the Gall
78+ Guide](/docs/userspace/gall-guide/7-sur-and-marks#mark-files) - This part of
79+ the Gall Guide goes through the basics of mark files.
80+
81+- [The JSON Guide](/docs/hoon/guides/json-guide) - This also covers writing mark
82+ files to convert to/from JSON.
83+
84+#### Eyre
85+
86+- [The Eyre vane documentation](/docs/arvo/eyre/eyre) - This section of the vane
87+ docs covers all aspects of Eyre.
88+- [Eyre External API Reference](/docs/arvo/eyre/external-api-ref) - This section
89+ of the Eyre documentation contains reference material for Eyre's external API.
90+
91+- [The Eyre Guide](/docs/arvo/eyre/guide) - This section of the Eyre
92+ documentation walks through using Eyre's external API at a low level (using
93+ `curl`).
94+
95+#### React App Setup and Logic
96+
97+- [HTTP API Guide](/docs/userspace/http-api-guide) - Reference documentation for
98+ `@urbit/http-api`.
99+
100+- [React app source
101+ code](https://github.com/urbit/docs-examples/tree/main/journal-app/ui) - The
102+ source code for the Journal app UI.
103+
104+- [`@urbit/http-api` source
105+ code](https://github.com/urbit/urbit/tree/master/pkg/npm/http-api) - The
106+ source code for the `@urbit/http-api` NPM package.
107+
108+#### Desk and Glob
109+
110+- [App publishing/distribution docs](/docs/userspace/dist/dist) -
111+ Documentation covering third party desk composition, publishing and
112+ distribution.
113+
114+- [Glob documentation](/docs/userspace/dist/glob) - Comprehensive documentation
115+ of handling front-end files.
116+
117+- [Desk publishing guide](/docs/userspace/dist/guide) - A step-by-step guide to
118+ creating and publishing a desk.
content/docs/userspace/full-stack/2-types.mdnew file+215
@@ -0,0 +1,215 @@
1++++
2+title = "2. Types"
3+weight = 2
4+template = "doc.html"
5++++
6+
7+The best place to start when building a new agent is its type definitions in its
8+`/sur` structure file. The main things to think through are:
9+
10+1. What basic types of data does my agent deal with?
11+2. What actions/commands does my agent need to handle?
12+3. What updates/events will my agent need to send out to subscribers?
13+4. What does my agent need to store in its state?
14+
15+Let's look at each of these questions in turn, and put together our agent's
16+`/sur` file, which we'll call `/sur/journal.hoon`.
17+
18+### 1. Basic types
19+
20+Our journal entries will just be plain text, so a simple `@t` will work fine to
21+store their contents. Entries will be organized by date, so we'll also need to
22+decide a format for that.
23+
24+One option would be to use an `@da`, and then use the date functions included
25+in the `@urbit/api` NPM package on the front-end to convert them to ordinary
26+Javascript `Date` objects. In this case, to keep it simple, we'll just use the
27+number of milliseconds since the Unix Epoch as an `atom`, since it's natively
28+supported by the Javascript `Date` object.
29+
30+The structure for a journal entry can therefore be:
31+
32+```hoon
33++$ id @
34++$ txt @t
35++$ entry [=id =txt]
36+```
37+
38+### 2. Actions
39+
40+Now that we know what a journal entry looks like, we can think about what kind
41+of actions/commands our agent will handle in its `++on-poke` arm. For our
42+journal app, there are three basic things we might do:
43+
44+1. Add a new journal entry.
45+2. Edit an existing journal entry.
46+3. Delete an existing journal entry.
47+
48+We can create a tagged union structure for these actions, like so:
49+
50+```hoon
51++$ action
52+ $% [%add =id =txt]
53+ [%edit =id =txt]
54+ [%del =id]
55+ ==
56+```
57+
58+### 3. Updates
59+
60+Updates are a little more complicated than our actions. Firstly, our front-end
61+needs to be able to retrieve an initial list of journal entries to display. Once
62+it has that, it also needs to be notified of any changes. For example, if a new
63+entry is added, it needs to know so it can add it to the list it's displaying.
64+If an entry gets deleted, it needs to remove it from the list. Etc.
65+
66+The simplest approach to the initial entries is just a `(list entry)`. Then, for
67+the subsequent updates, we could send out the `$action`. Since an `$action` is a
68+tagged union, it's simpler to have all updates be a tagged union, so when we get
69+to doing mark conversions we can just switch on the head tag. Therefore, we can
70+define an `$update` structure like so:
71+
72+```hoon
73++$ update
74+ $% action
75+ [%jrnl list=(list entry)]
76+ ==
77+```
78+
79+There's one drawback to this structure. Suppose either an agent on a remote ship
80+or an instance of the front-end client is subscribed for updates, and the
81+network connection is disrupted. In the remote ship case, Gall will only allow
82+so many undelivered messages to accumulate in Ames before it automatically kicks
83+the unresponsive subscriber. In the front-end case, the subscription will also
84+be ended if enough unacknowledged messages accumulate, and additionally the
85+client may sometimes need to establish an entirely new connection with the ship,
86+discarding existing subscriptions. When this happens, the remote ship or web
87+client has no way to know how many (if any) updates they've missed.
88+
89+The only way to resynchronize their state with ours is to discard their existing
90+state, refetch the entire initial state once again, and then resubscribe for
91+updates. This might be fine if the state of our agent is small, but it becomes a
92+problem if it's very large. For example, if our agent holds tens of thousands of
93+chat messages, having to resend them all every time anyone has connectivity
94+issues is quite inefficient.
95+
96+One solution to this is to keep an _update log_. Each update can be tagged with
97+the time it occurred, and stored in our agent's state, separately to the
98+entries. If an agent or web client needs to resynchronize with our agent, it can
99+just request all updates since the last one it received. This approach is used
100+by the `%graph-store` agent, for example. Our agent is local-only and doesn't
101+have a huge state so it might not be strictly necessary, but we'll use it to
102+demonstrate the approach.
103+
104+We can define a logged update like so, where the `@` is the update timestamp in
105+milliseconds since the Unix Epoch:
106+
107+```hoon
108++$ logged (pair @ action)
109++$ update
110+ %+ pair @
111+ $% action
112+ [%jrnl list=(list entry)]
113+ [%logs list=(list logged)]
114+ ==
115+```
116+
117+### 4. State
118+
119+We need to store two things in our state: the journal entries and the update
120+log. We could just use a couple of `map`s like so:
121+
122+```hoon
123++$ journal (map id txt)
124++$ log (map @ action)
125+```
126+
127+Ordinary `map`s are fine if we just want to access one value at a time, but we
128+want to be able to:
129+
130+1. Retrieve only some of the journal entries at a time, so we can have "lazy
131+ loading" in the front-end, loading more entries each time the user scrolls to
132+ the bottom of the list.
133+2. Retrieve only logged updates newer than a certain time, in the case where the
134+ subscription is interrupted due to connectivity issues.
135+3. Retrieve journal entries between two dates.
136+
137+Maps are ordered by the hash of their key, so if we convert them to a list
138+they'll come out in seemingly random order. That means we'd have to convert the
139+map to a list, sort the list, and then iterate over it again to pull out the
140+items we want. We could alternatively store things in a list directly, but
141+retrieving or modifying arbitrary items would be less efficient.
142+
143+To solve this, rather than using a `map` or a `list`, we can use an _ordered
144+map_. The mold builder for an ordered map is a `mop`, and it's included in the
145+[`zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284)
146+utility library rather than the standard library.
147+
148+A `mop` is defined similarly to a `map`, but it takes an extra argument in the
149+following manner:
150+
151+```hoon
152+((mop key-mold val-mold) comparator-gate)
153+```
154+
155+The gate is a binary gate which takes two keys and produces a `?`. The
156+comparator is used to decide how to order the items in the mop. In our case,
157+we'll create a `$journal` and `$log` `mop` like so:
158+
159+```hoon
160++$ journal ((mop id txt) gth)
161++$ log ((mop @ action) lth)
162+```
163+
164+The entries in `$journal` are arranged in ascending time order using `++gth`, so
165+the right-most item is the newest. The `$log` `mop` contains the update log, and
166+is arranged in descending time order, so the right-most item is the oldest.
167+
168+We'll look at how to use ordered maps later when we get to writing the agent
169+itself.
170+
171+## Conclusion
172+
173+When we put each of these parts together, we have our complete
174+`/sur/journal.hoon` file:
175+
176+```hoon
177+|%
178+:: Basic types of the data we're dealing with
179+::
180++$ id @
181++$ txt @t
182++$ entry [=id =txt]
183+:: Poke actions
184+::
185++$ action
186+ $% [%add =id =txt]
187+ [%edit =id =txt]
188+ [%del =id]
189+ ==
190+:: Types for updates to subscribers or returned via scries
191+::
192++$ logged (pair @ action)
193++$ update
194+ %+ pair @
195+ $% action
196+ [%jrnl list=(list entry)]
197+ [%logs list=(list logged)]
198+ ==
199+:: Types for our agent's state
200+::
201++$ journal ((mop id txt) gth)
202++$ log ((mop @ action) lth)
203+--
204+```
205+
206+## Resources
207+
208+- [Gall Guide /sur section](/docs/userspace/gall-guide/7-sur-and-marks#sur) -
209+ This section of the Gall Guide covers writing a `/sur` structure library for
210+ an agent.
211+
212+- [Ordered map functions in
213+ `zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284-L5688) -
214+ This section of `zuse.hoon` contains all the functions for working with
215+ `mop`s, and is well commented.
content/docs/userspace/full-stack/3-agent.mdnew file+338
@@ -0,0 +1,338 @@
1++++
2+title = "3. Agent"
3+weight = 3
4+template = "doc.html"
5++++
6+
7+Now that we have our agent's types defined and have thought through its
8+behavior, we can write the `%journal` agent itself.
9+
10+## Imports
11+
12+```hoon
13+/- *journal
14+/+ default-agent, dbug, agentio
15+```
16+
17+We first import the `/sur/journal.hoon` file we previously created and expose
18+its structures. We import the standard `default-agent` and `dbug`, and also an additional library called `agentio`.
19+
20+Agentio contains a number of convenience functions to make common agent tasks
21+simpler. For example, rather than writing out the full `$card`s when sending
22+`%fact`s to subscribers, we can call `++fact` in `agentio` with the `cage` and
23+`path`s and it will compose them for us. There are many more functions in
24+`agentio` than we'll use here - you can have a look through the library in
25+[`/base/lib/agentio.hoon`](https://github.com/urbit/urbit/blob/master/pkg/base-dev/lib/agentio.hoon)
26+to see what else it can do.
27+
28+## State and type core
29+
30+```hoon
31+|%
32++$ versioned-state
33+ $% state-0
34+ ==
35++$ state-0 [%0 =journal =log]
36++$ card card:agent:gall
37+++ j-orm ((on id txt) gth)
38+++ log-orm ((on @ action) lth)
39+++ unique-time
40+ |= [=time =log]
41+ ^- @
42+ =/ unix-ms=@
43+ (unm:chrono:userlib time)
44+ |-
45+ ?. (has:log-orm log unix-ms)
46+ unix-ms
47+ $(time (add unix-ms 1))
48+--
49+```
50+
51+As we discussed in the previous section, our state will contain a `$journal`
52+structure containing all our journal entries, and a `$log` structure containing
53+the update log. These are both _ordered maps_, defined as `((mop id txt) gth)`
54+and `((mop @ action) lth)` respectively. We can therefore define our _versioned
55+state_ as `[%0 =journal =log]`, in the usual manner.
56+
57+We've define `$card` for convenience as usual, and we've also added three more
58+arms. The first two relate to our two ordered maps. If you'll recall, an
59+ordinary `map` is called with the `++by` door in the standard library, like so:
60+
61+```hoon
62+(~(get by foo) %bar)
63+```
64+
65+An ordered map uses the `++on` gate in `zuse.hoon` rather than `++by`, and its
66+invocation is slightly different. It must first be setup in a similar manner to
67+the `mop` type, by providing it the key/value molds and comparator gates. Once
68+that's done, its individual functions can be called with the `mop` and
69+arguments, like:
70+
71+```hoon
72+(get:((on @ud @ud) gth) foo %bar)
73+```
74+
75+This is quite a cumbersome expression to use every time we want to interact with
76+our `mop`. To make it easier, we can store the `((on @ud @ud) gth)` part in an
77+arm, and then when we need to use it we can just do `(get:arm-name foo %bar)`.
78+In this case, we've done one each of our ordered maps like so:
79+
80+```hoon
81+++ j-orm ((on id txt) gth)
82+++ log-orm ((on @ action) lth)
83+```
84+
85+The last arm in our state definition core is `++unique-time`. Since we'll use
86+`now.bowl` to derive the timestamp for updates, we run into an issue if multiple
87+pokes arrive in a single Arvo event. In that case, `now.bowl` would be the same
88+for each poke, so they'd be given the same key and override each other in the
89+`mop`. To avoid this, `++unique-time` is just a simple recursive function that
90+will increment the timestamp by one millisecond if the key already exists in the
91+`$log` `mop`, ensuring all updates get unique timestamps and there are no
92+collisions.
93+
94+## Agent core setup
95+
96+```hoon
97+%- agent:dbug
98+=| state-0
99+=* state -
100+^- agent:gall
101+|_ =bowl:gall
102++* this .
103+ def ~(. (default-agent this %|) bowl)
104+ io ~(. agentio bowl)
105+++ on-init on-init:def
106+++ on-save
107+ ^- vase
108+ !>(state)
109+::
110+++ on-load
111+ |= old-vase=vase
112+ ^- (quip card _this)
113+ `this(state !<(versioned-state old-vase))
114+::
115+```
116+
117+Here we setup our agent core and define the three lifecycle arms. Since we only
118+have a single state version at present, these are very simple functions. You'll
119+notice in our `+*` arm, along with the usual `this` and `def`, we've also setup
120+the `agentio` library we imported, giving it the bowl and an alias of `io`.
121+
122+## Pokes
123+
124+```hoon
125+++ on-poke
126+ |= [=mark =vase]
127+ ^- (quip card _this)
128+ |^
129+ ?> (team:title our.bowl src.bowl)
130+ ?. ?=(%journal-action mark) (on-poke:def mark vase)
131+ =/ now=@ (unique-time now.bowl log)
132+ =/ act !<(action vase)
133+ =. state (poke-action act)
134+ :_ this(log (put:log-orm log now act))
135+ ~[(fact:io journal-update+!>(`update`[now act]) ~[/updates])]
136+ ::
137+ ++ poke-action
138+ |= act=action
139+ ^- _state
140+ ?- -.act
141+ %add
142+ ?< (has:j-orm journal id.act)
143+ state(journal (put:j-orm journal id.act txt.act))
144+ ::
145+ %edit
146+ ?> (has:j-orm journal id.act)
147+ state(journal (put:j-orm journal id.act txt.act))
148+ ::
149+ %del
150+ ?> (has:j-orm journal id.act)
151+ state(journal +:(del:j-orm journal id.act))
152+ ==
153+ --
154+::
155+```
156+
157+Here we have our `++on-poke` arm, where we handle `$action`s. Since our
158+`%journal` agent is intended for local use only, we make sure only our ship or
159+our moons may perform actions with:
160+
161+```hoon
162+?> (team:title our.bowl src.bowl)
163+```
164+
165+We haven't yet written our mark files, but our mark for `$action`s will be
166+`%journal-action`, so we make sure that's what we've received and if not, call
167+`++on-poke:def` to crash with an error message. We make sure the the timestamps
168+are unique with our `++unique-time` function described earlier, and then we
169+extract the poke's vase to an `$action` structure and call `++poke-action` to
170+handle it. We've made `++on-poke` a door with a separate `++poke-action` arm to
171+make the logic a little simpler, but in principle we could have had it all
172+directly inside the main `++poke-action` gate, or even separated it out into a
173+helper core below.
174+
175+The logic in `++poke-action` is very simple, with three cases for each of the possible `$action`s:
176+
177+- `%add` - Add a new journal entry. We check it doesn't already exist with
178+ `++has:j-orm`, and then add it to our `$journal` with `++put:j-orm`.
179+
180+- `%edit` - Edit an existing journal entry. We make sure it _does_ exist with
181+ `++has:j-orm`, and then override the old entry with the new one using
182+ `++put:j-orm` again.
183+
184+- `%del` - Delete an existing journal entry. We make sure it exists again with
185+ `++has:j-orm`, and then use `++del:j-orm` to delete it from our `$journal`
186+ `mop`.
187+
188+Back in the main part of `++on-poke`, `++poke-action` updates the state with the
189+new `$journal`, then we proceed to:
190+
191+```hoon
192+:_ this(log (put:log-orm log now act))
193+~[(fact:io journal-update+!>(`update`[now act]) ~[/updates])]
194+```
195+
196+We add the timestamp to the action, converting it to a logged update. We add it
197+to the `$log` update log using `++put:log-orm`, and also send the logged update
198+out to subscribers on the `/updates` subscription path. We haven't written our
199+mark files yet, but `%journal-update` is the mark we'll use for `$update`s, so
200+we pack the `$update` in a vase and add the mark to make it a `$cage`. Notice
201+we're using the `++fact` function in `agentio` (which we aliased as `io`) rather
202+than manually composing the `%fact`.
203+
204+## Subscriptions
205+
206+```hoon
207+++ on-watch
208+ |= =path
209+ ^- (quip card _this)
210+ ?> (team:title our.bowl src.bowl)
211+ ?+ path (on-watch:def path)
212+ [%updates ~] `this
213+ ==
214+::
215+```
216+
217+Our subscription logic is extremely simple - we just have a single `/updates`
218+path, which the front-end or other local agents may subscribe to. All updates
219+get sent out on this path. We enforce local-only with the `team:title` check.
220+
221+We could have had our `++on-watch` arm send out some initial state to new
222+subscribers, but for our front-end we'll instead fetch the initial state
223+separately with a scry. This just makes it slightly easier if our front-end
224+needs to resubscribe at some point - it'll already have some state in that case
225+so we don't want it to get sent again.
226+
227+## Scry Endpoints
228+
229+```hoon
230+++ on-peek
231+ |= =path
232+ ^- (unit (unit cage))
233+ ?> (team:title our.bowl src.bowl)
234+ =/ now=@ (unm:chrono:userlib now.bowl)
235+ ?+ path (on-peek:def path)
236+ [%x %entries *]
237+ ?+ t.t.path (on-peek:def path)
238+ [%all ~]
239+ :^ ~ ~ %journal-update
240+ !> ^- update
241+ [now %jrnl (tap:j-orm journal)]
242+ ::
243+ [%before @ @ ~]
244+ =/ before=@ (rash i.t.t.t.path dem)
245+ =/ max=@ (rash i.t.t.t.t.path dem)
246+ :^ ~ ~ %journal-update
247+ !> ^- update
248+ [now %jrnl (tab:j-orm journal `before max)]
249+ ::
250+ [%between @ @ ~]
251+ =/ start=@
252+ =+ (rash i.t.t.t.path dem)
253+ ?:(=(0 -) - (sub - 1))
254+ =/ end=@ (add 1 (rash i.t.t.t.t.path dem))
255+ :^ ~ ~ %journal-update
256+ !> ^- update
257+ [now %jrnl (tap:j-orm (lot:j-orm journal `end `start))]
258+ ==
259+ ::
260+ [%x %updates *]
261+ ?+ t.t.path (on-peek:def path)
262+ [%all ~]
263+ :^ ~ ~ %journal-update
264+ !> ^- update
265+ [now %logs (tap:log-orm log)]
266+ ::
267+ [%since @ ~]
268+ =/ since=@ (rash i.t.t.t.path dem)
269+ :^ ~ ~ %journal-update
270+ !> ^- update
271+ [now %logs (tap:log-orm (lot:log-orm log `since ~))]
272+ ==
273+ ==
274+::
275+```
276+
277+Here we have our `++on-scry` arm. The scry endpoints we've defined are divided
278+into two parts: querying the update `$log` and retrieving entries from the
279+`$journal`. Each end-point is as follows:
280+
281+- `/x/entries/all` - Retrieve all entries in the `$journal`. Our front-end will
282+ use lazy-loading and only get a few at a time, so it won't use this. It's nice
283+ to have it though, in case other agents want to get that data.
284+
285+- `/x/entries/before/[before]/[max]` - Retrieve at most `[max]` entries older
286+ than the entry on `[before]` date. This is so our lazy-loading front-end can
287+ progressively load more as the user scrolls down the page. The Javascript
288+ front-end will format numbers without dot separators, so the path will look
289+ like `/x/entries/before/1648051573109/10`. We therefore have to use the
290+ [`++dem`](docs/hoon/reference/stdlib/4i#dem) parsing `rule` in a
291+ [`++rash`](/docs/hoon/reference/stdlib/4g#rash) parser to convert it to an
292+ ordinary atom. We then use the `++tap:log-orm` `mop` function to retrieve the
293+ requested range as a list and return it as an `$update` with a
294+ `%journal-update` mark.
295+
296+- `/x/entries/between/[start]/[end]` - Retrieve all journal entries between two
297+ dates. This is so our front-end can have a search function, where the user can
298+ enter a start and end date and get all the entries in between. The
299+ `++lot:j-orm` `mop` function returns the subset of a `mop` between the two
300+ given keys as a `mop`, and then we call `++tap:j-orm` to convert it to a list.
301+ The `++lot:j-orm function` excludes the start and end values, so we subtract 1
302+ from the start and add 1 to the end to make sure it includes the full range.
303+
304+- `/x/updates/all` - Retrieve the entire update `$log`. Our front-end won't use
305+ this but it might be useful for other agents, so we've included it here.
306+
307+- `/x/updates/since/[since]` - Retrieve all `$update`s that have happened since
308+ the specified timestamp, if any. This is so our front-end (or another agent)
309+ can resynchronize its state in the event its subscription is interrupted,
310+ without having to fetch everything from scratch again.
311+
312+We don't use any of the other agent arms, so the remainder have all been passed
313+to `default-agent` for handling:
314+
315+```hoon
316+++ on-leave on-leave:def
317+++ on-agent on-agent:def
318+++ on-arvo on-arvo:def
319+++ on-fail on-fail:def
320+--
321+```
322+
323+The full agent source can be viewed
324+[here](https://github.com/urbit/docs-examples/blob/main/journal-app/desk/app/journal.hoon).
325+
326+## Resources
327+
328+- [The Gall Guide](/docs/userspace/gall-guide/intro) - The Gall Guide covers all
329+ aspects of writing Gall agents in detail.
330+
331+- [Ordered map functions in
332+ `zuse.hoon`](https://github.com/urbit/urbit/blob/master/pkg/arvo/sys/zuse.hoon#L5284-L5688) -
333+ This section of `zuse.hoon` contains all the functions for working with
334+ `mop`s, and is well commented.
335+
336+- [`/lib/agentio.hoon`](https://github.com/urbit/urbit/blob/master/pkg/base-dev/lib/agentio.hoon) -
337+ The `agentio` library in the `%base` desk contains a large number of useful
338+ functions which making writing Gall agents easier.
content/docs/userspace/full-stack/4-json.mdnew file+357
@@ -0,0 +1,357 @@
1++++
2+title = "4. JSON"
3+weight = 4
4+template = "doc.html"
5++++
6+
7+Data sent between our agent and our front-end will all be encoded as JSON. In
8+this section, we'll briefly look at how JSON works in Urbit, and write a library
9+to convert our agent's structures to and from JSON for our front-end.
10+
11+JSON data comes into Eyre as a string, and Eyre parses it with the
12+[`++de-json:html`](/docs/hoon/reference/zuse/2e_2-3#de-jsonhtml) function in
13+[`zuse.hoon`](/docs/hoon/reference/zuse/table-of-contents). The
14+hoon type it's parsed to is `$json`, which is defined as:
15+
16+```hoon
17++$ json :: normal json value
18+ $@ ~ :: null
19+ $% [%a p=(list json)] :: array
20+ [%b p=?] :: boolean
21+ [%o p=(map @t json)] :: object
22+ [%n p=@ta] :: number
23+ [%s p=@t] :: string
24+ == ::
25+```
26+
27+Once Eyre has converted the raw JSON string to a `$json` structure, it will be
28+converted to the mark the web client specified and then delivered to the target
29+agent (unless the mark specified is already `%json`, in which case it will be
30+delivered directly). Outbound facts will go through the same process in
31+reverse - converted from the agent's native mark to `$json`, then encoded in a
32+string by Eyre using
33+[`++en-json:html`](<(/docs/hoon/reference/zuse/2e_2-3#en-jsonhtml)>) and delivered
34+to the web client. The basic flow for both inbound messages (pokes) and outbound
35+messages (facts and scry results) looks like this:
36+
37+![eyre mark flow diagram](https://media.urbit.org/docs/userspace/full-stack-guide/eyre-mark-flow-diagram.svg)
38+
39+The mark conversion will be done by the corresponding mark file in `/mar` on the
40+agent's desk. In our case it would be `/mar/journal/action.hoon` and
41+`/mar/journal/update.hoon` in the `%journal` desk for our `%journal-action` and
42+`%journal-update` marks, which are for the `$action` and `$update` structures we
43+defined previously.
44+
45+Mark conversion functions can be included directly in the mark file, or they can
46+be written in a separate library, then imported and called by the mark file. We
47+will do the latter in this case, so before we create the mark files themselves,
48+we'll write a library called `/lib/journal.hoon` with the conversion functions.
49+
50+## `$json` utilities
51+
52+[`zuse.hoon`](/docs/hoon/reference/zuse/table-of-contents) contains three main
53+cores for converting to and from `$json`:
54+
55+- [`++enjs:format`](/docs/hoon/reference/zuse/2d_1-5#enjsformat) - Functions to
56+ help encode data structures as `$json`.
57+- [`++dejs:format`](/docs/hoon/reference/zuse/2d_6#dejsformat) - Functions to
58+ decode `$json` to other data structures.
59+- [`++dejs-soft:format`](/docs/hoon/reference/zuse/2d_7#dejs-softformat) -
60+ Mostly the same as `++dejs:format` except the functions produce units which
61+ are null if decoding fails, rather than just crashing.
62+
63+### `++enjs:format`
64+
65+This contains ten functions for encoding `$json`. Most of them are for specific
66+hoon data types, such as `++tape:enjs:format`, `++ship:enjs:format`,
67+`++path:enjs:format`, etc. We'll just have a look at the two most general and
68+useful ones: `++frond:enjs:format` and `++pairs:enjs:format`.
69+
70+#### `++frond`
71+
72+This function is for forming a JSON object from a single key-value pair. For
73+example:
74+
75+```
76+> (frond:enjs:format 'foo' s+'bar')
77+[%o p={[p='foo' q=[%s p='bar']]}]
78+```
79+
80+When stringified by Eyre, this will look like:
81+
82+```json
83+{ "foo": "bar" }
84+```
85+
86+#### `++pairs`
87+
88+This is similar to `++frond` and also forms a JSON object, but it takes multiple
89+key-value pairs rather than just one:
90+
91+```
92+> (pairs:enjs:format ~[['foo' n+~.123] ['bar' s+'abc'] ['baz' b+&]])
93+[%o p={[p='bar' q=[%s p='abc']] [p='baz' q=[%b p=%.y]] [p='foo' q=[%n p=~.123]]}]
94+```
95+
96+When stringified by Eyre, this will look like:
97+
98+```json
99+{
100+ "foo": 123,
101+ "baz": true,
102+ "bar": "abc"
103+}
104+```
105+
106+Notice that we used a knot for the value of `foo` (`n+~.123`). Numbers in JSON
107+can be signed or unsigned and integers or floating point values. The `$json`
108+structure uses a knot so that you can decide whether a particular number should
109+be treated as `@ud`, `@sd`, `@rs`, etc.
110+
111+### `++dejs:format`
112+
113+This core contains many functions for decoding `$json`. We'll touch on some
114+useful families of `++dejs` functions in brief, but because there's so many, in
115+practice you'll need to look through the [`++dejs`
116+reference](/docs/hoon/reference/zuse/2d_6) to find the correct functions for
117+your use case.
118+
119+#### Number functions
120+
121+- `++ne` - decode a number to a `@rd`.
122+- `++ni` - decode a number to a `@ud`.
123+- `++no` - decode a number to a `@ta`.
124+- `++nu` - decode a hexadecimal string to a `@ux`.
125+
126+For example:
127+
128+```
129+> (ni:dejs:format n+'123')
130+123
131+```
132+
133+#### String functions
134+
135+- `++sa` - decode a string to a `tape`.
136+- `++sd` - decode a string containing a `@da` aura date value to a `@da`.
137+- `++se` - decode a string containing the specified aura to that aura.
138+- `++so` - decode a string to a `@t`.
139+- `++su` - decode a string by parsing it with the given [parsing
140+ rule](/docs/hoon/reference/stdlib/4f).
141+
142+#### Array functions
143+
144+`++ar`, `++as`, and `++at` decode a `$json` array to a `list`, `set`, and
145+n-tuple respectively. These gates take other `++dejs` functions as an argument,
146+producing a new gate that will then take the `$json` array. For example:
147+
148+```
149+> ((ar so):dejs:format a+[s+'foo' s+'bar' s+'baz' ~])
150+<|foo bar baz|>
151+```
152+
153+Notice that `++so` is given as the argument to `++ar`. `++so` is a `++dejs`
154+function that decodes a `$json` string to a `cord`. The gate resulting from `(ar so)` is then called with a `$json` array as its argument, and its product is a
155+`(list @t)` of the elements of the array.
156+
157+Many `++dejs` functions take other `++dejs` functions as their arguments. A
158+complex nested `$json` decoding function can be built up in this manner.
159+
160+#### Object functions
161+
162+- `++of` - decode an object containing a single key-value pair to a head-tagged
163+ cell.
164+- `++ot` - decode an object to a n-tuple.
165+- `++ou` - decode an object to an n-tuple, replacing optional missing values
166+ with a given value.
167+- `++oj` - decode an object of arrays to a `jug`.
168+- `++om` - decode an object to a `map`.
169+- `++op` - decode an object to a `map`, and also parse the object keys with a
170+ [parsing rule](/docs/hoon/reference/stdlib/4f).
171+
172+For example:
173+
174+```
175+> =js %- need %- de-json:html
176+ '''
177+ {
178+ "foo": "hello",
179+ "baz": true,
180+ "bar": 123
181+ }
182+ '''
183+
184+> %- (ot ~[foo+so bar+ni]):dejs:format js
185+['hello' 123]
186+```
187+
188+## Our types as JSON
189+
190+We need to decide how our `$action` and `$update` types will be represented as
191+JSON in order to write our conversion functions. There are many ways to do this,
192+but in this case we'll do it as follows:
193+
194+### Actions
195+
196+| JSON | Noun |
197+| ------------------------------------------------- | ---------------------------------------------- |
198+| `{"add":{"id":1648366311070,"txt":"some text"}}` | `[%add id=1.648.366.034.844 txt='some text']` |
199+| `{"edit":{"id":1648366311070,"txt":"some text"}}` | `[%edit id=1.648.366.034.844 txt='some text']` |
200+| `{"del":{"id":1648366311070}}` | `[%del id=1.648.366.034.844]` |
201+
202+### Updates
203+
204+| Noun | JSON |
205+| ----------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
206+| `[1.648.366.492.459 %add id=1.648.366.034.844 txt='some text']` | `{time:1648366481425,"add":{"id":1648366311070,"txt":"some text"}}` |
207+| `[1.648.366.492.459 %edit id=1.648.366.034.844 txt='some text']` | `{time:1648366481425,"edit":{"id":1648366311070,"txt":"some text"}}` |
208+| `[1.648.366.492.459 %del id=1.648.366.034.844]` | `{time:1648366481425,"del":{"id":1648366311070}}` |
209+| `[1.648.366.492.459 %jrnl ~[[id=1.648.366.034.844 txt='some text'] ...]` | `{time:1648366481425,"entries":[{"id":1648366311070,"txt":"some text"},...]}` |
210+| `[1.648.366.492.459 %logs ~[[1.648.366.492.459 %add id=1.648.366.034.844 txt='some text'] ...]` | `{time:1648366481425,"logs":[{time:1648366481425,"add":{id":1648366311070,"txt":"some text"}},...]}` |
211+
212+Now let's write our library of encoding/decoding functions.
213+
214+## `/lib/journal.hoon`
215+
216+```hoon
217+/- *journal
218+|%
219+```
220+
221+First, we'll import the `/sur/journal.hoon` structures we previously created.
222+Next, we'll create two arms in our core, `++dejs-action` and `++enjs-update`, to
223+handle incoming poke `$action`s and outgoing facts or scry result `$update`s.
224+
225+### `$json` to `$action`
226+
227+```hoon
228+++ dejs-action
229+ =, dejs:format
230+ |= jon=json
231+ ^- action
232+ %. jon
233+ %- of
234+ :~ [%add (ot ~[id+ni txt+so])]
235+ [%edit (ot ~[id+ni txt+so])]
236+ [%del (ot ~[id+ni])]
237+ ==
238+```
239+
240+The first thing we do is use the [`=,`
241+rune](/docs/hoon/reference/rune/tis#-tiscom) to expose the `++dejs:format`
242+namespace. This allows us to reference `ot`, `ni`, etc rather than having to
243+write `ot:dejs:format` every time. Note that you should be careful using `=,`
244+generally as the exposed wings can shadow previous wings if they have the same
245+name.
246+
247+We then create a gate that takes `$json` and returns a `$action` structure.
248+Since we'll only take one action at a time, we can use the `++of` function,
249+which takes a single key-value pair. `++of` takes a list of all possible `$json`
250+objects it will receive, tagged by key.
251+
252+For each key, we specify a function to handle its value. Ours will be objects,
253+so we use `++ot` and specify the pairs of the key and `+dejs` function to decode
254+it. We then cast the output to our `$action` structure.
255+
256+You'll notice the nesting of these `++dejs` functions approximately reflects the
257+nested structure of the `$json` it's decoding.
258+
259+### `$update` to `$json`
260+
261+```hoon
262+++ enjs-update
263+ =, enjs:format
264+ |= upd=update
265+ ^- json
266+ |^
267+ ?+ -.q.upd (logged upd)
268+ %jrnl
269+ %- pairs
270+ :~ ['time' (numb p.upd)]
271+ ['entries' a+(turn list.q.upd entry)]
272+ ==
273+ ::
274+ %logs
275+ %- pairs
276+ :~ ['time' (numb p.upd)]
277+ ['logs' a+(turn list.q.upd logged)]
278+ ==
279+ ==
280+ ++ entry
281+ |= ent=^entry
282+ ^- json
283+ %- pairs
284+ :~ ['id' (numb id.ent)]
285+ ['txt' s+txt.ent]
286+ ==
287+ ++ logged
288+ |= lgd=^logged
289+ ^- json
290+ ?- -.q.lgd
291+ %add
292+ %- pairs
293+ :~ ['time' (numb p.lgd)]
294+ :- 'add'
295+ %- pairs
296+ :~ ['id' (numb id.q.lgd)]
297+ ['txt' s+txt.q.lgd]
298+ == ==
299+ %edit
300+ %- pairs
301+ :~ ['time' (numb p.lgd)]
302+ :- 'edit'
303+ %- pairs
304+ :~ ['id' (numb id.q.lgd)]
305+ ['txt' s+txt.q.lgd]
306+ == ==
307+ %del
308+ %- pairs
309+ :~ ['time' (numb p.lgd)]
310+ :- 'del'
311+ (frond 'id' (numb id.q.lgd))
312+ ==
313+ ==
314+ --
315+--
316+```
317+
318+Our `$update` encoding function's a little more complex than our `$action`
319+decoding function, since our `$update` structure is more complex.
320+
321+Like the previous one, we use `=,` to expose the namespace of `++enjs:format`.
322+
323+Our gate takes an `$update` and returns a `$json` structure. We use `|^` so we
324+can separate out the encoding functions for individual entries (`++entry`) and
325+individual logged actions (`++logged`).
326+
327+We first test the head of the `$update`, and if it's `%jrnl` (a list of
328+entries), we `turn` over the entries and call `++entry` to encode each one. If
329+it's `%logs`, we do the same, but call `++logged` for each item in the list.
330+Otherwise, if it's just a single update, we encode it with `++logged`.
331+
332+We primarily use `++pairs` to form the object, though sometimes `++frond` if it
333+only contains a single key-value pair. We also use `++numb` to encode numerical
334+values.
335+
336+You'll notice more of our encoding function is done manually than our previous
337+decoding function. For example, we form arrays by tagging an ordinary `list`
338+with `%a`, and strings by tagging an ordinary `cord` with `%s`. This is typical
339+when you write `$json` encoding functions, and is the reason there are far fewer
340+`+enjs` functions than `+dejs` functions.
341+
342+## Resources
343+
344+- [The JSON Guide](/docs/hoon/guides/json-guide) - The stand-alone JSON guide
345+ covers JSON encoding/decoding in great detail.
346+- [The Zuse reference](/docs/hoon/reference/zuse/table-of-contents) - The
347+ `zuse.hoon` reference documents all JSON-related functions in detail.
348+
349+- [`++enjs:format` reference](/docs/hoon/reference/zuse/2d_1-5#enjsformat) -
350+ This section of the `zuse.hoon` documentation covers all JSON encoding
351+ functions.
352+
353+- [`++dejs:format` reference](/docs/hoon/reference/zuse/2d_6) - This section of
354+ the `zuse.hoon` documentation covers all JSON _decoding_ functions.
355+
356+- [Eyre overview](/docs/arvo/eyre/eyre) - This section of the Eyre vane
357+ documentation goes over the basic features of the Eyre vane.
content/docs/userspace/full-stack/5-marks.mdnew file+97
@@ -0,0 +1,97 @@
1++++
2+title = "5. Marks"
3+weight = 5
4+template = "doc.html"
5++++
6+
7+In this section we'll write the mark files for our agent. We'll need two marks,
8+one for poke `$action`s and one for subscription updates and scry results, both
9+of which are `$update`s. Our `$action` mark will be called `%journal-action` and
10+our `$update` mark will be called `%journal-update`. These will be located at
11+`/mar/journal/action.hoon` and `/mar/journal/update.hoon`.
12+
13+Note that a mark called `%foo-bar` will first be looked for in
14+`/mar/foo-bar.hoon`, and if it's not there it will be looked for in
15+`/mar/foo/bar.hoon`. That's why we can have a single name like `%journal-action`
16+but have it in `/mar/journal/action.hoon`
17+
18+## `%journal-action`
19+
20+```hoon
21+/- *journal
22+/+ *journal
23+|_ act=action
24+++ grow
25+ |%
26+ ++ noun act
27+ --
28+++ grab
29+ |%
30+ ++ noun action
31+ ++ json dejs-action
32+ --
33+++ grad %noun
34+--
35+```
36+
37+First we import our `/sur/journal.hoon` structure file and also our
38+`/lib/journal.hoon` library (containing our `$json` conversion functions). The
39+sample of our mark door is just our `$action` structure. The `++grow` arm of a
40+mark core, if you recall, contains methods for converting _from_ our mark _to_
41+another mark. Actions only ever come inwards in pokes, so we don't need to worry
42+about converting an `$action` _to_ `$json`. The `++grow` arm can therefore just
43+handle the generic `%noun` case, simply returning our mark door's sample without
44+doing anything.
45+
46+`++grab`, conversely, defines methods for converting _to_ our mark _from_
47+another mark. Since `$action`s will come in from the front-end as `$json`, we
48+need to be able to convert `$json` data to our `$action` structure. Our
49+`/lib/journal.hoon` library contains the `++dejs-action` function for performing
50+this conversion, so we can just specify that function for the `%json` case.
51+We'll also define a standard `%noun` method, which will just "clam" (or "mold")
52+the incoming noun with the `$action` mold. Clamming/molding coerces a noun to a
53+type and is done by calling a mold as a function.
54+
55+Lastly, `++grad` defines revision control methods, but can be delegated to
56+another mark. Since this mark will never be used for actually storing files in
57+Clay, we can just delegate it to the generic `%noun` mark rather than writing a
58+proper set of `++grad` methods.
59+
60+## `%journal-update`
61+
62+```hoon
63+/- *journal
64+/+ *journal
65+|_ upd=update
66+++ grow
67+ |%
68+ ++ noun upd
69+ ++ json (enjs-update upd)
70+ --
71+++ grab
72+ |%
73+ ++ noun update
74+ --
75+++ grad %noun
76+--
77+```
78+
79+Next we have our `%journal-update` mark file. The sample of our mark door is our
80+`$update` structure. Our `$update`s are always outbound, never inbound, so we
81+only need to define a method for converting our `$update` structure to `$json`
82+in the `++grow` arm, and not the opposite direction in `++grad`. Our
83+`/lib/journal.hoon` library contains the `++enjs-update` function for performing
84+this conversion, so we can call it with the sample `$update` as its argument. We
85+can add `%noun` conversion methods and delegate revision control to the `%noun`
86+mark in the same manner as our `%journal-action` mark above.
87+
88+## Resources
89+
90+- [The Marks section of the Clay documentation](/docs/arvo/clay/marks/marks) -
91+ This section of the Clay vane documentation covers mark files comprehensively.
92+- [The mark file section of the Gall
93+ Guide](/docs/userspace/gall-guide/7-sur-and-marks#mark-files) - This part of
94+ the Gall Guide goes through the basics of mark files.
95+
96+- [The JSON Guide](/docs/hoon/guides/json-guide) - This also covers writing mark
97+ files to convert to/from JSON.
content/docs/userspace/full-stack/6-eyre.mdnew file+79
@@ -0,0 +1,79 @@
1++++
2+title = "6. Eyre"
3+weight = 6
4+template = "doc.html"
5++++
6+
7+Now that we have our structure file, agent, `$json` conversion library and mark
8+file, our back-end is complete. Before we start writing our front-end, though,
9+we should give a brief overview of how Eyre works.
10+
11+[Eyre](/docs/arvo/eyre/eyre) is the HTTP server [vane](/docs/glossary/vane) of
12+Arvo. Eyre has a handful of different subsystems, but the main two are the
13+channel system and the scry interface. These two are what we'll focus on here.
14+
15+In order to use the channel system or perform scries, a web client must have
16+authenticated with the ship's web login code (e.g.
17+`lidlut-tabwed-pillex-ridrup`) and obtained a session cookie. Our front-end will
18+be served directly from the ship by the `%docket` agent, so we can assume a
19+session cookie was already obtained when the user logged into landscape, and
20+skip over authentication.
21+
22+## Channels
23+
24+Eyre's channel system is the main way to interact with agents from a web client.
25+It provides a JSON interface to the ordinary poke and subscription system for
26+Gall agents.
27+
28+First, a unique channel ID is generated by the web client (`@urbit/http-api`
29+uses the current Unix time concatenated with a random hex string). The client
30+then sends a poke or subscription request for the channel, and Eyre
31+automatically opens a new channel with that ID. Once open, the client can then
32+connect to the channel and receive any events such as poke acks, watch acks,
33+facts from subscriptions, etc.
34+
35+The new channel is an SSE ([Server Sent
36+Event](https://html.spec.whatwg.org/#server-sent-events)) stream, and can be
37+handled by an `EventSource` object in Javascript. The `@urbit/http-api` library
38+we'll use abstracts this for us, so we won't need to deal with an `EventSource`
39+object directly. The channel can handle multiple concurrent subscriptions to
40+different agent subscription paths, and different agents can be poked through
41+the one channel. This means a client only needs to open a single channel for all
42+of its interactions with the ship. Each subscription is given a different ID, so
43+they can be individually unsubscribed later.
44+
45+A channel will timeout after 12 hours of inactivity, and the timeout is reset
46+every time Eyre receives a message of any kind from the client. Additionally,
47+each subscription on the channel may only accumulate 50 unacknowledged facts
48+before it's considered "clogged", in which case the individual clogged
49+subscription will be closed by Eyre after a short delay. All events of any kind
50+which Eyre sends out on the channel must be ack'd by the client. Ack'ing one
51+event will also ack all previous events too. The `@urbit/http-api` library we'll
52+use automatically acks events for us, so we don't need to worry about clogged
53+subscriptions or manually ack'ing events.
54+
55+Eyre expects a particular JSON object structure for each of these different
56+requests, but the `@urbit/http-api` library we'll use includes functions to send
57+pokes, subscription requests, etc, so we won't need to manually construct the
58+JSON objects in our front-end.
59+
60+## Scries
61+
62+Eyre's scry interface is separate to the channel system. Scries are performed by
63+a simple GET request to a path with a format of `/~/scry/{agent}{path}.{mark}`.
64+If successful, the HTTP response will contain the result with the mark
65+specified. If unsuccessful, an HTTP error will be thrown in response.
66+
67+The `@urbit/http-api` library we'll use includes a function for performing
68+scries, so we'll not need to manually send GET requests to the ship.
69+
70+## Resources
71+
72+- [The Eyre vane documentation](/docs/arvo/eyre/eyre) - This section of the vane
73+ docs covers all aspects of Eyre.
74+- [Eyre External API Reference](/docs/arvo/eyre/external-api-ref) - This section
75+ of the Eyre documentation contains reference material for Eyre's external API.
76+
77+- [The Eyre Guide](/docs/arvo/eyre/guide) - This section of the Eyre
78+ documentation walks through using Eyre's external API at a low level (using
79+ `curl`).
content/docs/userspace/full-stack/7-react-setup.mdnew file+214
@@ -0,0 +1,214 @@
1++++
2+title = "7. React app setup"
3+weight = 7
4+template = "doc.html"
5++++
6+
7+Now that we have a basic idea of how Eyre works, we can begin working on our
8+React app front-end.
9+
10+## Create React app
11+
12+Node.js must be installed, and can be downloaded from their
13+[website](https://nodejs.org/en/download). With that installed, we'll have the
14+`npm` package manager available. The first thing we'll do is globally install
15+the `create-react-app` package with the following command:
16+
17+```sh
18+npm install -g create-react-app
19+```
20+
21+Once installed, we can use it to create a new `journal-ui` directory and setup a
22+new React app in it with the following command:
23+
24+```sh
25+create-react-app journal-ui
26+```
27+
28+We can then open our new directory:
29+
30+```sh
31+cd journal-ui
32+```
33+
34+Its contents should look something like this:
35+
36+```
37+journal-ui
38+├── node_modules
39+├── package.json
40+├── package-lock.json
41+├── public
42+├── README.md
43+└── src
44+```
45+
46+## Install `http-api`
47+
48+Inside our React app directory, let's install the `@urbit/http-api` NPM package:
49+
50+```sh
51+npm i @urbit/http-api
52+```
53+
54+We also install a handful of other packages for the UI components (`bootstrap react-bootstrap react-textarea-autosize date-fns react-bottom-scroll-listener react-day-picker`), but that's not important to our purposes here.
55+
56+## Additional tweaks
57+
58+Our front-end will be served directly from the ship by the `%docket` app, where
59+a user will open it by clicking on its homescreen tile. Docket serves such
60+front-ends with a base URL path of `/apps/[desk]/`, so in our case it will be
61+`/apps/journal`. In order for our app to be built with correct resource paths,
62+we must add the following line to `package.json`:
63+
64+```json
65+"homepage": "/apps/journal/",
66+```
67+
68+Our app also needs to know the name of the ship it's being served from in order
69+to talk with it. The `%docket` agent serves a small file for this purpose at
70+`[host]/session.js`. This file is very simple and just contains:
71+
72+```js
73+window.ship = "sampel-palnet";
74+```
75+
76+`sampel-palnet` will of course be replaced by the actual name of the ship. We
77+include this script by adding the following line to the `&lt;head>` section of
78+`public/index.html`:
79+
80+```
81+<script src="/session.js"></script>
82+```
83+
84+## Basic API setup
85+
86+With everything now setup, we can begin work on the app itself. In this case
87+we'll just edit the existing `App.js` file in the `/src` directory. The first thing is to import the `Urbit` class from `@urbit/http-api`:
88+
89+```js
90+import Urbit from "@urbit/http-api";
91+```
92+
93+We also need to import a few other things, mostly relating to UI components (but
94+these aren't important for our purposes here):
95+
96+```js
97+import React, { Component } from "react";
98+import "bootstrap/dist/css/bootstrap.min.css";
99+import "react-day-picker/lib/style.css";
100+import TextareaAutosize from "react-textarea-autosize";
101+import Button from "react-bootstrap/Button";
102+import Card from "react-bootstrap/Card";
103+import Stack from "react-bootstrap/Stack";
104+import Tab from "react-bootstrap/Tab";
105+import Tabs from "react-bootstrap/Tabs";
106+import ToastContainer from "react-bootstrap/ToastContainer";
107+import Toast from "react-bootstrap/Toast";
108+import Spinner from "react-bootstrap/Spinner";
109+import CloseButton from "react-bootstrap/CloseButton";
110+import Modal from "react-bootstrap/Modal";
111+import DayPickerInput from "react-day-picker/DayPickerInput";
112+import endOfDay from "date-fns/endOfDay";
113+import startOfDay from "date-fns/startOfDay";
114+import { BottomScrollListener } from "react-bottom-scroll-listener";
115+```
116+
117+Inside the existing `App` class:
118+
119+```js
120+class App extends Component {
121+```
122+
123+...we'll clear out the existing demo code and start adding ours. The first thing
124+is to define our app's state. We'll look at most of the state entries in the
125+next section. For now, we'll just consider `status`.
126+
127+```js
128+state = {
129+ // .....
130+ status: null,
131+ // .....
132+};
133+```
134+
135+Next, we'll setup the `Urbit` API object in `componentDidMount`. We could do
136+this outside the `App` class since we're adding it to `window`, but we'll do it
137+this way so it's all in one place:
138+
139+```js
140+componentDidMount() {
141+ window.urbit = new Urbit("");
142+ window.urbit.ship = window.ship;
143+ window.urbit.onOpen = () => this.setState({status: "con"});
144+ window.urbit.onRetry = () => this.setState({status: "try"});
145+ window.urbit.onError = (err) => this.setState({status: "err"});
146+ this.init();
147+};
148+```
149+
150+The first thing we do is create a new instance of the `Urbit` class we imported
151+from `@urbit/http-api`, and save it to `window.urbit`. The `Urbit` class
152+constructor takes three arguments: `url`, `desk` and `code`, of which only `url`
153+is mandatory.
154+
155+- `url` is the URL of the ship we want to talk to. Since our React app will be
156+ served by the ship, we can just leave it as an empty `""` string and let
157+ `Urbit` use root-relative paths.
158+- `desk` is only necessary if we want to run threads through Eyre, and since
159+ we're not going to do that, we can exclude it.
160+- `code` is the web login code for authentication, but since the user will
161+ already have logged in, we can also exclude that.
162+
163+Therefore, we call the class contructor with just the empty `url` string:
164+
165+```js
166+window.urbit = new Urbit("");
167+```
168+
169+Next, we need to set the ship name in our `Urbit` instance. Eyre requires the
170+ship name be specified in all requests, so if we don't set it, Eyre will reject
171+all the messages we send. We previously included `session.js` which sets
172+`window.ship` to the ship name, so we just set `window.urbit.ship` as that:
173+
174+```js
175+window.urbit.ship = window.ship;
176+```
177+
178+Next, we set three callbacks: `onOpen`, `onRetry`, and `onError`. These
179+callbacks are fired when the state of our channel connection changes:
180+
181+- `onOpen` is called when a connection is established.
182+- `onRetry` is called when a channel connection has been interrupted (such as by
183+ network issues) and the `Urbit` object is trying to reconnect. Reconnection
184+ will be attempted up to three times: immediately, after 750ms, and after
185+ 3000ms.
186+- `onError` is called with an `Error` message once all retries have failed, or
187+ otherwise when a fatal error occurs.
188+
189+We'll look at how we handle these cases in the next section. For now, we'll just
190+set the `status` entry in the state to either `"con"`, `"try"`, or `"err"` as
191+the case may be. Note that it's not mandatory to set these callbacks, but
192+leaving connection problems unhandled is usually a bad idea.
193+
194+The last thing we do is call:
195+
196+```js
197+this.init();
198+```
199+
200+This function will fetch initial entries and subscribe for updates. We'll look
201+at it in the next section.
202+
203+## Resources
204+
205+- [HTTP API Guide](/docs/userspace/http-api-guide) - Reference documentation for
206+ `@urbit/http-api`.
207+
208+- [React app source
209+ code](https://github.com/urbit/docs-examples/tree/main/journal-app/ui) - The
210+ source code for the Journal app UI.
211+
212+- [`@urbit/http-api` source
213+ code](https://github.com/urbit/urbit/tree/master/pkg/npm/http-api) - The
214+ source code for the `@urbit/http-api` NPM package.
content/docs/userspace/full-stack/8-app-logic.mdnew file+406
@@ -0,0 +1,406 @@
1++++
2+title = "8. React app logic"
3+weight = 8
4+template = "doc.html"
5++++
6+
7+With the basic things setup, we can now go over the logic of our app. We'll just
8+focus on functions that are related to ship communications using the `Urbit`
9+object we previously setup, and ignore UI components and other helper functions.
10+
11+## State
12+
13+In the previous section we just mentioned the connection `status` field of our
14+state. Here's the full state of our App:
15+
16+```js
17+state = {
18+ entries: [], // list of journal entries for display
19+ drafts: {}, // edits which haven't been submitted yet
20+ newDraft: {}, // new entry which hasn't been submitted yet
21+ results: [], // search results
22+ searchStart: null, // search query start date
23+ searchEnd: null, // search query end date
24+ resultStart: null, // search results start date
25+ resultEnd: null, // search results end date
26+ searchTime: null, // time of last search
27+ latestUpdate: null, // most recent update we've received
28+ entryToDelete: null, // deletion target for confirmation modal
29+ status: null, // connection status (con, try, err)
30+ errorCount: 0, // number of errors so far
31+ errors: new Map(), // list of error messages for display
32+};
33+```
34+
35+We'll see how these are used subsequently.
36+
37+## Initialize
38+
39+The first thing our app does is call `init()`:
40+
41+```js
42+init = () => {
43+ this.getEntries().then(
44+ (result) => {
45+ this.handleUpdate(result);
46+ this.setState({ latestUpdate: result.time });
47+ this.subscribe();
48+ },
49+ (err) => {
50+ this.setErrorMsg("Connection failed");
51+ this.setState({ status: "err" });
52+ }
53+ );
54+};
55+```
56+
57+This function just calls `getEntries()` to retrieve the initial list of journal
58+entries then, if that succeeded, it calls `subscribe()` to subscribe for new
59+updates. If the initial entry retrieval failed, we set the connection `status`
60+and save an error message in the `errors` map. We'll look at what we do with
61+errors later.
62+
63+## Getting entries
64+
65+![entries screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/entries.png)
66+
67+The `getEntries` function scries our `%journal` agent for up to 10 entries
68+before the oldest we currently have. We call this initially, and then each time
69+the user scrolls to the bottom of the list.
70+
71+```js
72+getEntries = async () => {
73+ const { entries: e } = this.state;
74+ const before = e.length === 0 ? Date.now() : e[e.length - 1].id;
75+ const max = 10;
76+ const path = `/entries/before/${before}/${max}`;
77+ return window.urbit.scry({
78+ app: "journal",
79+ path: path,
80+ });
81+};
82+```
83+
84+The scry is done with the `Urbit.scry` method. This function takes two arguments
85+in an object:
86+
87+- `app` - the agent to scry.
88+- `path` - the scry path. Note the `care` is not included - all scries through
89+ Eyre are `%x` scries.
90+
91+The `Urbit.scry` method only allows JSON results, but note that scries done via
92+direct GET requests allow other marks too.
93+
94+The `Urbit.scry` method returns a Promise which will contain an HTTP error
95+message if the scry failed. We handle it with a `.then` expression back in the
96+function that called it, either [`init()`](#initialize) or `moreEntries()`. If
97+the Promise is successfuly, the results are passed to the
98+[`handleUpdate`](#updates) function which appends the new entries to the
99+existing ones in state.
100+
101+## Subscription
102+
103+A subscription to the `/updates` path of our `%journal` agent is opened with our
104+`subscribe()` function:
105+
106+```js
107+subscribe = () => {
108+ try {
109+ window.urbit.subscribe({
110+ app: "journal",
111+ path: "/updates",
112+ event: this.handleUpdate,
113+ err: () => this.setErrorMsg("Subscription rejected"),
114+ quit: () => this.setErrorMsg("Kicked from subscription"),
115+ });
116+ } catch {
117+ this.setErrorMsg("Subscription failed");
118+ }
119+};
120+```
121+
122+We use the `Urbit.subscribe` method for this, which takes five arguments in an
123+object:
124+
125+- `app` - the target agent.
126+- `path` - the `%watch` path we're subscribing to.
127+- `event` - a function to handle each fact the agent sends out. We call our
128+ `handleUpdate` function, which we'll describe below.
129+- `err` - a function to call if the subscription request is rejected (nacked).
130+ We just display an error in this case.
131+- `quit` - a function to call if we get kicked from the subscription. We also
132+ just display an error in this case.
133+
134+Note that the `Urbit.subscribe` method returns a subscription ID number. Since
135+we only have one subscription in our app which we never close, we don't bother
136+to record it. If your app has multiple subscriptions to manage, you may wish to
137+keep track of these IDs in your app's state.
138+
139+## Updates
140+
141+This `handleUpdate` function handles all updates we receive. It's called
142+whenever an event comes in for our subscription, and it's also called with the
143+results of [`getEntries`](#getting-entries) and [`getUpdates`](#error-handling)
144+(described later).
145+
146+It's a bit complex, but basically it just checks whether the JSON object is
147+`add`, `edit`, `delete`, or `entries`, and then updates the state appropriately.
148+The object it's receiving is just the `$update` structure converted to JSON by
149+the mark conversion functions we wrote previously.
150+
151+```js
152+handleUpdate = (upd) => {
153+ const { entries, drafts, results, latestUpdate } = this.state;
154+ if (upd.time !== latestUpdate) {
155+ if ("entries" in upd) {
156+ this.setState({ entries: entries.concat(upd.entries) });
157+ } else if ("add" in upd) {
158+ const { time, add } = upd;
159+ const eInd = this.spot(add.id, entries);
160+ const rInd = this.spot(add.id, results);
161+ const toE =
162+ entries.length === 0 || add.id > entries[entries.length - 1].id;
163+ const toR = this.inSearch(add.id, time);
164+ toE && entries.splice(eInd, 0, add);
165+ toR && results.splice(rInd, 0, add);
166+ this.setState({
167+ ...(toE && { entries: entries }),
168+ ...(toR && { results: results }),
169+ latestUpdate: time,
170+ });
171+ } else if ("edit" in upd) {
172+ const { time, edit } = upd;
173+ const eInd = entries.findIndex((e) => e.id === edit.id);
174+ const rInd = results.findIndex((e) => e.id === edit.id);
175+ const toE = eInd !== -1;
176+ const toR = rInd !== -1 && this.inSearch(edit.id, time);
177+ if (toE) entries[eInd] = edit;
178+ if (toR) results[rInd] = edit;
179+ (toE || toR) && delete drafts[edit.id];
180+ this.setState({
181+ ...(toE && { entries: entries }),
182+ ...(toR && { results: results }),
183+ ...((toE || toR) && { drafts: drafts }),
184+ latestUpdate: time,
185+ });
186+ } else if ("del" in upd) {
187+ const { time, del } = upd;
188+ const eInd = entries.findIndex((e) => e.id === del.id);
189+ const rInd = results.findIndex((e) => e.id === del.id);
190+ const toE = eInd !== -1;
191+ const toR = this.inSearch(del.id, time) && rInd !== -1;
192+ toE && entries.splice(eInd, 1);
193+ toR && results.splice(rInd, 1);
194+ (toE || toR) && delete drafts[del.id];
195+ this.setState({
196+ ...(toE && { entries: entries }),
197+ ...(toR && { results: results }),
198+ ...((toE || toR) && { drafts: drafts }),
199+ latestUpdate: time,
200+ });
201+ }
202+ }
203+};
204+```
205+
206+## Add, edit, delete
207+
208+![add screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/add.png)
209+
210+When a user writes a new journal entry and hits submit, the `submitNew` function
211+is called. It uses the `Urbit.poke` method to poke our `%journal` agent.
212+
213+```js
214+submitNew = (id, txt) => {
215+ window.urbit.poke({
216+ app: "journal",
217+ mark: "journal-action",
218+ json: { add: { id: id, txt: txt } },
219+ onSuccess: () => this.setState({ newDraft: {} }),
220+ onError: () => this.setErrorMsg("New entry rejected"),
221+ });
222+};
223+```
224+
225+The `Urbit.poke` method takes five arguments:
226+
227+- `app` is the agent to poke.
228+- `mark` is the mark of the data we're sending. We specify `"journal-action"`,
229+ so Eyre will use the `/mar/journal/action.hoon` mark we created to convert it
230+ to a `$action` structure with a `%journal-action` mark before it's delivered
231+ to our agent.
232+- `json` is the actual data we're poking our agent with. In this case it's the
233+ JSON form of the `%add` `$action`.
234+- `onSuccess` is a callback that fires if we get a positive ack in response. In
235+ this case we just clear the draft.
236+- `onError` is a callback that fires if we get a negative ack (nack) in
237+ response, meaning the poke failed. In this case we just set an error message
238+ to be displayed.
239+
240+`onSuccess` and `onError` are optional, but it's usually desirable to handle
241+these cases.
242+
243+The `delete` and `submitEdit` functions are similar to `submitNew`, but for the
244+`%del` and `%edit` actions rather than `%add`:
245+
246+![edit screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/edit.png)
247+
248+```js
249+submitEdit = (id, txt) => {
250+ if (txt !== null) {
251+ window.urbit.poke({
252+ app: "journal",
253+ mark: "journal-action",
254+ json: { edit: { id: id, txt: txt } },
255+ onError: () => this.setErrorMsg("Edit rejected"),
256+ });
257+ } else this.cancelEdit(id);
258+};
259+```
260+
261+![delete screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/delete.png)
262+
263+```js
264+delete = (id) => {
265+ window.urbit.poke({
266+ app: "journal",
267+ mark: "journal-action",
268+ json: {"del": {"id": id}},
269+ onError: ()=>this.setErrorMsg("Deletion rejected")
270+ })
271+ this.setState({rmModalShow: false, entryToDelete: null})
272+};
273+```
274+
275+Note that whether we're adding, editing or deleting entries, we update our state
276+when we receive the update back on the `/updates` subscription, not when we poke
277+our agent.
278+
279+## Search
280+
281+![search screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/search.png)
282+
283+When searching for entries between two dates, the `getSearch` function is
284+called, which uses the `Urbit.scry` method to scry for the results in a similar
285+fashion to [`getEntries`](#getting-entries), but using the
286+`/x/entries/between/[start]/[end]` endpoint.
287+
288+```js
289+getSearch = async () => {
290+ const { searchStart: ss, searchEnd: se, latestUpdate: lu } = this.state;
291+ if (lu !== null && ss !== null && se !== null) {
292+ let start = ss.getTime();
293+ let end = se.getTime();
294+ if (start < 0) start = 0;
295+ if (end < 0) end = 0;
296+ const path = `/entries/between/${start}/${end}`;
297+ window.urbit
298+ .scry({
299+ app: "journal",
300+ path: path,
301+ })
302+ .then(
303+ (result) => {
304+ this.setState({
305+ searchTime: result.time,
306+ searchStart: null,
307+ searchEnd: null,
308+ resultStart: ss,
309+ resultEnd: se,
310+ results: result.entries,
311+ });
312+ },
313+ (err) => {
314+ this.setErrorMsg("Search failed");
315+ }
316+ );
317+ } else {
318+ lu !== null && this.setErrorMsg("Searh failed");
319+ }
320+};
321+```
322+
323+## Error handling
324+
325+When the channel connection is interrupted, the `Urbit` object will begin trying to reconnect. On each attempt, it sets the connection `status` to `"try"`, as we specified for the `onRetry` callback. When this is set, a "reconnecting" message is displayed at the bottom of the screen:
326+
327+![reconnecting screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/reconnecting.png)
328+
329+If all three reconnection attempts fail, the `onError` callback is fired and we replace the "reconnecting" message with a "reconnect" button:
330+
331+![reconnect screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/reconnect.png)
332+
333+When clicked, the following function is called:
334+
335+```js
336+reconnect = () => {
337+ window.urbit.reset();
338+ const latest = this.state.latestUpdate;
339+ if (latest === null) {
340+ this.init();
341+ } else {
342+ this.getUpdates().then(
343+ (result) => {
344+ result.logs.map((e) => this.handleUpdate(e));
345+ this.subscribe();
346+ },
347+ (err) => {
348+ this.setErrorMsg("Connection failed");
349+ this.setState({ status: "err" });
350+ }
351+ );
352+ }
353+};
354+```
355+
356+Our `reconnect()` function first calls the `Urbit.reset` method. This closes the
357+channel connection, wipes event counts and subscriptions, and generates a new
358+channel ID. We could have tried reconnecting without resetting the connection,
359+but we don't know whether the channel still exists. We could time how long the
360+connection has been down and estimate whether it still exists, but it's easier
361+to just start fresh in this case.
362+
363+Since we've reset the channel, we don't know if we've missed any updates. Rather
364+than having to refresh our whole state, we can use the `getUpdates()` function
365+to get any missing update:
366+
367+```js
368+getUpdates = async () => {
369+ const { latestUpdate: latest } = this.state;
370+ const since = latest === null ? Date.now() : latest;
371+ const path = `/updates/since/${since}`;
372+ return window.urbit.scry({
373+ app: "journal",
374+ path: path,
375+ });
376+};
377+```
378+
379+This function uses the `Urbit.scry` method to scry the
380+`/x/updates/since/[since]` path, querying the update `$log` for entries more
381+recent than `latestUpdate`, which is always set to the last logged action we
382+received. The `getUpdates` function returns a Promise to the `reconnect`
383+function above which called it. The `reconnect` function handles it in a `.then`
384+expression, where the success case passes each update retrieved to the
385+[`handleUpdate`](#updates) function, updating our state.
386+
387+Lastly, as well as handling channel connection errors, we also handle errors
388+such as poke nacks or failed scries by printing error messages added to the
389+`error` map by the `setErrorMsg` function. You could of course handle nacks,
390+kicks, scry failures, etc differently than just printing an error, it depends on
391+the needs of your app.
392+
393+![search failed screenshot](https://media.urbit.org/docs/userspace/full-stack-guide/search-failed.png)
394+
395+## Resources
396+
397+- [HTTP API Guide](/docs/userspace/http-api-guide) - Reference documentation for
398+ `@urbit/http-api`.
399+
400+- [React app source
401+ code](https://github.com/urbit/docs-examples/tree/main/journal-app/ui) - The
402+ source code for the Journal app UI.
403+
404+- [`@urbit/http-api` source
405+ code](https://github.com/urbit/urbit/tree/master/pkg/npm/http-api) - The
406+ source code for the `@urbit/http-api` NPM package.
content/docs/userspace/full-stack/9-desk.mdnew file+226
@@ -0,0 +1,226 @@
1++++
2+title = "9. Desk and glob"
3+weight = 9
4+template = "doc.html"
5++++
6+
7+With our React app now complete, we can put together the final desk and publish
8+it.
9+
10+## Config files
11+
12+So far we've written the following files for the back-end:
13+
14+```
15+ourfiles
16+├── app
17+│ └── journal.hoon
18+├── lib
19+│ └── journal.hoon
20+├── mar
21+│ └── journal
22+│ ├── action.hoon
23+│ └── update.hoon
24+└── sur
25+ └── journal.hoon
26+```
27+
28+There's a handful of extra files we need in the root of our desk:
29+
30+- `desk.bill` - the list of agents that should be started when our app is
31+ installed.
32+- `sys.kelvin` - the kernel version our app is compatible with.
33+- `desk.docket-0` - configuration of our app tile, front-end glob and other
34+ metadata.
35+
36+We only have one agent to start, so `desk.bill` is very simple:
37+
38+```
39+:~ %journal
40+==
41+```
42+
43+Likewise, `sys.kelvin` just contains:
44+
45+```
46+[%zuse 419]
47+```
48+
49+The `desk.docket-0` file is slightly more complicated:
50+
51+```
52+:~
53+ title+'Journal'
54+ info+'Dear diary...'
55+ color+0xd9.b06d
56+ version+[0 1 0]
57+ website+'https://urbit.org'
58+ license+'MIT'
59+ base+'journal'
60+ glob-ames+[~zod 0v0]
61+==
62+```
63+
64+The fields are as follows:
65+
66+- `title` is the name of the app - this will be displayed on the tile and when
67+ people search for the app to install it.
68+- `info` is a brief description of the app.
69+- `color` - the RGB hex color of the tile.
70+- `version` - the version number of the app. The fields represent major, minor
71+ and patch version.
72+- `website` - a link to a website for the app. This would often be its Github repo.
73+- `license` - the license of for the app.
74+- `base` - the desk name of the app.
75+- `glob-ames` - the ship to retrieve the front-end files from, and the hash of
76+ those files. We've put `~zod` here but this would be the actual ship
77+ distributing the app when it's live on the network. The hash is `0v0`
78+ initially, but once we upload the front-end files it will be updated to the
79+ hash of those files automatically. Note that it's also possible to distribute
80+ front-end files from a separate web server. In that case, you'd use
81+ `glob-http` rather than `glob-ames`. The [Glob section of the distribution
82+ guide](/docs/userspace/dist/glob) covers this alternative approach in more
83+ detail.
84+
85+Our files should now look like this:
86+
87+```
88+ourfiles
89+├── app
90+│ └── journal.hoon
91+├── desk.bill
92+├── desk.docket-0
93+├── lib
94+│ └── journal.hoon
95+├── mar
96+│ └── journal
97+│ ├── action.hoon
98+│ └── update.hoon
99+├── sur
100+│ └── journal.hoon
101+└── sys.kelvin
102+```
103+
104+## New desk
105+
106+Next, we'll create a new `%journal` desk on our ship by forking an existing one.
107+Once created, we can mount it to the unix filesystem.
108+
109+In the dojo of a fake ship:
110+
111+```
112+> |merge %journal our %webterm
113+>=
114+> |mount %journal
115+>=
116+```
117+
118+Now we can browse to it in the unix terminal:
119+
120+```sh
121+cd ~/zod/journal
122+```
123+
124+Currently it has the same files as the `%webterm` desk, so we need to delete
125+those:
126+
127+```sh
128+rm -r .
129+```
130+
131+Apart from the kernel and standard library, desks need to be totally
132+self-contained, including all mark files and libraries necessary to build them.
133+For example, since our app contains a number of `.hoon` files, we need the
134+`hoon.hoon` mark, and its dependencies. The easiest way to ensure our desk has
135+everything it needs is to copy in the "dev" versions of the `%base` and
136+`%garden` desks. To do this, we first clone the Urbit git repository:
137+
138+```sh
139+git clone https://github.com/urbit/urbit.git urbit-git
140+```
141+
142+If we navigate to the `pkg` directory in the cloned repo:
143+
144+```sh
145+cd ~/urbit-git/pkg
146+```
147+
148+...we can combine the `base-dev` and `garden-dev` desks with the included
149+`symbolic-merge.sh` script:
150+
151+```sh
152+./symbolic-merge.sh base-dev journal
153+./symbolic-merge.sh garden-dev journal
154+```
155+
156+Now, we copy the contents of the new `journal` folder into our empty desk:
157+
158+```sh
159+cp -rL journal/* ~/zod/journal/
160+```
161+
162+Note we've used the `L` flag to resolve symbolic links, because the dev-desks
163+contain symlinks to files in the actual `arvo` and `garden` folders.
164+
165+We can copy across all of our own files too:
166+
167+```sh
168+cp -r ~/ourfiles/* ~/zod/journal/
169+```
170+
171+Finally, in the dojo, we can commit the whole lot:
172+
173+```
174+|commit %journal
175+```
176+
177+## Glob
178+
179+The next step is to build our front-end and upload the files to our ship. In the
180+`journal-ui` folder containing our React app, we can run:
181+
182+```sh
183+npm run build
184+```
185+
186+This will create a `build` directory containing the compiled front-end files. To
187+upload it to our ship, we need to first install the `%journal` desk. In the
188+dojo:
189+
190+```
191+|install our %journal
192+```
193+
194+Next, in the browser, we navigate to the `%docket` globulator at
195+`http://localhost:8080/docket/upload` (replacing localhost with the actual host):
196+
197+![globulator screenshot](https://m.tinnus-napbus.xyz/pub/globulator.png)
198+
199+We select our `journal` desk, then we hit `Choose file`, and select the whole
200+`build` directory which was created when we build our React app. Finally, we hit
201+`glob!` to upload it.
202+
203+If we now return to the homescreen of our ship, we'll see our tile displayed, and we can open our app by clicking on it:
204+
205+![tiles screenshot](https://m.tinnus-napbus.xyz/pub/tiles.png)
206+
207+## Publishing
208+
209+The last thing we need to do is publish our app, so other users can install it
210+from our ship. To do that, we just run the following command in the dojo:
211+
212+```
213+:treaty|publish %journal
214+```
215+
216+## Resources
217+
218+- [App publishing/distribution documentation](/docs/userspace/dist/dist) -
219+ Documentation covering third party desk composition, publishing and
220+ distribution.
221+
222+- [Glob documentation](/docs/userspace/dist/glob) - Comprehensive documentation
223+ of handling front-end files.
224+
225+- [Desk publishing guide](/docs/userspace/dist/guide) - A step-by-step guide to
226+ creating and publishing a desk.
content/docs/userspace/full-stack/_index.mdnew file+52
@@ -0,0 +1,52 @@
1++++
2+title = "Full-Stack Walkthrough"
3+weight = 10
4+sort_by = "weight"
5+template = "sections/docs/chapters.html"
6+insert_anchor_links = "right"
7++++
8+
9+#### [1. Introduction](/docs/userspace/gall-2/1-intro)
10+
11+An overview of the guide and table of contents.
12+
13+#### [2. Types](/docs/userspace/gall-2/2-types)
14+
15+Creating the `/sur` structure file for our `%journal` agent.
16+
17+#### [3. Agent](/docs/userspace/gall-2/3-agent)
18+
19+Creating the `%journal` agent itself.
20+
21+#### [4. JSON](/docs/userspace/gall-2/5-json)
22+
23+Writing a library to convert between our agent's marks and JSON. This lets our
24+React front-end poke our agent, and our agent send updates back to it.
25+
26+#### [5. Marks](/docs/userspace/gall-2/4-marks)
27+
28+Creating the mark files for the pokes our agent takes and updates it sends out.
29+
30+#### [6. Eyre](/docs/userspace/gall-2/6-eyre)
31+
32+A brief overview of how the webserver vane Eyre works.
33+
34+#### [7. React App Setup](/docs/userspace/gall-2/7-react-setup)
35+
36+Creating a new React app, installing the required packages, and setting up some
37+basic things for our front-end.
38+
39+#### [8. React App Logic](/docs/userspace/gall-2/8-http-api)
40+
41+Analysing the core logic of our React app, with particular focus on using
42+methods of the `Urbit` class from `@urbit/http-api` to communicate with our
43+agent.
44+
45+#### [9. Desk and Glob](/docs/userspace/gall-2/9-web-scries)
46+
47+Building and "globbing" our front-end, and putting together a desk for
48+distribution.
49+
50+#### [10. Summary](/docs/userspace/gall-2/10-final)
51+
52+Some final comments and additional resources.
content/docs/userspace/gall-guide/10-scry.md+5−4
@@ -154,10 +154,11 @@ can do something like:
154154
155155Previously we discussed custom `mark` files. Such mark files are most commonly
156156used when the data might be accessed through Eyre's HTTP API, and therefore
157−required JSON conversion methods. We'll cover such things in part two of the
158−Gall Guide, but note that if that's the case for your agent, you may wish to
159−also have your scry endpoints return data with your custom `mark` so it can
160−easily be converted to JSON when accessed from the web.
157+required JSON conversion methods. We cover such things separately in the
158+[Full-Stack Walkthrough](/docs/userspace/full-stack/1-intro), but note that if
159+that's the case for your agent, you may wish to also have your scry endpoints
160+return data with your custom `mark` so it can easily be converted to JSON when
161+accessed from the web.
161162
162163In some cases, typically with scry `path`s that contain wildcards like the `[%x %blah @ ~]` example above, your agent may not always be able to find the
163164requested data. In such cases, you can just produce a cell of `[~ ~]` for the
content/docs/userspace/gall-guide/12-next-steps.md+11−3
@@ -9,9 +9,11 @@ to start writing your own agent.
99
1010The things haven't touched on yet are front-end development and integration,
1111Eyre's HTTP API for communicating with agents from the web, and dealing with
12−JSON data. A second part to the Gall Guide is coming soon, and will focus
13−exclusively on those aspects. In the mean time, you can refer to the following
14−documents for help writing a web front-end for your app:
12+JSON data. The [Full-stack Walkthrough](/docs/userspace/full-stack/1-intro)
13+covers these aspects of Urbit app development, and it also puts into practice
14+many of the concepts we've discussed in this guide, so you might like to work
15+through that next. In addition to that walkthrough, you can refer to the
16+following documents for help writing a web front-end for your app:
1517
1618- [Eyre's external API reference](/docs/arvo/eyre/external-api-ref) - This
1719 explains Eyre's HTTP API, through which a browser or other HTTP client can
@@ -27,6 +29,12 @@ documents for help writing a web front-end for your app:
2729- [The software distribution guide](/docs/userspace/dist/dist) - This covers
2830 everything you need to know to distribute apps to other ships. It includes
2931 details of bundling a web front-end and serving it to the user in the browser.
32+- [The HTTP API guide](/docs/userspace/http-api-guide) - This is a reference
33+ and guide to using the `@urbit/http-api` NPM module.
34+- [The Sail guide](/docs/hoon/guides/sail) - Sail is a domain-specific language
35+ for composing XML structure in Hoon. It can be used to compose front-ends for
36+ Urbit apps directly in agents, as an alternative approach to having a
37+ separate Javascript app.
3038
3139In addition to these documents about creating a web-based user interface for
3240your app, there are some other guides you might like to have a look at:
content/docs/userspace/gall-guide/7-sur-and-marks.md+9−9
@@ -282,12 +282,13 @@ arm in the previous example could test for it instead of `%noun` like so:
282282Note how `%todo-action` will be resolved to `/mar/todo/action.hoon` - the hyphen
283283will be interpreted as `/` if there's not already a `/mar/todo-action.hoon`.
284284
285−This simple mark file isn't all that useful. Typically, you'd add `json` arms to
286−`grow` and `grab`, which allow your data to be converted to and from JSON, and
287−therefore allow your agent to communicate with a web front-end. Front-ends,
288−JSON, and Eyre's APIs which facilitate such communications will be covered in a
289−forthcoming part two of the Gall guide. For now though, it's still useful to use
290−marks and understand how they work.
285+This simple mark file isn't all that useful. Typically, you'd add `json` arms
286+to `grow` and `grab`, which allow your data to be converted to and from JSON,
287+and therefore allow your agent to communicate with a web front-end. Front-ends,
288+JSON, and Eyre's APIs which facilitate such communications will be covered in
289+the separate [Full-Stack Walkthrough](/docs/userspace/full-stack/1-intro),
290+which you might like to work through after completing this guide. For now
291+though, it's still useful to use marks and understand how they work.
291292
292293One further note on marks - while data from remote ships must have a matching
293294mark file in `/mar`, it's possible to exchange data between local agents with
@@ -384,8 +385,7 @@ Mark files:
384385- Messages passed between agents on a local ship don't necessarily need mark
385386 files in `/mar`.
386387- Mark files are most commonly used for converting an agent's native types to
387− JSON, in order to interact with a web front-end (which we'll cover in part two
388− of the Gall guide).
388+ JSON, in order to interact with a web front-end.
389389
390390Permissions:
391391
@@ -404,5 +404,5 @@ Permissions:
404404- Have a quick look at the [tisket
405405 documentation](/docs/hoon/reference/rune/tis#-tisket).
406406- Try writing a mark file for the `update:todo` type, in a similar fashion to
407− the `action:todo` one in the [mark file section](#mark-file). You can compare
407+ the `action:todo` one in the [mark file section](#mark-files). You can compare
408408 yours to the one we'll use in the next lesson.
content/docs/userspace/threads/gall/poke-thread.md+1−1
@@ -29,7 +29,7 @@ Here's a modified agent that pokes our thread. I've replaced some off the previo
2929 (pair term term)
3030 =/ tid `@ta`(cat 3 'thread_' (scot %uv (sham eny.bowl)))
3131 =/ ta-now `@ta`(scot %da now.bowl)
32− =/ start-args [~ `tid byk.bowl p.q.vase !>(q.q.vase)]
32+ =/ start-args [~ `tid byk.bowl(q da+now.bowl) p.q.vase !>(q.q.vase)]
3333 :_ this
3434 :~
3535 [%pass /thread/[ta-now] %agent [our.bowl %spider] %watch /thread-result/[tid]]
content/docs/userspace/threads/gall/start-thread.md+2−2
@@ -29,7 +29,7 @@ Here's an example of a barebones gall agent that just starts a thread:
2929 (pair term term)
3030 =/ tid `@ta`(cat 3 'thread_' (scot %uv (sham eny.bowl)))
3131 =/ ta-now `@ta`(scot %da now.bowl)
32− =/ start-args [~ `tid byk.bowl p.q.vase !>(q.q.vase)]
32+ =/ start-args [~ `tid byk.bowl(q da+now.bowl) p.q.vase !>(q.q.vase)]
3333 :_ this
3434 :~
3535 [%pass /thread/[ta-now] %agent [our.bowl %spider] %poke %spider-start !>(start-args)]
@@ -116,7 +116,7 @@ We can ignore the input logic, here's the important part:
116116```hoon
117117=/ tid `@ta`(cat 3 'thread_' (scot %uv (sham eny.bowl)))
118118=/ ta-now `@ta`(scot %da now.bowl)
119−=/ start-args [~ `tid byk.bowl p.q.vase !>(q.q.vase)]
119+=/ start-args [~ `tid byk.bowl(q da+now.bowl) p.q.vase !>(q.q.vase)]
120120:_ this
121121:~
122122 [%pass /thread/[ta-now] %agent [our.bowl %spider] %poke %spider-start !>(start-args)]
content/docs/userspace/threads/gall/stop-thread.md+1−1
@@ -29,7 +29,7 @@ Here we've added one last card to `on-poke` to stop the thread and a little extr
2929 (pair term term)
3030 =/ tid `@ta`(cat 3 'thread_' (scot %uv (sham eny.bowl)))
3131 =/ ta-now `@ta`(scot %da now.bowl)
32− =/ start-args [~ `tid byk.bowl p.q.vase !>(q.q.vase)]
32+ =/ start-args [~ `tid byk.bowl(q da+now.bowl) p.q.vase !>(q.q.vase)]
3333 :_ this
3434 :~
3535 [%pass /thread/[ta-now] %agent [our.bowl %spider] %watch /thread-result/[tid]]
content/docs/userspace/threads/gall/take-facts.md+1−1
@@ -31,7 +31,7 @@ Here we've added another card to subscribe for any facts sent by the thread and
3131 (pair term term)
3232 =/ tid `@ta`(cat 3 'thread_' (scot %uv (sham eny.bowl)))
3333 =/ ta-now `@ta`(scot %da now.bowl)
34− =/ start-args [~ `tid byk.bowl p.q.vase !>(q.q.vase)]
34+ =/ start-args [~ `tid byk.bowl(q da+now.bowl) p.q.vase !>(q.q.vase)]
3535 :_ this
3636 :~
3737 [%pass /thread/[ta-now] %agent [our.bowl %spider] %watch /thread-result/[tid]]
content/docs/userspace/threads/gall/take-result.md+1−1
@@ -29,7 +29,7 @@ Here we've added an extra card to subscribe for the result and a couple of lines
2929 (pair term term)
3030 =/ tid `@ta`(cat 3 'thread_' (scot %uv (sham eny.bowl)))
3131 =/ ta-now `@ta`(scot %da now.bowl)
32− =/ start-args [~ `tid byk.bowl p.q.vase !>(q.q.vase)]
32+ =/ start-args [~ `tid byk.bowl(q da+now.bowl) p.q.vase !>(q.q.vase)]
3333 :_ this
3434 :~
3535 [%pass /thread/[ta-now] %agent [our.bowl %spider] %watch /thread-result/[tid]]
content/events/2022-04-21-developer-call-uqbar-and-urbits-first-blockchain.mdnew file+24
@@ -0,0 +1,24 @@
1++++
2+title = "Developer Call: Uqbar and Urbit’s first Blockchain"
3+starts = "2022-04-21T09:00:00"
4+ends = "2022-04-21T10:00:00"
5+timezone = "America/Los_Angeles"
6+location = "Online Event"
7+image = ""
8+registration_url = "https://www.meetup.com/urbit-sf/events/285206697/"
9+description = ""
10+dark = false
11+youtube = "XZD0ob1XTKE"
12+
13+[[hosts]]
14+name = "Neal Davis"
15+patp = "~lagrev-nocfep"
16+
17+[[guests]]
18+patp = "~hodzod-walrus"
19+
20++++
21+
22+This time Uqbar’s ~hodzod-walrus will join Neal Davis to discuss Uqbar’s latest developments in building what will be the first Blockchain on Urbit. ~hodzod-walrus will go into Uqbar’s basic outline and how they will make their plans reality. Why their ZK interpreter, Zock, is the perfect form for smart contract integration on Urbit. How they will execute parallel smart contract on Hoon and the endeavours with uHoon.
23+
24+~hodzod-walrus is a programmer working on Uqbar. Previously he built Pokur as an Urbit Foundation apprenticeship grant. He’s quickly become a Hoon evangelist and hopes to showcase its strengths in the world of smart contracts.
content/events/2022-04-26-urbit-foundation-meetup-kingsnight-amsterdam.mdnew file+24
@@ -0,0 +1,24 @@
1++++
2+title = "Urbit Foundation Meetup - Kingsnight"
3+starts = "2022-04-26T18:00:00"
4+ends = "2022-04-26T21:30:00"
5+timezone = "Europe/Amsterdam"
6+location = "Hortus Botanicus, Plantage Middenlaan 2a, 1018 DD Amsterdam, Netherlands"
7+image = "https://i.imgur.com/g75pj4M.png"
8+registration_url = "https://www.meetup.com/urbit-sf/events/285233312/"
9+description = "The Martian way of P2P social computing"
10+dark = false
11+
12+
13+[[hosts]]
14+name = "Thomas"
15+patp = "~dalweb-donfun"
16+
17+[[hosts]]
18+patp = "~palfun-foslup"
19+
20++++
21+
22+On the 26th of April the Urbit Foundation is coming to Kingsnight in Amsterdam, we will host you all in one of the oldest Botanical Gardens in the world. A short series of talks will introduce you to the technical background of Urbit, the vision on social computing by ~palfun-foslup from Tlon and an introduction to the first Urbit blockchain by Uqbar.
23+
24+Bring your orange Tee and we will sort the drinks and lots of cool people, come join us from 18:00 at the Hortus Botanicus in Amsterdam.
content/grants/app-engram.mdnew file+117
@@ -0,0 +1,117 @@
1++++
2+title = "Engram: a collaborative knowledge managment app"
3+date = "2022-03-25"
4+
5+[taxonomies]
6+grant_type = [ "Bounty" ]
7+grant_category = [ "App Dev" ]
8+
9+[extra]
10+image = ""
11+description = "Engram is a knowledge management application that allows individuals or teams to build rich note documents."
12+reward = "$70,000 in WSTR"
13+assignee = ""
14+id = "B0155"
15+champion = "Holium"
16+completed = false
17+work_request_link = "https://airtable.com/shr4qt9t9kz7RaOIa?prefill_Grant+ID=B0155&prefill_Grant+Name=Engram"
18++++
19+
20+Engram is a knowledge management application that allows individuals or teams to build rich note documents.
21+
22+## Rationale
23+
24+Right now, the only way to collaborate on documents in Urbit is in the notebook channel via comments. There are no features around knowledge management, tagging, history, or any feature required for collaborating on documents with others. The goal of this project is to provide a robust solution for groups or individuals to collaborate on documents.
25+
26+## Overview
27+
28+For V1, Engram will focus on enabling rich document editing for an individual experience, but multiple collaborators. Different people should be able to modify the documents based on permission.
29+
30+There are several features and agents that may need to be created to support this.
31+
32+- Document editor
33+- Git-like revision history
34+- Inline comments
35+- Sentence parser for storing each sentence as a node in a graph for easy referencing, commenting, and more future features.
36+- Document permissions
37+
38+## User stories
39+
40+As a single user, I would like to:
41+
42+- See a list of documents for my own ship context.
43+- Be able to edit a document myself.
44+- Have others invited edit the document
45+- See a history of changes on the document.
46+
47+As a group member, I would like to
48+
49+- See notes associated with my group.
50+- Be able to comment on sentences at the current revision.
51+- See a log of changes via a popout panel
52+- Be able to resolve conflicting revisions.
53+
54+As a document editor, I would like to:
55+
56+- Have the current document state saved in local storage if I haven't clicked save and accidentally close the tab.
57+- Be able to edit in readme mode
58+- Be able to edit in preview mode
59+- Have a hoverable style block appear when editing
60+- Add collaborators to the document with an edit permission or view permission.
61+
62+As a document viewer, I would like to see
63+
64+- A rich document with formatting.
65+- A list of authors and editors.
66+- View comments on the document .
67+- View phrase comments on sentence hover.
68+- Have custom embeds in the document:
69+ - code block
70+ - link
71+ - video
72+ - image
73+
74+## Technical requirements
75+
76+The most complex part of this is the revision log. It should be easy to revert to a certain point in time. There will also need to be a sentence parser that stores each sentence in a store with associated comments and other metadata. Each sentence or collection of sentences should be able to be referenced by other documents.
77+
78+Holium will work with the developer(s) to provide frontend help.
79+
80+## Milestone 1 - Single user documents - $30,000 in WSTR
81+
82+The first milestone is to release a version of the app that allows for single user documents, meaning there is a document owner and only they can edit it. The app should include the rich document editor features without collaboration.
83+
84+The app should be released.
85+
86+## Milestone 2 - Revision history, comments, and referencing - $15,000 in WSTR
87+
88+This second milestone is building out the revision history log to allow for undo, redo, and time traveling through revision snapshots. This will serve as the foundation of the third milestone.
89+
90+There should be a revision history log in the UI that allows users to go back to previous snapshots as well as the ability to leave comments on the document or on sentences.
91+
92+## Milestone 3 - Collaboration - $25,000 in WSTR
93+
94+Allows multiple editors to modify the document and resolves the differences between the documents. There is not a need for live co-editing in this milestone as that is reliant on WebRTC or another streaming solution.
95+
96+You should have the ability to add new users to the document, manage those users permissions, and remove the users from the collaborator list.
97+
98+## Future work
99+There is the potential for several future releases enabling more collaboration features:
100+
101+- Public pages
102+- Live co-editing (WebRTC or Urbit native)
103+- Group-to-group document sharing (if groups enable)
104+- Inter-document referencing (hypertext-like)
105+
106+## Requirements
107+- At least three years of professional programming experience
108+- Experience with full-stack development, including JS and some server-side language
109+- Demonstrable experience with writing Gall agents
110+- Coordinate with Holium on designs, frontend, and agents
111+
112+## Schedule
113+This project has many pieces and would likely require a team size of two or more. There would need to be a significant amount of frontend work, which Holium will help build or facilitate using our design system. V1 deadline is 6 months from start.
114+
115+## Reward
116+$70,000 in WSTR (team of two)
117+
content/grants/apprenticeship-studio-email.md+1−1
@@ -11,7 +11,7 @@ image = ""
1111description = "Build out email capabilities of ~tirrel's Studio app"
1212reward = "$8000 in WSTR"
1313mentor = "~novlud-padtyv"
14−assignee = ""
14+assignee = "~rivmud-fabwen"
1515completed = false
1616work_request_link = "https://airtable.com/shr4qt9t9kz7RaOIa?prefill_Grant+ID=A0139&prefill_Grant+Name=~tirrel%20studio%20email%20capabilities"
1717+++
content/grants/digital-editor-MROB.md+1−1
@@ -10,7 +10,7 @@ grant_category = ["Other"]
1010image = ""
1111description = "Digital Editor for the Mars Review of Books"
1212reward = "$4,500 in WSTR"
13−assignee = ""
13+assignee = "~randem-wishes"
1414champion = "~librex-dozryc"
1515id = "B0140"
1616completed = false
content/grants/native-planet-2.mdnew file+89
@@ -0,0 +1,89 @@
1++++
2+title = "Native Planet Phase 2"
3+date = "2022-04-07"
4+
5+[taxonomies]
6+grant_type = [ "Proposal" ]
7+grant_category = [ "Other" ]
8+
9+[extra]
10+image = ""
11+description = "Native Planet - A domestic hosting solution for Urbit"
12+reward = "$50,000 in WSTR"
13+assignee = "~dalhec-banler"
14+champion = "~poldec-tonteg"
15+completed = false
16+canceled = false
17++++
18+
19+## Native Planet Phase 2 - Software Stack Development + Integration, Service development, Production model
20+
21+## Overview
22+
23+The purest Urbit experience is hosting your own ship at your home, on your desk at work, or wherever you want to keep a small, well-designed piece of hardware. This "box" will include a small but powerful chipset that will be custom built for Urbit's unique needs, featuring fast read/write speeds, a durable HD to handle re-writes, RAID or remote encrypted backup, and easily expandable storage. Ideally, the components should be available as an open source hardware project. The Native Planet is a powerful, simple and hackable implementation of these features. You will be able to purchase the Native Planet as a hacker kit, or as a fully assembled piece of high-end hardware that offers a plug and play solution for less tech-savvy users. Building hardware for Urbit is a bit of a moving target this early in the game, but building something that is modular, customizable, hackable, and powerful enough for now, and the foreseeable future, will both aid in development of future hardware and satisfy the needs of current users.
24+
25+Phase two of the project will focus on UX and custom integration of the Home-Urbit software with the Native Planet Hardware. It is worth noting that the Home-Urbit software project is currently being developed by Odysseas Lamtzidis under a separate grant, and work will continue on Home-Urbit outside of its Native Planet integration. That said, integration of these two projects is key to building a true plug-and-play product for the NativePlanet end-user. Integrations will require front-end development, hardware specific low level programming, UX design, and documentation for future development and initial customer service. Phase two will grow the Native Planet team and add features to the project, such as encrypted backup, hosted web endpoints , and native S3 buckets.
26+
27+## Home-Urbit Integration + Front End Development
28+
29+The final version of the Native Planet will extend the proof-of-concept prototype to a consumer-ready, first class Urbit device. Thoughtful design, a pre-configured software stack, and powerful hardware will make the experience of running a ship akin to operating a simple home appliance. A non-technical user will be up and running within minutes of downloading their keyfile, with all of the accouterments available to technical enthusiasts and none of the maintenance burden.
30+
31+## Milestones
32+
33+### Phase Two - Milestone 1 - Anchor server and Onboard Screen UI
34+
35+Expected Completion: 30 May 2022
36+
37+Payment: $25,000 WSTR
38+
39+Anchor Server
40+
41+- Hassle-free provisioning of a publicly-accessible proxy for Ames and Landscape
42+
43+- Transparent NAT traversal via automated WireGuard configuration
44+
45+- Automatically configured domain and TLS for HTTPS ship access
46+
47+- Automated provisioning of public S3 bucket accessible via hosted subdomain
48+
49+- Ships with commercial version, open source/self-hosted version of server software available
50+
51+Onboard screen UI
52+
53+- Simple informational screen on the chassis
54+
55+- Display access URL information & access code on initial boot
56+
57+- Display ship information and hardware diagnostics post-boot
58+
59+- Idle screen ‘planet pass’ graphic a la Bridge (contingent on HDMI screen)
60+
61+- User interaction via pleasant analog dial knob
62+
63+### Phase Two - Milestone 2 - Production
64+
65+Expected Completion: 30 July, 2022
66+
67+Payment: $25,000 WSTR
68+
69+Production UX
70+
71+- True plug-and-play – copy keyfile onto a USB drive, insert, and turn on
72+
73+- Zero-configuration setup for public web access
74+
75+- Simple web management UI for device & underlying OS
76+
77+- Automated, secure remote backup
78+
79+- Support multiple ships on one device
80+
81+Production hardware
82+
83+- Sourcing and assembling the consumer model of the Native Planet
84+
85+- Open source design and configuration
86+
87+- Optional RAID1 configuration for auxiliary Sata SSD drives
88+
89+- 3D printer STL models for case components
content/grants/native-urbit.md+1−1
@@ -11,7 +11,7 @@ image = ""
1111description = "Native Planet - A domestic hosting solution for Urbit"
1212reward = "5 stars"
1313assignee = "~dalhec-banler"
14−completed = false
14+completed = true
1515canceled = false
1616+++
1717
pages/index.js+34−1
@@ -359,7 +359,40 @@ export async function getStaticProps() {
359359
360360 const events = getAllEvents(eventKeys, "events");
361361
362+ const now = DateTime.now();
363+
364+ const pastEvents = events.filter((event) => {
365+ const ends = generateRealtimeDate(event.ends);
366+ return ends < now;
367+ });
368+
369+ const futureEvents = events
370+ .filter((event) => {
371+ const starts = generateRealtimeDate(event.starts);
372+ return starts > now;
373+ })
374+ .sort((a, b) => {
375+ const aStarts = generateRealtimeDate(a.starts).ts;
376+ const bStarts = generateRealtimeDate(b.starts).ts;
377+ if (aStarts > bStarts) {
378+ return 1;
379+ } else if (aStarts === bStarts) {
380+ return 0;
381+ }
382+ return -1;
383+ });
384+
385+ const happeningNow = events.filter((event) => {
386+ const starts = generateRealtimeDate(event.starts);
387+ const ends = generateRealtimeDate(event.ends);
388+ return starts > DateTime.now() && ends < now;
389+ });
390+
362391 return {
363− props: { posts, events, openGrantsCount },
392+ props: {
393+ posts,
394+ openGrantsCount,
395+ events: happeningNow.concat(futureEvents).concat(pastEvents),
396+ },
364397 };
365398}