bbd4189 · urbit.org · git.iko.soy

git.iko.soy/urbit.org

on master help sign in

add a guide to strings in hoon

bbd4189tinnus-napbus5 years ago

content/docs/hoon/guides/_index.md+2

compare with master

unifiedsplit

@@ -11,3 +11,5 @@ insert_anchor_links = "right"

11## [CLI Apps](/docs/hoon/guides/cli-tutorial)

13## [Parsing](/docs/hoon/guides/parsing)

14+

15+## [Strings](/docs/hoon/guides/strings)

content/docs/hoon/guides/strings.mdnew file+502

@@ -0,0 +1,502 @@

1++++

2+title = "Strings"

3+weight = 4

4+template = "doc.html"

5++++

7+This document discusses hoon's two main string types: `cord`s (as well as its

8+subsets `knot` and `term`) and `tape`s. Cords are strings encoded in atoms, and

9+tapes are strings encoded as lists of single characters. The focus of this

10+document is on their basic properties, syntax and the most common text-related

11+functions you'll regularly encounter. In particular, it discusses conversions

12+and the encoding/decoding of atom auras in strings.

13+

14+Hoon has a system for writing more elaborate functional parsers, but that is not

15+touched on here. Instead, see the [Parsing](/docs/hoon/guides/parsing) guide.

16+Hoon also has a type for UTF-32 strings, but those are rarely used and not

17+discussed in this document.

18+

19+There are a good deal more text manipulation functions than are discussed here.

20+See the [Further Reading](#further-reading) section for details.

21+

22+## text atoms

23+

24+### `cord`

25+

26+A [`cord`](/docs/hoon/reference/stdlib/2q#cord) has an aura of

27+`@t`. It denotes UTF-8 text encoded in an atom, little-endian. That is, the

28+first character in the text is the least-significant byte. A cord may contain

29+any UTF-8 characters, there are no restrictions.

30+

31+The `hoon` syntax for a cord is some text wrapped in single-quotes like:

32+

33+```hoon

34+'This is a cord!'

35+```

36+

37+single-quotes and backslashes must be escaped with a backslash like:

38+

39+```hoon

40+'\'quotes\' \\backslashes\\'

41+```

42+

43+Characters can also be entered as hex, they just have to be escaped by a

44+backslash. For example, `'\21\21\21'` will render as `'!!!'`. This is useful for

45+entering special characters such as line breaks like `'foo\0abar'`.

46+

47+Cords divided over multiple lines are allowed. There are two ways to do this.

48+The first is to start and end with three single-quotes like:

49+

50+```hoon

51+'''

52+foo

53+bar

54+baz

55+'''

56+```

57+

58+The line endings will be encoded Unix-style as line feed characters like:

59+

60+```hoon

61+'foo\0abar\0abaz'

62+```

63+

64+The second is to begin with a single-quote like usual, then break the line by

65+ending it with a backslash and start the next line with a forward-slash like:

66+

67+```hoon

68+'foo\

69+/bar\

70+/baz'

71+```

72+

73+This will be parsed to:

74+

75+```hoon

76+'foobarbaz'

77+```

78+

79+### `knot`

80+

81+A [`knot`](/docs/hoon/reference/stdlib/2q#knot) has an aura of

82+`@ta`, and is a subset of a [`cord`](#cord). It allows lower-case letters,

83+numbers, and four special characters: Hyphen, tilde, underscore and period. Its

84+restricted set of characters is intended to be URL-safe.

85+

86+The `hoon` syntax for a knot is a string containing any of the aforementioned

87+characters prepended with `~.` like:

88+

89+```hoon

90+~.abc-123.def_456~ghi

91+```

92+

93+### `term`

94+

95+A [`term`](/docs/hoon/reference/stdlib/2q#term) has an aura of

96+`@tas`, and is a subset of a [`knot`](#knot). It only allows lower-case letters,

97+numbers, and hyphens. Additionally, the first character cannot be a hyphen. This

98+is a very restricted text atom, and is intended for naming data structures and

99+the like.

100+

101+The `hoon` syntax for a term is a string conforming to the prior description,

102+prepended with a `%` like:

103+

104+```hoon

105+%foo-123

106+```

107+

108+#### A note about `term` type inference

109+

110+There is actually an even more restricted text atom form with the same `%foo`

111+syntax as a term, where the type of the text is the text itself. For example, in

112+the dojo:

113+

114+```

115+> `%foo`%foo

116+%foo

117+```

118+

119+The hoon parser will, by default, infer the type of `%foo`-style syntax this

120+way. If we try with the dojo type printer:

121+

122+```

123+> ? %foo

124+ %foo

125+%foo

126+```

127+

128+This type-as-itself is used for many things, such as unions like:

129+

130+```hoon

131+?(%foo %bar %bas)

132+```

133+

134+In order to give `%foo` the more generic `@tas` aura, it must be explicitly

135+upcast like:

136+

137+```

138+> ? `@tas`%foo

139+ @tas

140+%foo

141+```

142+

143+This is something to be wary of. For example, if you wanted to form a `(set @tas)` you might think to do:

144+

145+```hoon

146+(silt (limo ~[%foo %bar %baz]))

147+```

148+

149+However, this will actually form a set of the union `?(%foo %bar %baz)` due to

150+the specificity of type inference:

151+

152+```

153+> ? (silt (limo ~[%foo %bar %baz]))

154+ ?(%~ [?(n=%bar n=%baz n=%foo) l=nlr(?(%bar %baz %foo)) r=nlr(?(%bar %baz %foo))])

155+[n=%baz l={%bar} r={%foo}]

156+```

157+

158+One further note about the type-as-itself form: Ocassionally you may wish to

159+form a union of strings which contain characters disallowed in `term`s. To get

160+around this, you can enclose the text after the `%` with single-quotes like

161+`%'HELLO!'`.

162+

163+### Aura type validity

164+

165+The hoon parser will balk at `cord`s, `knot`s and `term`s containing invalid

166+characters. However, because they're merely auras, any atom can be cast to them.

167+When cast (or clammed), they will **not** be validated in terms of whether the

168+characters are allowed in the specified aura.

169+

170+For example, you can do this:

171+

172+```

173+> `@tas`'!%* $@&'

174+%!%* $@&

175+```

176+

177+This means you cannot rely on mere aura-casting if you need the text to conform

178+to the specified aura's restrictions. Instead, there are a couple of function in

179+the standard library to check text aura validity:

180+[`+sane`](/docs/hoon/reference/stdlib/4b#sane) and

181+[`+sand`](/docs/hoon/reference/stdlib/4b#sane).

182+

183+The `+sane` function takes an argument of either `%ta` or `%tas` to validate

184+`@ta` and `@tas` respectively (you can technically give it `%t` for `@t` too but

185+there's no real point). It will return `%.y` if the given atom is valid for the

186+given aura, and `%.n` if it isn't. For example:

187+

188+```

189+> ((sane %tas) 'foo')

190+%.y

191+> ((sane %tas) 'foo!')

192+%.n

193+```

194+

195+The `+sand` function does the same thing, but rather than returning a `?` it

196+returns a `unit` of the given atom, or `~` if validation failed. For example:

197+

198+```

199+> `(unit @tas)`((sand %tas) 'foo')

200+[~ %foo]

201+> `(unit @tas)`((sand %tas) 'foo!')

202+~

203+```

204+

205+## `tape`

206+

207+A [`tape`](/docs/hoon/reference/stdlib/2q#tape) is the other

208+main string type in hoon. Rather than a single atom, it's instead a list of

209+individual `@tD` characters (the `D` specifies a bit-length of 8, see the

210+[Auras](/docs/hoon/reference/auras#bitwidth) documentation for

211+details). The head of the list is the first character in the string.

212+

213+The `hoon` syntax for a tape is some text wrapped in double-quotes like:

214+

215+```hoon

216+"This is a tape!"

217+```

218+

219+Double-quotes, backslashes and left-braces must be escaped by a backslash

220+character:

221+

222+```hoon

223+"\"double-quotes\" \\backslash\\ left-brace:\{"

224+```

225+

226+Like with `cord`s, characters can also be entered as hex escaped by a backslash

227+so `"\21\21\21"` renders as `"!!!"`.

228+

229+Tapes divided over multiple lines are allowed. Unlike [`cord`](#cord)s, there is

230+only one way to do this, which is by starting and ending with three

231+double-quotes like:

232+

233+```hoon

234+"""

235+foo

236+bar

237+baz

238+"""

239+```

240+

241+The line endings will be encoded Unix-style as line feed characters like:

242+

243+```hoon

244+"foo\0abar\0abaz"

245+```

246+

247+As mentioned earlier, tapes are lists of single characters:

248+

249+```

250+> `tape`~['f' 'o' 'o']

251+"foo"

252+```

253+

254+This means they can be manipulated with ordinary list functions:

255+

256+```

257+> `tape`(turn "foobar" succ)

258+"gppcbs"

259+```

260+

261+### Interpolation

262+

263+Tapes, unlike cords, allow string interpolation. Arbitrary `hoon` may be

264+embedded in the tape syntax and its product will be included in the resulting

265+tape. There are two ways to do it:

266+

267+#### Manual

268+

269+In the first case, the code to be evaluated is enclosed in braces. The type of

270+the product of the code must itself be a tape. For example, if the `@p` of our

271+ship is stored in `our`, simply doing `"{our}"` will fail because its type will

272+be `@p` rather than `tape`. Instead, we must explicitly use the

273+[`+scow`](/docs/hoon/reference/stdlib/4m#scow) function to

274+render `our` as a tape:

275+

276+```

277+> "{(scow %p our)}"

278+"~zod"

279+```

280+

281+Another example:

282+

283+```

284+> "[{(scow %p our)} {(scow %da now)}]"

285+"[~zod ~2021.10.3..08.59.10..2335]"

286+```

287+

288+#### Automatic

289+

290+Rather than having to manually render data as a `tape`, angle brackets _inside_

291+the braces tell the interpreter to automatically pretty-print the product of the

292+expression as a tape. This way we needn't use functions like `+scow` and can

293+just reference things like `our` directly:

294+

295+```

296+> "{<our>}"

297+~zod

298+```

299+

300+Another example:

301+

302+```

303+> "{<(add 1 2)>}"

304+"3"

305+```

306+

307+And another:

308+

309+```

310+> "{<our now>}"

311+"[~zod ~2021.10.3..09.01.14..1654]"

312+```

313+

314+## Conversions

315+

316+Tapes can easily be converted to cords and vice versa. There are two stdlib

317+functions for this purpose: [`+crip`](/docs/hoon/reference/stdlib/4b#crip) and

318+[`+trip`](/docs/hoon/reference/stdlib/4b#trip). The former converts a `tape` to

319+a `cord` and the latter does the opposite. For example:

320+

321+```

322+> (crip "foobar")

323+'foobar'

324+> (trip 'foobar')

325+"foobar"

326+```

327+

328+Knots and terms can also be converted to tapes with `+trip`:

329+

330+```

331+> (trip %foobar)

332+"foobar"

333+> (trip ~.foobar)

334+"foobar"

335+```

336+

337+Likewise, the output of `+crip` can be cast to a knot or term:

338+

339+```

340+> `@tas`(crip "foobar")

341+%foobar

342+> `@ta`(crip "foobar")

343+~.foobar

344+> `@tas`(need ((sand %tas) (crip "foobar")))

345+%foobar

346+```

347+

348+## Encoding in text

349+

350+It's common to encode atoms in cords or knots, particularly when constructing a

351+[scry](/docs/arvo/concepts/scry) [`path`](/docs/hoon/reference/stdlib/2q#path)

352+or just a `path` in general. There are two main functions for this purpose:

353+[`+scot`](/docs/hoon/reference/stdlib/4m#scot) and

354+[`+scow`](/docs/hoon/reference/stdlib/4m#scow). The former produces a `knot`,

355+and the latter produces a `tape`. Additionally, there are two more functions for

356+encoding `path`s in cords and tapes respectively:

357+[`+spat`](/docs/hoon/reference/stdlib/4m#spat) and

358+[`+spud`](/docs/hoon/reference/stdlib/4m#spud).

359+

360+### `+scot` and `+spat`

361+

362+`+scot` encodes atoms of various auras in a `knot` (or `cord`/`term` with

363+casting). It takes two arguments: the aura in a `@tas` and the atom to be

364+encoded. For example:

365+

366+```

367+> (scot %p ~zod)

368+~.~zod

369+

370+> (scot %da now)

371+~.~2021.10.4..07.35.54..6d41

372+

373+> (scot %ux 0xaa.bbbb)

374+~.0xaa.bbbb

375+```

376+

377+Note the aura of the atom needn't actually match the specified aura:

378+

379+```

380+> (scot %ud ~zod)

381+~.0

382+```

383+

384+Hoon can of course be evaluated in its arguments as well:

385+

386+```

387+> (scot %ud (add 1 1))

388+~.2

389+```

390+

391+You'll most commonly see this used in constructing a `path` like:

392+

393+```

394+> /(scot %p our)/garden/(scot %da now)/foo/(scot %ud 123.456)

395+[~.~zod %garden ~.~2021.10.4..07.43.14..a556 %foo ~.123.456 ~]

396+

397+> `path`/(scot %p our)/garden/(scot %da now)/foo/(scot %ud 123.456)

398+/~zod/garden/~2021.10.4..07.43.23..9a0f/foo/123.456

399+```

400+

401+`+spat` simply encodes a `path` in a cord like:

402+

403+```

404+> (spat /foo/bar/baz)

405+'/foo/bar/baz'

406+```

407+

408+### `+scow` and `+spud`

409+

410+`+scow` is the same as [`+scot`](#scot-and-spat) except it produces a tape

411+rather than a knot. For example:

412+

413+```

414+> (scow %p ~zod)

415+"~zod"

416+

417+> (scow %da now)

418+"~2021.10.4..07.45.25..b720"

419+

420+> (scow %ux 0xaa.bbbb)

421+"0xaa.bbbb"

422+```

423+

424+`+spud` simply encodes a `path` in a tape:

425+

426+```

427+> (spud /foo/bar/baz)

428+"/foo/bar/baz"

429+```

430+

431+## Decoding from text

432+

433+For decoding atoms of particular auras encoded in cords, there are three

434+functions: [`+slat`](/docs/hoon/reference/stdlib/4m#slat),

435+[`+slav`](/docs/hoon/reference/stdlib/4m#slav), and

436+[`+slaw`](/docs/hoon/reference/stdlib/4m#slaw). Additionally, there is

437+[`+stab`](/docs/hoon/reference/stdlib/4m#stab) for decoding a cord to a path.

438+

439+`+slav` parses the given cord with the aura specified as a `@tas`, crashing if

440+the parsing failed. For example:

441+

442+```

443+> `@da`(slav %da '~2021.10.4..11.26.54')

444+~2021.10.4..11.26.54

445+

446+> `@p`(slav %p '~zod')

447+~zod

448+

449+> (slav %p 'foo')

450+dojo: hoon expression failed

451+```

452+

453+`+slaw` is like `+slav` except it produces a `unit` which is null if parsing

454+failed, rather than crashing. For example:

455+

456+```

457+> `(unit @da)`(slaw %da '~2021.10.4..11.26.54')

458+[~ ~2021.10.4..11.26.54]

459+

460+> `(unit @p)`(slaw %p '~zod')

461+[~ ~zod]

462+

463+> (slaw %p 'foo')

464+~

465+```

466+

467+`+slat` is a curried version of `+slaw`, meaning it's given the aura and

468+produces a new gate which takes the actual cord. For example:

469+

470+```

471+> `(unit @da)`((slat %da) '~2021.10.4..11.26.54')

472+[~ ~2021.10.4..11.26.54]

473+

474+> `(unit @p)`((slat %p) '~zod')

475+[~ ~zod]

476+

477+> ((slat %p) 'foo')

478+~

479+```

480+

481+Finally, `+stab` parses a cord containing a path to a `path`. For example:

482+

483+```

484+> (stab '/foo/bar/baz')

485+/foo/bar/baz

486+```

487+

488+## Futher reading

489+

490+- [Parsing](/docs/hoon/guides/parsing) - A guide to writing fully-fledged

491+ functional parsers in hoon.

492+- [Auras](/docs/hoon/reference/auras) - Details of auras in hoon.

493+

494+- [stdlib 2b: List logic](/docs/hoon/reference/stdlib/2b) - Standard library

495+ functions for manipulating lists, which are useful for dealing with tapes.

496+- [stdlib 2q: Molds and Mold-builders](/docs/hoon/reference/stdlib/2q) - Several

497+ text types are defined in this section of the standard library.

498+- [stdlib 4b: Text processing](/docs/hoon/reference/stdlib/4b) - Standard

499+ library functions for manipulating and converting tapes and strings.

500+

501+- [stdlib 4m: Formatting functions](/docs/hoon/reference/stdlib/4m) - Standard

502+ library functions for encoding and decoding atom auras in strings.