2020-04-17 11:43:47 +03:00
|
|
|
|
# Macros
|
|
|
|
|
|
|
|
|
|
Macros are among the most divisive features about any Lisp. There are many
|
|
|
|
|
different design decisions to be made, and all of them have proponents and
|
|
|
|
|
detractors.
|
|
|
|
|
|
2021-10-12 18:52:24 +03:00
|
|
|
|
This document aims to give a comprehensive overview of the macro system and how
|
|
|
|
|
to use it. If you’re in a hurry or want to see whether Carp implements your
|
2020-04-17 11:43:47 +03:00
|
|
|
|
favorite macro feature, you probably want to read the section [“In a
|
2021-10-12 18:52:24 +03:00
|
|
|
|
Nutshell”](#in-a-nutshell). If you want to spend some quality time
|
|
|
|
|
understanding how to work on or with the macro systems, the sections [“Working
|
|
|
|
|
with Macros”](#working-with-macros) and [“Inner Workings”](#inner-workings)
|
|
|
|
|
will probably be more useful to you.
|
2020-04-17 11:43:47 +03:00
|
|
|
|
|
|
|
|
|
## In a Nutshell
|
|
|
|
|
|
|
|
|
|
The macro system we’ve settled on for Carp is fairly simple. It is:
|
|
|
|
|
|
|
|
|
|
- not hygienic, but provides `gensym` capabilities,
|
|
|
|
|
- does not currently provide quasiquoting (this is not a requirement, it is
|
|
|
|
|
currently just not implemented); thus the bread and butter in your macro
|
|
|
|
|
toolbox will be `car`, `cdr`, `cons`, and `list`,
|
|
|
|
|
- defines macros with a fairly simple `defmacro`-based syntax, and has support
|
|
|
|
|
for compile-time or dynamic functions (for more information on this aspect,
|
|
|
|
|
please read [“Working with Macros”](#working-with-macros) below), and
|
|
|
|
|
- it sees the dynamic environment not just as an environment in which to
|
|
|
|
|
generate code through expanding macros, but also as a place for telling the
|
|
|
|
|
compiler more about the source. As an example, consider the dynamic function
|
|
|
|
|
`Project.config`, which allows you to set such things as the C compiler to
|
|
|
|
|
use, the name of the compiled project, or the output directory. To see this
|
|
|
|
|
in action, consider [this Carp snippet](https://github.com/carpentry-org/snippets/blob/master/build_system.carp)
|
|
|
|
|
which implements a simple multi-compiler build system for Carp in the dynamic
|
|
|
|
|
environment.
|
|
|
|
|
|
|
|
|
|
## Working with Macros
|
|
|
|
|
|
|
|
|
|
Macros are defined using the `defmacro` primitive form, like this:
|
|
|
|
|
|
|
|
|
|
```clojure
|
|
|
|
|
(defmacro apply [f args] (cons f args))
|
|
|
|
|
|
|
|
|
|
(apply + (1 2)) ; => (+ 1 2)
|
|
|
|
|
(apply Array.replicate (5 "hello")) ; => (Array.replicate 5 "hello")
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The example above defines `apply`, a macro that takes a function and a set of
|
|
|
|
|
arguments defined as a list and rewrites it so that the function gets applied
|
|
|
|
|
to these arguments by `cons`tructing a list with `f` as a head and `args` as
|
|
|
|
|
tail.
|
|
|
|
|
|
|
|
|
|
Because `apply` is a macro you will not need to quote the list passed to it. If
|
|
|
|
|
that looks strange, you might want to define `apply` as a dynamic function
|
|
|
|
|
instead. The main difference between macros and dynamic functions is that
|
|
|
|
|
dynamic functions evaluate their arguments and macros are expanded inside their
|
|
|
|
|
definitions. You may define a dynamic function like this:
|
|
|
|
|
|
|
|
|
|
```clojure
|
|
|
|
|
(defndynamic apply [f args] (cons f args))
|
|
|
|
|
|
|
|
|
|
(apply '+ '(1 2)) ; => (+ 1 2)
|
|
|
|
|
(apply 'Array.replicate '(5 "hello")) ; => (Array.replicate 5 "hello")
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
If you compare this code example to the macro example above, you’ll see that
|
|
|
|
|
they are extremely similar, except for the invocation `defndynamic` and the
|
|
|
|
|
quotes in their invocation.
|
|
|
|
|
|
|
|
|
|
Macros also provide rest arguments; this basically means that you may define
|
|
|
|
|
variadic macros by providing a “catch-all” argument as the last argument.
|
|
|
|
|
|
|
|
|
|
```clojure
|
|
|
|
|
(defmacro apply-or-sym [head :rest tail]
|
|
|
|
|
(if (= (length tail) 0)
|
|
|
|
|
head
|
|
|
|
|
(cons head tail)))
|
|
|
|
|
|
|
|
|
|
(apply-or-sym *global*) ; => *global*
|
|
|
|
|
(apply-or-sym + 1 2) ; => (+ 1 2)
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The macro `apply-or-sym` is slightly ridiculous, but it should drive the point
|
|
|
|
|
home. It takes one formal argument, `head`. You may provide any number of
|
|
|
|
|
arguments after that—they will be bound to `tail`. Thus, tail will be a list of
|
|
|
|
|
zero or more arguments. If we do not provide any, `apply-or-sym` will just
|
|
|
|
|
return `head`. If we do, we treat it as a regular invocation. This kind of
|
|
|
|
|
macro might look slightly silly, but rest assured that using rest arguments has
|
|
|
|
|
many legitimate use cases.
|
|
|
|
|
|
|
|
|
|
If you’d like to see more examples of macros big and small, you should now be
|
|
|
|
|
equipped to understand a lot of the macros in [the standard
|
|
|
|
|
library](/core/Macros.carp) and even [`fmt`](/core/Format.carp), a fairly
|
|
|
|
|
complex piece of macro machinery.
|
|
|
|
|
|
|
|
|
|
Some helpful functions for exploring macros in the REPL are `expand`, `eval`,
|
|
|
|
|
and `macro-log`. `expand` will expand macros for you, while `eval` evaluates
|
|
|
|
|
the resulting code. `macro-log` is useful for tracing your macro, a form of
|
|
|
|
|
“printf debugging”.
|
|
|
|
|
|
|
|
|
|
## Inner Workings
|
|
|
|
|
|
2020-04-17 18:25:53 +03:00
|
|
|
|
The Carp compiler is split in a few different stages. The diagram below
|
|
|
|
|
illustrates the flow of the compiler.
|
|
|
|
|
|
|
|
|
|
![The compiler passes](./compiler-passes.svg)
|
|
|
|
|
|
|
|
|
|
The dynamic evaluator is arguably one of the most central pieces of the Carp
|
|
|
|
|
compiler. It orchestrates macro expansion, borrow checking, and type inference,
|
|
|
|
|
as it encounters forms that have requirements for these services, such as
|
|
|
|
|
function definitions, variables, or `let` bindings.
|
|
|
|
|
|
2020-04-17 19:34:20 +03:00
|
|
|
|
Therefore, understanding the evaluator will give you a lot of insight into how
|
|
|
|
|
Carp works generally.
|
2020-04-17 18:25:53 +03:00
|
|
|
|
|
|
|
|
|
The most tried-and-true starting point for understanding the dynamic evaluator
|
|
|
|
|
is `eval` in [`src/Eval.hs`](/src/Eval.hs).
|
|
|
|
|
|
|
|
|
|
### Data Structures
|
|
|
|
|
|
|
|
|
|
The type signature of `eval` is as follows:
|
|
|
|
|
|
|
|
|
|
```haskell
|
|
|
|
|
eval :: Context -> XObj -> IO (Context, Either EvalError XObj)
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Thus, to understand it, we’ll have to understand at least `Context`, `XObj`,
|
|
|
|
|
and `EvalError`. The types `IO` and `Either` are part of the Haskell standard
|
|
|
|
|
library and will not be covered extensively—please refer to your favorite
|
|
|
|
|
tool for Haskell documentation (we recommend [Stackage](https://stackage.org))
|
|
|
|
|
to find out more about them.
|
|
|
|
|
|
|
|
|
|
All data structures that are discussed here are defined in
|
2020-04-17 19:34:20 +03:00
|
|
|
|
[`src/Obj.hs`](/src/Obj.hs).
|
2020-04-17 18:25:53 +03:00
|
|
|
|
|
|
|
|
|
#### `XObj`
|
|
|
|
|
|
|
|
|
|
`XObj` is short for “`Obj` with eXtras”. `Obj` is the type for AST nodes in
|
|
|
|
|
Carp, and it’s used throughout the compiler. Most often, you’ll find it wrapped
|
|
|
|
|
in an `XObj`, though, which annotates such an `Obj` with an optional source
|
|
|
|
|
location information—in the field `info`, modelled as a `Maybe Info`—and
|
|
|
|
|
type information—in the field `ty`, modelled as a `Maybe Ty`. While both of
|
|
|
|
|
these fields are important, for the purposes of this document we will overlook
|
|
|
|
|
them and treat a `XObj` as an ordinary AST node. Thus, `eval` becomes a
|
|
|
|
|
function that takes a context and an AST node, and returns a pair consisting of
|
|
|
|
|
a new context, and either an `EvalError` or a new AST node.
|
|
|
|
|
|
|
|
|
|
#### `Context`
|
|
|
|
|
|
|
|
|
|
`Context` is a data structure that holds all of the state of the Carp compiler.
|
|
|
|
|
It is fairly extensive, holding information ranging from the type and value
|
|
|
|
|
environments to the history of evaluation frames that were traversed for
|
|
|
|
|
tracebacks.
|
|
|
|
|
|
|
|
|
|
The entire state of the compiler should be inspectable by inspecting its
|
|
|
|
|
context.
|
|
|
|
|
|
|
|
|
|
#### `EvalError`
|
|
|
|
|
|
|
|
|
|
An `EvalError` is emitted whenever the dynamic evaluator encounters an error.
|
|
|
|
|
It consists of an error message and meta information (such as a traceback and
|
|
|
|
|
source location information).
|
|
|
|
|
|
|
|
|
|
### Evaluation
|
|
|
|
|
|
|
|
|
|
The dynamic evaluator in Carp takes care both of evaluation and meta-level
|
|
|
|
|
information like definitions. This means that definitions are treated much like
|
|
|
|
|
dynamic primitives to evaluate rather than special constructs. In fact, many of
|
|
|
|
|
them are not treated as special forms, but are implemented as `Primitive`s.
|
|
|
|
|
|
|
|
|
|
Because we already introduced multiple constructs by name, let us define what
|
|
|
|
|
kinds of Carp constructs there are for the evaluator:
|
|
|
|
|
|
|
|
|
|
- Special forms: these are forms that have their own representation in the
|
|
|
|
|
abstract syntax tree and are treated directly in the evaluator loop. `fn` and
|
2020-04-19 18:35:59 +03:00
|
|
|
|
`the` are examples for this category. They cannot be passed around by value,
|
|
|
|
|
as you would do in higher order functions.
|
2020-04-17 18:25:53 +03:00
|
|
|
|
- Primitives: these are regular Carp forms that do not evaluate their
|
|
|
|
|
arguments, and they resemble builtin macros implemented in Haskell. Examples
|
2020-05-04 01:16:26 +03:00
|
|
|
|
for this category include `defmacro`, `defn`, and `quote`.
|
2020-04-17 18:25:53 +03:00
|
|
|
|
- Commands: these, too, are regular Carp forms. They evaluate their arguments
|
|
|
|
|
and behave like builtin functions. Examples for this category include
|
|
|
|
|
`Project.config`, `car`, and `cons`.
|
|
|
|
|
|
|
|
|
|
Primitives are mostly defined in [`src/Primitives.hs`](/src/Primitives.hs),
|
|
|
|
|
commands can be found in [`src/Commands.hs`](/src/Commands.hs), and special
|
|
|
|
|
forms can be found directly inside `eval`.
|
|
|
|
|
|
2020-05-05 16:21:36 +03:00
|
|
|
|
They are wired up into the environment and given names in
|
|
|
|
|
[`src/StartingEnv.hs`](/src/StartingEnv.hs).
|
|
|
|
|
|
2020-04-17 18:25:53 +03:00
|
|
|
|
#### Adding your own special forms, primitives, or commands
|
|
|
|
|
|
|
|
|
|
While there is a lot of machinery involved in getting your own primitives or
|
|
|
|
|
commands into the Carp evaluator, there are a lot of simple functions around to
|
|
|
|
|
help you get started.
|
|
|
|
|
|
|
|
|
|
If the name for the primitive or command is already present as a runtime
|
|
|
|
|
function, it should try to mimic its behavior as closely as possible.
|
|
|
|
|
|
|
|
|
|
Adding special forms is a little more involved and we try to exercise caution
|
|
|
|
|
in what to add, since every form makes `eval` harder to understand and reason
|
|
|
|
|
about. You should probably get in touch [on the
|
|
|
|
|
chat](https://gitter.im/carp-lang/carp) before embarking on a quest to
|
|
|
|
|
implement a new special form to avoid frustration.
|
2020-04-19 18:35:59 +03:00
|
|
|
|
|
|
|
|
|
#### A current list of special forms
|
|
|
|
|
|
|
|
|
|
Since special forms are “magical”, they deserve an enumeration. Currently there
|
|
|
|
|
are:
|
|
|
|
|
|
|
|
|
|
- `if` for branching,
|
|
|
|
|
- `defn` for defining functions,
|
|
|
|
|
- `def` for defining global variables,
|
|
|
|
|
- `let` for defining local variables,
|
|
|
|
|
- `the` for type annotations,
|
|
|
|
|
- `fn` for function literals.
|