9.0 KiB
Macros
Macros are among the most divisive features about any Lisp. There are many different design decisions to be made, and all of them have proponents and detractors.
This document aims to give a comprehensive overview of the macro system and how to use it. If you’re in a hurry or want to see whether Carp implements your favorite macro feature, you probably want to read the section “In a Nutshell”. If you want to spend some quality understanding how to work on or with the macro systems, the sections “Working with Macros” and “Inner Workings” will probably be more useful to you.
In a Nutshell
The macro system we’ve settled on for Carp is fairly simple. It is:
- not hygienic, but provides
gensym
capabilities, - does not currently provide quasiquoting (this is not a requirement, it is
currently just not implemented); thus the bread and butter in your macro
toolbox will be
car
,cdr
,cons
, andlist
, - defines macros with a fairly simple
defmacro
-based syntax, and has support for compile-time or dynamic functions (for more information on this aspect, please read “Working with Macros” below), and - it sees the dynamic environment not just as an environment in which to
generate code through expanding macros, but also as a place for telling the
compiler more about the source. As an example, consider the dynamic function
Project.config
, which allows you to set such things as the C compiler to use, the name of the compiled project, or the output directory. To see this in action, consider this Carp snippet which implements a simple multi-compiler build system for Carp in the dynamic environment.
Working with Macros
Macros are defined using the defmacro
primitive form, like this:
(defmacro apply [f args] (cons f args))
(apply + (1 2)) ; => (+ 1 2)
(apply Array.replicate (5 "hello")) ; => (Array.replicate 5 "hello")
The example above defines apply
, a macro that takes a function and a set of
arguments defined as a list and rewrites it so that the function gets applied
to these arguments by cons
tructing a list with f
as a head and args
as
tail.
Because apply
is a macro you will not need to quote the list passed to it. If
that looks strange, you might want to define apply
as a dynamic function
instead. The main difference between macros and dynamic functions is that
dynamic functions evaluate their arguments and macros are expanded inside their
definitions. You may define a dynamic function like this:
(defndynamic apply [f args] (cons f args))
(apply '+ '(1 2)) ; => (+ 1 2)
(apply 'Array.replicate '(5 "hello")) ; => (Array.replicate 5 "hello")
If you compare this code example to the macro example above, you’ll see that
they are extremely similar, except for the invocation defndynamic
and the
quotes in their invocation.
Macros also provide rest arguments; this basically means that you may define variadic macros by providing a “catch-all” argument as the last argument.
(defmacro apply-or-sym [head :rest tail]
(if (= (length tail) 0)
head
(cons head tail)))
(apply-or-sym *global*) ; => *global*
(apply-or-sym + 1 2) ; => (+ 1 2)
The macro apply-or-sym
is slightly ridiculous, but it should drive the point
home. It takes one formal argument, head
. You may provide any number of
arguments after that—they will be bound to tail
. Thus, tail will be a list of
zero or more arguments. If we do not provide any, apply-or-sym
will just
return head
. If we do, we treat it as a regular invocation. This kind of
macro might look slightly silly, but rest assured that using rest arguments has
many legitimate use cases.
If you’d like to see more examples of macros big and small, you should now be
equipped to understand a lot of the macros in the standard
library and even fmt
, a fairly
complex piece of macro machinery.
Some helpful functions for exploring macros in the REPL are expand
, eval
,
and macro-log
. expand
will expand macros for you, while eval
evaluates
the resulting code. macro-log
is useful for tracing your macro, a form of
“printf debugging”.
Inner Workings
The Carp compiler is split in a few different stages. The diagram below illustrates the flow of the compiler.
The dynamic evaluator is arguably one of the most central pieces of the Carp
compiler. It orchestrates macro expansion, borrow checking, and type inference,
as it encounters forms that have requirements for these services, such as
function definitions, variables, or let
bindings.
Therefore, understanding the evaluator will give you a lot of insight into how Carp works generally.
The most tried-and-true starting point for understanding the dynamic evaluator
is eval
in src/Eval.hs
.
Data Structures
The type signature of eval
is as follows:
eval :: Context -> XObj -> IO (Context, Either EvalError XObj)
Thus, to understand it, we’ll have to understand at least Context
, XObj
,
and EvalError
. The types IO
and Either
are part of the Haskell standard
library and will not be covered extensively—please refer to your favorite
tool for Haskell documentation (we recommend Stackage)
to find out more about them.
All data structures that are discussed here are defined in
src/Obj.hs
.
XObj
XObj
is short for “Obj
with eXtras”. Obj
is the type for AST nodes in
Carp, and it’s used throughout the compiler. Most often, you’ll find it wrapped
in an XObj
, though, which annotates such an Obj
with an optional source
location information—in the field info
, modelled as a Maybe Info
—and
type information—in the field ty
, modelled as a Maybe Ty
. While both of
these fields are important, for the purposes of this document we will overlook
them and treat a XObj
as an ordinary AST node. Thus, eval
becomes a
function that takes a context and an AST node, and returns a pair consisting of
a new context, and either an EvalError
or a new AST node.
Context
Context
is a data structure that holds all of the state of the Carp compiler.
It is fairly extensive, holding information ranging from the type and value
environments to the history of evaluation frames that were traversed for
tracebacks.
The entire state of the compiler should be inspectable by inspecting its context.
EvalError
An EvalError
is emitted whenever the dynamic evaluator encounters an error.
It consists of an error message and meta information (such as a traceback and
source location information).
Evaluation
The dynamic evaluator in Carp takes care both of evaluation and meta-level
information like definitions. This means that definitions are treated much like
dynamic primitives to evaluate rather than special constructs. In fact, many of
them are not treated as special forms, but are implemented as Primitive
s.
Because we already introduced multiple constructs by name, let us define what kinds of Carp constructs there are for the evaluator:
- Special forms: these are forms that have their own representation in the
abstract syntax tree and are treated directly in the evaluator loop.
fn
andthe
are examples for this category. They cannot be passed around by value, as you would do in higher order functions. - Primitives: these are regular Carp forms that do not evaluate their
arguments, and they resemble builtin macros implemented in Haskell. Examples
for this category include
defmacro
,defn
, andquote
. - Commands: these, too, are regular Carp forms. They evaluate their arguments
and behave like builtin functions. Examples for this category include
Project.config
,car
, andcons
.
Primitives are mostly defined in src/Primitives.hs
,
commands can be found in src/Commands.hs
, and special
forms can be found directly inside eval
.
Adding your own special forms, primitives, or commands
While there is a lot of machinery involved in getting your own primitives or commands into the Carp evaluator, there are a lot of simple functions around to help you get started.
If the name for the primitive or command is already present as a runtime function, it should try to mimic its behavior as closely as possible.
Adding special forms is a little more involved and we try to exercise caution
in what to add, since every form makes eval
harder to understand and reason
about. You should probably get in touch on the
chat before embarking on a quest to
implement a new special form to avoid frustration.
A current list of special forms
Since special forms are “magical”, they deserve an enumeration. Currently there are:
if
for branching,defn
for defining functions,def
for defining global variables,let
for defining local variables,the
for type annotations,fn
for function literals.