Macro functions are the means by which users can define forms that
behave very much like Common Lisp's
special forms. Many Common Lisp
special functions, such as let
and cond
, could be
defined as macros, assuming the existence of more primitive forms,
such as lambda
and if
.
Far and away, the most common need for macros is to implement
definers. Definers usually have names that begin with
"def
" or "define-
" and are used to attach data to
names. defun
, defstruct
, defclass
, and so
on, are examples of definers already in Common Lisp.
Here are two examples from the CS325 code library:
(defmop m-elephant (m-animal) :color m-grey)
(define-test pick-greater (assert-equal 5 (pick-greater 2 5)) (assert-equal 5 (pick-greater 5 2)) (assert-equal 10 (pick-greater 10 10)) (assert-equal 0 (pick-greater -5 0)) )
Why can't you implement these as functions?[1]
Another common macro type is the with-
macro.
Built-in examples are with-open-file
,
with-input-from-string
,
with-output-to-string
, and
with-accessors
. These macros usually have
the form (with-... (variable specifications)
body of code)
, e.g.,
(with-input-from-string (in "1 2 3") (list (read in) (read in) (read in)))
Occasionally, though less often than you might think,
there are macros for iteration. The
list-of
exercise is one example. It lets
you construct, for example, "the list of all n in a list that
are odd" like this:
(list-of (n :in '(1 2 3 4 5)) (oddp n))
For many people, definers are the only kinds of macros they will need to create. Fortunately, they are also the simplest to implement, once you understand how macros work.
How do macros work?
A macro call looks just like a function call, but the Lisp interpreter has the following special rules for evaluating macro calls:
- The macro function is applied (using
apply
) to the argument forms. Note: the arguments are not evaluated. The value returned is called the expansion of the macro. - The expansion is evaluated.
The Lisp compiler has similar rules:
- The macro function is applied (using
apply
) to the argument forms. - The expansion is compiled.
Because the compiler has to be able to expand a macro call at compile time, it makes no sense for a macro definition to depend on the run-time value of a variable.
How are macros defined?
A macro definition looks just like a function definition, except
that you use defmacro
instead of defun
. E.g.,
(defmacro macro-name (parameters) ...)
The rules for evaluating macros imply a very important, but often forgotten, principle for macro definitions:
A macro function should not do things. It should construct code to do things.
For example, suppose we wanted
(test-exp expression)
to print a message of the form
expression => value
e.g.,
(let ((x 1) (y 2)) (test-exp (+ x y)))
would print
(+ x y) => 3
Why can't test-exp
be a
function?[2]
Here is a possible definition of the macro test-exp
:
(defmacro test-exp (exp) (format t "~&~S => ~S~%" exp (eval exp)))
What's wrong with this definition? When would it fail?[3]
Here's the proper definition:
(defmacro test-exp (exp) `(format t "~&~S => ~S~%" ',exp ,exp))
Notice that all the macro function does is build a piece of code. That's all any macro should do.
The Macro Defining Method
The following method for defining macros is strongly recommended. It's designed to avoid:
- macros that break when compiled because they do things rather than generate code to do things
- inefficient macros that calculate things at run-time that could have been resolved at compile time
Here's the method:
- Define a function that does the "real" work. (This does not usually apply to control-flow macros, like looping forms.)
- Write down several examples of how the macro would be used.
- For each example, write down the equivalent code you'd write, using the function defined in Step 1, if the macro didn't exist.
- (a) Write the
defmacro
header, (b) insert one of the function call examples as the body, and (c) "parameterize" it, using backquote and "comma" and "at-comma" forms to replace specific data items with macro parameters. - Test the definition on the other examples and tweak as needed.
Implementing the macro test-exp
Step 1: Possible, but not really necessary.
Step 2: Write down some examples:
(test-exp (+ x y))
Step 3: Write down the equivalent code without the macro:
(format t "~&~S => ~S~%" '(+ x y) (+ x y))
Step 4(a): Write the defmacro
header. :
(defmacro test-exp (form) ...)
Step 4(b): Insert one of the examples in the body:
(defmacro test-exp (form) `(format t "~&~S => ~S~%" '(+ x y) (+ x y)))
Step 4(c): Parameterize the body with backquote, etc. That is, we
add a backquote, and replace specific data items with the macro's
parameters. Thus, all occurrences of (+ x y)
become
,form
. Notice that the quote's remain in the macro body.
(defmacro test-exp (form) `(format t "~&~S => ~S~%" ',form ,form))
Step 5: Test on other examples.
Implementing the macro defmop
Let's use the method to define defmop
.
Step 1: define a function to do the real work, e.g.,
(defun add-mop (...) ...)
We have to decide how this function would be called and what it
does. Mops have a name, a list of abstractions, and a list of slots,
and they're stored under the name. The principle of "Short Attention
Span Programming" dictates that separate tasks be implemented in
separate functions, so we'll define make-mop
to make a mop
structure, and store-mop
to save that structure under its
name in a table.
(defun add-mop (name absts slots) (store-mop name (make-mop :name name :absts absts :slots slots)))
Step 2: write down the some defmop
calls:
(defmop m-elephant (m-animal) :color m-grey) (defmop m-animal (m-thing))
Step 3: write down the equivalent calls to add-mop
:
(add-mop 'm-elephant '(m-animal) '(:color m-grey)) (add-mop 'm-animal '(m-thing) '())
Step 4(a): Write the defmacro
header. From our examples,
we know that we want defmop
to take a name, a list of
abstractions, and then zero or more slots, so the header would be:
(defmacro defmop (name absts &rest slots) ...)
Step 4(b): Insert one of the examples in the body:
(defmacro defmop (name absts &rest slots) (add-mop 'm-elephant '(m-animal) '(:color m-grey)))
Step 4(c): Parameterize the body with backquote, etc. That is, we
add a backquote, and replace specific data items with the macro's
parameters. Thus, m-elephant
becomes ,name
,
(m-animal)
becomes ,,absts
, and (:color
m-grey)
becomes ,slots
. Notice that the quotes are
not part of the specific data, so they remain in the macro
body.
(defmacro defmop (name absts &rest slots) `(add-mop ',name ',absts ',slots))
Step 5: Test on other examples. E.g., does (defmop m-animal
(m-thing))
work with the above definition, or does it need
tweaking?
Debugging Macros
Don't rely on what the macro call returns when debugging. Use the following idiom to see what a macro call expands into. If the expansion looks suspicious, it probably is wrong, even if the right answer comes back in some test cases:
(pprint (macroexpand-1 '(example macro call)))
Defining Complex Macros
There are two ways in which a macro can be too complex for Step 4 in the above method:
- It may need to calculate its expansion from the arguments, rather than follow a fixed tempate.
- It may need to process the arguments significantly before expansion.
Using Expander Functions
Some macros expand into a form that has to be "calculated" from their arguments. The most common situation is a macro that expands into a set of nested subforms.
A very simple example of this is the macro compose
.
compose
is an extension of function
. It takes zero
or more function specifications (usually function names, but they
could be (lambda ...)
forms), and "composes" them into one
function. That is,
(compose fn1 fn2 fn3 ...)
returns a function fn with the property that
(funcall fn expression)
is the same as
(funcall fn1 (funcall fn2 (funcall fn3 ... expression)))
For example,
> (mapcar (compose 1+ car) '((1 a) (2 b))) (2 3)
What function should (compose)
-- i.e., no arguments --
return?[4]
compose
is not a hard macro to define, but clearly it
doesn't fit a fixed template. Because of the nested nature of the
expansion, it can be most naturally defined with recursion.
When an expansion is complex, define one or more expander functions to construct the expansion the macro needs. Being normal functions, they're easy to call recursively, and easy to debug, too.
Here's a simple definition for compose
using an expander
function:
(defmacro compose (&rest fns) (let ((var (gensym))) `#'(lambda (,var) ,(expand-compose fns var)))) (defun expand-compose (fns var) (if (null fns) var `(,(first fns) ,(expand-compose (rest fns) var))))
Using Argument Parsing Functions
The other way a macro can be complex is if it has a complicated
calling format, or if a fair amount of processing has to be done to
the arguments. For example, loop
takes many different
clauses, and each clause has its own format, including,
for variable in list for variable from start [to end] [by increment] for variable = initial [then subsequent] when test clause finally expression
Another example is bind
, defined in AI
Programming.
(bind ((place1 value2) (place2 value2) ...) exp1 exp2 ...)
This generalizes let
. Each place
is
assigned the corresponding value
, using
setf
, then the expressions are evaluated, then
place
's are restored to their original values.
bind
generates code with unwind-protect
to
guarantee that the old values are restored, even if there's a
non-local exit (becaues of an error or a call to return-from
or throw
) during the expression evaluation.
Although the calling format is simple (the same as let
),
and the expansion fits a fixed format, it turns out bind
has
to calculate a number of special values from the input arguments
before it can fill out the expansion form.
When the processing before expansion is complex, define one
or more "parser" functions to process the arguments and get the data
needed to build the expansion. Name these functions get-...
or parse-...
, to keep them straight from the expander
functions.
For example, here's a cleaner
definition of the final version of bind
in Section 3.16 of
AI Programming.
Macros and Maintainable Code
Macros, like functions, should be used when they make code more maintainable, by making it easier to write code that is:
- readable
- robust
- efficient
For example, the form
(defmop m-elephant (m-animal) :color m-grey)
is more readable than the equivalent function call with the quotes and extra parentheses. The form
(loop for x in l when (test x) collect x)
will usually compile into more efficient code than the standard alternatives. The form
(with-open-file (instream file) (do ((x (read instream nil nil)...)) ...))
generates more robust code than most programmers would write,
using unwind-protect
to guarantee that streams are closed,
no matter what happens.
Macro Pitfalls
Even though macros are a simple idea, and macro definitions use regular old Lisp list functions, there are still several pitfalls that novices fall into repeatedly when defining macros. These mistakes happen when you fail to keep in mind two key facts about macros:
- Macros are not functions.
- Macros must be callable at compile-time.
Here are some common macro mistakes. See if you can see what's wrong with each one.
In-lining with Macros
Some Lisps before Common Lisp did not include functions
like first
, second
and so on. So
programmers who liked those names, would sometimes define
these macros:
(defmacro first (l) `(car ,l)) (defmacro second (l) `(cadr ,l)) ...
Programmers would use macros instead of normal functions for efficiency.
When compiled, first
, second
and so
on, would be replaced with car
, cadr
and soon,
so the resulting compiled code would be just as efficient as if
car
, cadr
, and so on, had been used
What's wrong with the above code?
Macros are not functions. Therefore, even though (first lst)
looks like a normal function call, you'll get an
error if you try to do (mapcar #'first lst)
or (find x lst :key #'first)
.
Functions are wonderfully useful things. If something can be a function,
it should be a function. The right way in Common Lisp to get efficient function
calls,
when and only when there's an efficiency problem, is to
use the inline
declaration. So, if there was a bottleneck
in code (highly unlikely) because first
, second
,
and so on were functions, then do this:
(declaim (inline first second ...) (defun first (l) (car l)) (defun second (l) (cadr l)) ...
Reminder: Common Lisp already defines these functions for you.
Problems with macros that look like but are not functions often show up pretty quickly as "... is not a function" type messages. However, "function-like" macros can lead to bugs that are quite hard to find.
A Bad push
Macro
Some Lisps before Common Lisp did not include push
.
Here's a simple definition for (push
item list)
, renamed to x-push
to avoid name conflict:
(defmacro x-push (x l) `(setf ,l (cons ,x ,l)))
This will correctly handle (x-push 'a l)
,
(x-push (car x) l)
, and even (x-push x (car l)
).
The last form correctly pushes the value of x
onto the front
of the list stored in the car
of l
.
What's wrong with the above code?
Simple though it is, the above definition manages to violate two expectations about normal argument evaluation:
- arguments are evaluated left to right
- arguments are not evaluated more than once
Try running the following code. It takes a list
of lists, l
, and a list of items, e.g.,
(a b c)
, and pushes each item onto the
corresponding list. (There are better ways to do this, but
this is one of the simplest cases I can think of to show
the bug that could occur is more complicated code.)
(let ((l (list nil nil nil)) (i -1)) (dolist (x '(a b c)) (push x (nth (incf i) l))) l)
If you run the above, you should get ((A) (B) (C))
as a result.
Now try it with x-push
. What happens? Why? How could
you fix it?
A Bad pop
Macro
Some Lisps before Common Lisp did not include pop
.
Here's a simple definition for (pop
list)
, renamed to x-pop
to avoid name conflict. The code has to save the first
element of list so that it can return it after
resetting list to its tail.
(defmacro x-pop (l) `(let ((save (car ,l))) (setf ,l (cdr ,l)) save))
This code has the same multiple evaluation problem that
x-push
has, but let's ignore that for now. Here's
a test of x-pop
that seems to do fine.
(let ((l '(a b c))) (print (x-pop l)) (print (x-pop l)) (print (x-pop l)) l)
What's wrong with the above definition?
Trying running the test code above, but change the name
of the local variable from l
to save
.
What happens? Why? How can you fix it?
Tell me more
For an excellent in-depth treatment of macros, see Paul Graham's On Lisp (Prentice-Hall). It gives a more sophisticated variant of the macro definition methodology outlined above, many examples, caveats, and so on. Warning: the later chapters introduce uses of macros that may be hazardous to code maintenance!
Footnotes
[1] These can't be functions because one or more arguments are not evaluated. [return]
[2] If test-exp
was a function, it would be passed 3
, the
value of (+ x y)
, not the form itself. [return]
[3] Consider this use of test-exp
:
(let ((x 1) (y 2)) (test-exp (+ x y)))
The call to
(test-exp (+ x y))
would expand into
(format t "~&~S => ~S~%" exp (eval exp))
At run time,
exp
will have the value (+ x y)
.
The first problem is that (eval '(+ x y))
evaluates the sum of the special bindings of x
and y
,
not the local ones. If there are no such bindings,
there will be an error. If you make x
and y
global,
you're still in trouble. The value of format
is always nil
.
Hence (test-exp (+ x y))
expands into nil
. So,
if you compile a file with the above call to expression, the file will
contain
(let ((x 1) (y 2)) nil)
Not very useful. [return]
[4] (compose)
should return a function that simply returns the argument passed to it, i.e.,
the identity function. It's left as an exercise for the reader to determine if
that's what the code given for compose
actually does.
[return]