Python Macros: Use Cases?

Question

If Python had a macro facility similar to Lisp/Scheme (something like MetaPython), how would you use it?

If you are a Lisp/Scheme programmer, what sorts of things do you use macros for (other than things that have a clear syntactic parallel in Python such as a while loop)?

Solution 1

Some examples of lisp macros:

• ITERATE which is a funny and extensible loop facility
• CL-YACC/FUCC that are parser generators that generate parsers at compile time
• CL-WHO which allows specifying html documents with static and dynamic parts
• Parenscript which is a javascript code generator
• Various simple code-wrappers, e.g., error handlers (I have a with-gtk-error-message-handler that executes code and shows GtkMessageDialog if unhandled error occurs), executors (e.g., given a code, execute it in different thread; I have a within-main-thread macro that executes code in different threads; PCall library uses macros to wrap code to be executed concurrently)
• GUI builders with macros (e.g., specify widgets hierarchy and widgets' properties and have a macro generate code for creation of all widgets)
• Code generators that use external resources during compilation time. E.g., a macro that processes C headers and generates FFI code or a macro that generates classes definitions based on database schema
• Declarative FFI. E.g., specifying the foreign structures, functions, their argument types and having macros to generate corresponding lisp structures, functions with type mapping and marshaling code
• Continuations-based web frameworks for Common Lisp use macros that transform the code into CPS (continuation passing style) form.

Solution 2

I believe that macros run counter to Python's culture. Macros in Lisp allow the big ball of mud approach; you get to redefine the language to become more suited to your problem domain. Conversely Pythonic code uses the most natural built in feature of Python to solve a problem, instead of solving it in a way that would be more natural in a different language.

Macros are inherently unpythonic.

Solution 3

This is a somewhat late answer, but MacroPy is a new project of mine to bring macros to Python. We have a pretty substantial list of demos, all of which are use cases which require macros to implement, for example providing an extremely concise way of declaring classes:

@case
class Point(x, y)

p = Point(1, 2)
print p.x   # 1
print p     # Point(1, 2)

MacroPy has been used to implement features such as:

• Case Classes, easy Algebraic Data Types from Scala
• Pattern Matching from the Functional Programming world
• Tail-call Optimization
• Quasiquotes, a quick way to manipulate fragments of a program
• String Interpolation, a common feature in many languages, and Pyxl.
• Tracing and Smart Asserts
• PINQ to SQLAlchemy, a clone of LINQ to SQL from C#
• Quick Lambdas from Scala and Groovy,
• Parser Combinators, inspired by Scala's.

Check out the linked page to find out more; I think I can confidently say that the use cases we demonstrate far surpass anything anyone's suggested so far on this thread =D

Solution 4

There's a mailing list posting (archive.org mirror) which explains this rather well. The post is about Perl, but it applies to Python just as well.

Solution 5

In lisp, macros are just another way to abstract ideas.

This is an example from an incomplete ray-tracer written in clojure:

(defmacro per-pixel
  "Macro.
Excecutes body for every pixel. Binds i and j to the current pixel coord."
  [i j & body]
  `(dotimes [~i @width]
     (dotimes [~j @height]
       ~@body)))

If you want to do something to every pixel with coordinates (i,j), say, draw a black pixel if i is even, you would write:

(per-pixel i,j
  (if (even? i)
    (draw-black i,j)))

This is not possible to do without macros because @body can mean anything inside (per-pixel i j @body)

Something like this would be possible in python as well. You need to use decorators. You can't do everything you can do with lisp macros, but they are very powerful

Check out this decorator tutorial: http://www.artima.com/weblogs/viewpost.jsp?thread=240808

View more solutions

Comments

[Rick Copeland]

That's kind of begging the question, though, isn't it? If macros were added to the language, by that logic, they'd become "pythonic", just as decorators, list comprehensions, the conditional expression, metaclasses, etc. would not have been considered "pythonic" before they were added as built-in features to the language. What makes macros less pythonic than, say, using sys._getframe() and metaclasses?

[Dana the Sane]

You'd be right, but this is usually in reference to PEP 20 python.org/dev/peps/pep-0020 and the like which outline the general philosophy of the language. In this case, I think macros are likely to go against several of the linked guidelines.

[Rick Copeland]

I'm familiar with the Zen of Python; I'm just not convinced that macros are necessarily in opposition to it.

[Dana the Sane]

I think it would depend how they're implemented. Looking at the metapython link provided in the question, I see a few problems.

[Rainer Joswig]

in Lisp: Write a function PER-PIXEL and pass the body as another function. No macro needed.

[Hannes Ovrén]

Yes, I also fail to understand why this is a good macro example.

[RossFabricant]

Pythonic style is like the style of a font. List comprehensions et al. are like different letters in the Python font. Macros are like meta rules to let you generate your own font.

[user1066101]

Macros aren't necessary -- which makes them unpythonic. There aren't any sensible use cases. Examples from other languages don't apply here. Macros are a source-level workaround to language gaps.

[user1066101]

-1: Examples from lisp don't apply very well to Python. If these can be done in Python without macros, then these aren't really use cases for Python are they?

[Ken]

The only way I see to do most of these in Python is extending the syntax. But that's always true: if you get the language maintainer to extend the language for every feature you might want, that's essentially what macros do. Only with a macro facility, you don't need to fork the language or ask Guido every time.

[dmitry_vk]

Macros are a point in a space of trade-offs in language design. They tradeoff performance vs dynamism (by limiting the neccesity to call eval at runtime, allowing compiler to apply logic for optimization, type-checking, type-inference). Macros are neccessary to be able to introduce useful abstractions while retaining ability to do useful optimizations. This is one of the reasons why Common Lisp implementations are fast and CPython is slow - one can not easily remove interpreter overhead.

[dmitry_vk]

S.Lott: The question is not about applicability of use cases, but about actual use cases. Strictly speaking, when you have eval(), you can do everything without macros. I think that such kind of an argument is counter-productive. I do not think that every one of mentioned examples can be done in Python without losing important features (e.g., minimal/small runtime overhead, compile-time error detection, integration with the rest of the language).

[RossFabricant]

@dmitry I think the tradeoff is between powerful abstractions and code clarity. Performance seems to be unrelated.

[lionel-dev-fr]

"You can do everything with eval()" is only true if you're willing to put all your code in strings, and write your own parser/translator. But if those are true, you don't even need eval(): any Turing-equivalent language is sufficient.

[dmitry_vk]

Ken: it is possible to have eval without parser. Some lisps (e.g., newlisp) have F-Exprs. Fexpr is a kind of function that accepts unevaluated arguments as expressions (AST subtrees) and it can decide what to do with them: eval, change and eval, discard, etc. Of course, there are huge problems with fexprs.

[dmitry_vk]

rossfabricant: have a look at en.wikipedia.org/wiki/Fexpr . Fexprs allow same kind of code clarity and are more powerful than macros (in terms of computations that are possible with them; everything that is possible to do with macros is possible to do with fexprs; but not the other way).

[Aaron]

The other case being over looked here is early/eager evaluation at compile time. As dmitry-vk mentions in his answer, a parser library that generates parser state machines (very costly) at bytecode-compilation-time. python doesn't "compile" in the traditional sense, but it does convert source to bytecode and then caches it. It would be awfully useful to cache those expensive state machines.

[Aaron]

At the core, macros are about inventing new (better?) ways of saying the same thing. Doing the same thing only different sounds very un-pythonic, probably. Consult "import this".

[Admin]

You don't need macros for that in Lisp. You also don't need decorators for that in Python. It's just a multidimensional "foreach" type function. Why not per_pixel(callback) where per_pixel calls callback with j and i. In Lisp, callback could be a lambda. In Python it could too (but, pythonically, lambdas are limited to being single expressions).

[abarnert]

If you just define a function, you can write err("abc"), which is surely better than print >> err, "abc", and not that much worse than err "abc"…

[Evgeni Sergeev]

Good point: def err(*args): print >> sys.stderr, ' '.join(map(str, args)).

[ArtOfWarfare]

+1: You definitely get why macros should exist much better than anyone else who has responded. Having said that, I dislike your library. Using the macros created by it seems more complicated than necessary, and writing macros with it is far, far more complicated than necessary.

[yardsale8]

Nice simple examples. I would add a let expression and ifelse function to your list.