python - What can you do with Lisp macros that you can't do with first-class functions?

Question

I think I understand Lisp macros and their role in the compilation phase.

But in Python, you can pass a function into another function

def f(filename, g):
  try:                                
     fh = open(filename, "rb") 
     g(fh)
  finally:
     close(fh) 

So, we get lazy evaluation here. What can I do with macros and not with functions as first class objects?

Answer 1

First of all Lisp has first-class functions too, so you could as well ask: "Why do I need macros in Lisp if I already have first-class functions". The answer to that is that first-class functions don't allow you to play with syntax.

On a cosmetic level, first-class functions allow you to write f(filename, some_function) or f(filename, lambda fh: fh.whatever(x)), but not f(filename, fh, fh.whatever(x)). Though arguably that's a good thing because in that last case it is a lot less clear where fh suddenly comes from.

More importantly functions can only contain code that is valid. So you can't write a higher-order function reverse_function that takes a function as an argument and executes it "in reverse", so that reverse_function(lambda: "hello world" print) would execute print "hello world". With a macro you can do this. Of course this particular example is quite silly, but this ability is enormously useful when embedding domain specific languages.

For example you couldn't implement common lisp's loop construct in python. Hell, you couldn't even implement python's for ... in construct in python if it wasn't really built-in - at least not with that syntax. Sure you could implement something like for(collection, function), but that's a lot less pretty.