python - list comprehension in exec with empty locals: NameError

Question

Consider the following snippet:

def bar():
    return 1
print([bar() for _ in range(5)])

It gives an expected output [1, 1, 1, 1, 1].

However, if I try to exec the same snippet in empty environment (locals and globals both are set to {}), it gives NameError:

if 'bar' in globals() or 'bar' in locals():
    del bar
# make sure we reset settings

exec("""
def bar():
    return 1
print([bar() for _ in range(5)])
""", {}, {})

NameError: name 'bar' is not defined

If I invoke exec like exec(…, {}) or exec(…), it is executed as expected.

Why?

EDIT:

Consider also the following snippet:

def foo():
    def bar():
        return 1
    print('bar' in globals()) # False
    print('bar' in locals()) # True
    print(['bar' in locals() for _ in [1]]) # [False]
    print([bar() for _ in [1, 2]]) # [1, 1]

Just like in my first exec, we don't have bar in locals inside list comprehension. However, if we try to invoke it, it works!

Answer 1

The solution to your problem lies here:

In all cases, if the optional parts are omitted, the code is executed in the current scope. If only globals is provided, it must be a dictionary, which will be used for both the global and the local variables. If globals and locals are given, they are used for the global and local variables, respectively. If provided, locals can be any mapping object. Remember that at module level, globals and locals are the same dictionary. If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition.

https://docs.python.org/3/library/functions.html#exec

Basically, your problem is that bar is defined in the scope of locals and only in locals. Therefore, this exec() statement works:

exec("""
def bar():
    return 1
print(bar())
""", {}, {})

The list comprehension however creates a new local scope, one in which bar is not defined and can therefore not be looked up.

This behaviour can be illustrated with:

exec("""
def bar():
    return 1
print(bar())
print(locals())
print([locals() for _ in range(1)])
""", {}, {})

which returns

1
{'bar': <function bar at 0x108efde18>}
[{'_': 0, '.0': <range_iterator object at 0x108fa8780>}]

EDIT

In your original example, the definition of bar is found in the (module level) global scope. This corresponds to

Remember that at module level, globals and locals are the same dictionary.

In the exec example, you introduce an artificial split in scopes between globals and locals by passing two different dictionaries. If you passed the same one or only the globals one (which would in turn mean that this one will be used for both globals and locals) , your example would also work.

As for the example introduced in the edit, this boils down to the scoping rules in python. For a detailed explanation, please read: https://docs.python.org/3/tutorial/classes.html#python-scopes-and-namespaces

In short, while bar is not in the local scope of the list comprehension and neither in the global scope, it is in the scope of foo. And given Python scoping rules, if a variable is not found in the local scope, it will be searched for in the enclosing scopes until the global scope is reached. In your example, foo's scope sits between the local scope and the global scope, so bar will be found before reaching the end of the search.

This is however still different to the exec example, where the locals scope you pass in is not enclosing the scope of the list comprehension, but completely divided from it.

Another great explanation of scoping rules including illustrations can be found here: http://sebastianraschka.com/Articles/2014_python_scope_and_namespaces.html