for loop - Python `for` syntax: block code vs single line generator expressions

Question

I'm familiar with the for loop in a block-code context. eg:

for c in "word":
    print c

I just came across some examples that use for differently. Rather than beginning with the for statement, they tag it at the end of an expression (and don't involve an indented code-block). eg:

sum(x*x for x in range(10))

Can anyone point me to some documentation that outlines this use of for? I've been able to find examples, but not explanations. All the for documentation I've been able to find describes the previous use (block-code example). I'm not even sure what to call this use, so I apologize if my question's title is unclear.

Answer 1

What you are pointing to is Generator in Python. Take a look at: -

• http://wiki.python.org/moin/Generators
• http://www.python.org/dev/peps/pep-0255/
• http://docs.python.org/whatsnew/2.5.html#pep-342-new-generator-features

See the documentation: - Generator Expression which contains exactly the same example you have posted

From the documentation: -

Generators are a simple and powerful tool for creating iterators. They are written like regular functions but use the yield statement whenever they want to return data. Each time next() is called, the generator resumes where it left-off (it remembers all the data values and which statement was last executed)

Generators are similar to List Comprehension that you use with square brackets instead of brackets, but they are more memory efficient. They don't return the complete list of result at the same time, but they return generator object. Whenever you invoke next() on the generator object, the generator uses yield to return the next value.

List Comprehension for the above code would look like: -

[x * x for x in range(10)]

You can also add conditions to filter out results at the end of the for.

[x * x for x in range(10) if x % 2 != 0]

This will return a list of numbers multiplied by 2 in the range 1 to 5, if the number is not divisible by 2.

An example of Generators depicting the use of yield can be: -

def city_generator():
    yield("Konstanz")
    yield("Zurich")
    yield("Schaffhausen")
    yield("Stuttgart")

>>> x = city_generator()
>>> x.next()
Konstanz
>>> x.next()
Zurich
>>> x.next()
Schaffhausen
>>> x.next()
Stuttgart
>>> x.next()
Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
StopIteration

So, you see that, every call to next() executes the next yield() in generator. and at the end it throws StopIteration.