python - Efficient creation of numpy arrays from list comprehension and in general

Question

In my current work, I use Numpy and list comprehensions a lot and in the interest of the best possible performance I have the following questions:

What actually happens behind the scenes if I create a Numpy array as follows?

a = numpy.array( [1,2,3,4] )

My guess is that python first creates an ordinary list containing the values, then uses the list size to allocate a numpy array and afterwards copies the values into this new array. Is this correct, or is the interpreter clever enough to realize that the list is only intermediary and instead copy the values directly?

Similarly, if i wish to create a numpy array from list comprehension using numpy.fromiter():

a = numpy.fromiter( [ x for x in xrange(0,4) ], int )

will this result in an intermediary list of values being created before being fed into fromiter()?

Answer 1

I believe than answer you are looking for is using generator expressions with numpy.fromiter.

numpy.fromiter((<some_func>(x) for x in <something>),<dtype>,<size of something>)

Generator expressions are lazy - they evaluate the expression when you iterate through them.

Using list comprehensions makes the list, then feeds it into numpy, while generator expressions will yield one at a time.

Python evaluates things inside -> out, like most languages (if not all), so using [<something> for <something_else> in <something_different>] would make the list, then iterate over it.