iterate python nested lists efficiently

Question

I am working on a network traffic monitor project in Python. Not that familiar with Python, so I am seeking help here.

In short, I am checking both in and out traffic, I wrote it this way:

for iter in ('in','out'):
        netdata = myhttp()
        print data

netdata is a list consisting of nested lists, its format is like this:

[ [t1,f1], [t2,f2], ...]

Here t represents the moment and f is the flow. However I just want to keep these f at this moment for both in and out, I wonder any way to get an efficient code.

After some search, I think I need to use create a list of traffic(2 elements), then use zip function to iterate both lists at the same time, but I have difficulty writing a correct one. Since my netdata is a very long list, efficiency is also very important.

If there is anything confusing, let me know, I will try to clarify. Thanks for help

Answer 1

Apart from minor fixes on your code (the issues raised by @Zero Piraeus), your question was probably answered here. A possible code to traverse a list of lists in N degree (a tree) is the following:

def traverse(item):
    try:
        for i in iter(item):
            for j in traverse(i):
                yield j
    except TypeError:
        yield item

Example:

l = [1, [2, 3], [4, 5, [[6, 7], 8], 9], 10]
print [i for i in traverse(l)]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

The key to make it work is recursion and the key to make it work efficiently is using a generator (the keyword yield gives the hint). The generator will iterate through your list of lists an returning to you item by item, without needing to copy data or create a whole new list (unless you consume the whole generator assigning the result to a list, like in my example)

Using iterators and generators can be strange concepts to understand (the keyword yield mainly). Checkout this great answer to fully understand them