python - Read multiple block of file between start and stop flags

Question

I am trying to read sections of a file into numpy arrays that have similar start and stop flags for the different sections of the file. At the moment I have found a method that works, but for only one section of the input file before needing to re open the input file.

My code at the moment is:

    with open("myFile.txt") as f:
        array = []
        parsing = False
        for line in f:
            if line.startswith('stop flag'):
            parsing = False
        if parsing:
            #do things to the data
        if line.startswith('start flag'):
            parsing = True

I found the code from this question

With this code I need to re-open and read through the file.

Is there a way to read all sections without having to open the file for each section to be read?

Answer 1

You can use itertools.takewhile each time you reach the start flag to take until the stop:

from itertools import takewhile
with open("myFile.txt") as f:
        array = []
        for line in f:
            if line.startswith('start flag'):               
                data = takewhile(lambda x: not x.startswith("stop flag"),f)
                # use data and repeat

Or just use an inner loop:

with open("myFile.txt") as f:
    array = []
    for line in f:
        if line.startswith('start flag'):
            # beginning of section use first lin
            for line in f:
                # check for end of section breaking if we find the stop lone
                if line.startswith("stop flag"):
                    break
                 # else process lines from section

A file object returns its own iterator so the pointer will keep moving as you iterate over f, when you reach the start flag, start processing a section until you hit the stop. There is no reason to re-open the file at all, just use the sections as you iterate once over the lines of the file. If the start and stop flag lines are considered part of the section make sure to also use those too.