python - What is the idiomatic way to iterate over a binary file?

Question

With a text file, I can write this:

with open(path, 'r') as file:
    for line in file:
        # handle the line

This is equivalent to this:

with open(path, 'r') as file:
    for line in iter(file.readline, ''):
        # handle the line

This idiom is documented in PEP 234 but I have failed to locate a similar idiom for binary files.

With a binary file, I can write this:

with open(path, 'rb') as file:
    while True:
        chunk = file.read(1024 * 64)
        if not chunk:
            break
        # handle the chunk

I have tried the same idiom that with a text file:

def make_read(file, size):
    def read():
        return file.read(size)
    return read

with open(path, 'rb') as file:
    for chunk in iter(make_read(file, 1024 * 64), b''):
        # handle the chunk

Is it the idiomatic way to iterate over a binary file in Python?

Answer 1

Try:

>>> with open('dups.txt','rb') as f:
...    for chunk in iter((lambda:f.read(how_many_bytes_you_want_each_time)),''):
...       i+=1

iter needs a function with zero arguments.

• a plain f.read would read the whole file, since the size parameter is missing;
• f.read(1024) means call a function and pass its return value (data loaded from file) to iter, so iter does not get a function at all;
• (lambda:f.read(1234)) is a function that takes zero arguments (nothing between lambda and :) and calls f.read(1234).

There is equivalence between following:

somefunction = (lambda:f.read(how_many_bytes_you_want_each_time))

and

def somefunction(): return f.read(how_many_bytes_you_want_each_time)

and having one of these before your code you could just write: iter(somefunction, '').

Technically you can skip the parentheses around lambda, python's grammar will accept that.