python - How to write a csv file in binary mode?

Question

Does python's csv writer not support binary mode anymore?

I haven't had to write in 'b' mode until now and i'm getting very annoying errors like so:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-030fb0c9dc9a> in <module>()
  4 with open('test.csv', 'wb') as f:
  5     w = csv.writer(f)
----> 6     w.writerows(rows)

TypeError: a bytes-like object is required, not 'str'

Code:

import csv

rows = [b'1,2,3', b'4,5,6', b'7,8,9']
with open('test.csv', 'wb') as f:
    w = csv.writer(f)
    w.writerows(rows)

If anyone could explain the error that would be great. I'm passing in an iterable where every element is a byte sequence but I still get an error about the input not being 'bytes' but instead being 'str.' This behavior seems unexpected.

I know the above code snippet can write to a normal file if I turn off the binary mode. If anyone has a solution or suggestion that is constructive I would very much appreciate it.

Answer 1

The csv module in Python 3 always attempts to write strings, not bytes:

To make it as easy as possible to interface with modules which implement the DB API, the value None is written as the empty string. [...] All other non-string data are stringified with str() before being written.

That means you have to pass it a file object that accepts strings, which usually means opening it in text mode.

If you are stuck with a file object that wants bytes, wrap it in an io.TextIOWrapper to handle str->bytes encoding:

# assuming you want utf-8
with io.TextIOWrapper(binary_file, encoding='utf-8', newline='') as text_file:
    w = csv.writer(text_file)