python - append subprocess.Popen output to file?

Question

I can successfully redirect my output to a file, however this appears to overwrite the file's existing data:

import subprocess
outfile = open('test','w') #same with "w" or "a" as opening mode
outfile.write('Hello')
subprocess.Popen('ls',stdout=outfile)

will remove the 'Hello' line from the file.

I guess a workaround is to store the output elsewhere as a string or something (it won't be too long), and append this manually with outfile.write(thestring) - but I was wondering if I am missing something within the module that facilitates this.

Answer 1

You sure can append the output of subprocess.Popen to a file, and I make a daily use of it. Here's how I do it:

log = open('some file.txt', 'a')  # so that data written to it will be appended
c = subprocess.Popen(['dir', '/p'], stdout=log, stderr=log, shell=True)

(of course, this is a dummy example, I'm not using subprocess to list files...)

By the way, other objects behaving like file (with write() method in particular) could replace this log item, so you can buffer the output, and do whatever you want with it (write to file, display, etc) [but this seems not so easy, see my comment below].

Note: what may be misleading, is the fact that subprocess, for some reason I don't understand, will write before what you want to write. So, here's the way to use this:

log = open('some file.txt', 'a')
log.write('some text, as header of the file
')
log.flush()  # <-- here's something not to forget!
c = subprocess.Popen(['dir', '/p'], stdout=log, stderr=log, shell=True)

So the hint is: do not forget to flush the output!