python - process.stdout.readline() hangs. How to use it properly?

Question

I want to repeatedly send requests to process standard input and receive responses from standard output without calling subprocess multiple times. I can achieve a one-time request-response iteration using p.communicate however not to call the subprocess multiple times I need to use: process.stdout.readline() which hangs. How to use it properly? I use Python 2.7 64 bit, Windows 7. Thanks in advance.

main.py:

import subprocess
p = subprocess.Popen(['python','subproc.py'],stdin=subprocess.PIPE,stdout=subprocess.PIPE)

while True:
    s=raw_input('Enter message:')
    p.stdin.write(s)
    p.stdin.flush()
    response = p.stdout.readline()
    if response!= '':
        print "Process response:", response
    else:
        break

subproc.py:

from __future__ import division
import pyximport
s=raw_input()
print 'Input=',s

Answer 1

There are a couple of minor tweaks you can make to get this working. First is to disable buffered output in the child using the -u option. Second is to send a newline character along with the user-inputted message to the child process, so that the raw_input call in the child completes.

main.py

import subprocess

# We use the -u option to tell Python to use unbuffered output
p = subprocess.Popen(['python','-u', 'subproc.py'],
                     stdin=subprocess.PIPE,
                     stdout=subprocess.PIPE)

while True:
    s = raw_input('Enter message:')
    p.stdin.write(s + "
")  # Include '
'
    p.stdin.flush()
    response = p.stdout.readline()
    if response != '': 
        print "Process response:", response
    else:
        break

You should also wrap the child process in an infinite loop, or things will break after the first message is sent:

subproc.py:

while True:
    s = raw_input()
    print 'Input=',s

Output:

dan@dantop:~$ ./main.py 
Enter message:asdf
Process response: Input= asdf

Enter message:asdf
Process response: Input= asdf

Enter message:blah blah
Process response: Input= blah blah

Enter message:ok
Process response: Input= ok