python - Async IO - reading char from input blocks output

Question

Note: this example was tested on a linux terminal emulator, and due to the use of termios (which I have no idea if it's cross-platform) it might not work well on other operating systems' terminals.

---

I've been trying to make an "asynchronous" python prompt. What I mean by that is that, while the user is typing an input from a prompt, they can also receive messages, without cancelling the input.

Below is an implementation of it using asyncio.Queue and some termios flags (sorry in advance for the complexity, I tried to reduce it as much as possible):

import sys, termios, os
import asyncio

def readline(prompt: str = "Input: "):
    # termios stuff to: disable automatic echo so that, when a character is typed, it is not immediately printed on screen
    #                   read a single character from stdin without pressing <Enter> to finish
    fd = sys.stdin.fileno()
    orig_termios = termios.tcgetattr(fd)
    new_termios = termios.tcgetattr(fd)
    new_termios[3] &= ~(termios.ICANON | termios.ECHO)

    # set to new termios
    termios.tcsetattr(fd, termios.TCSADRAIN, new_termios)

    async def terminput(queue: asyncio.Queue):
        """Get terminal input and send it to the queue."""
        while True:
            ch = sys.stdin.read(1) # read a single char (works because of the termios config)

            if ch == "
":
                await queue.put(("finish", None)) # `None` here because we won't use the second argument
                await asyncio.sleep(0) # strange workaround so the queues actually work
                continue

            await queue.put(("input", ch))
            await asyncio.sleep(0) # strange workaround so the queues actually work

    async def timedsender(queue: asyncio.Queue):
        """Every 0.5 seconds, send a message to the queue."""
        while True:
            await queue.put(("message", "I'm a message!"))
            await asyncio.sleep(0.5)
    
    async def receiver(queue: asyncio.Queue):
        """Handle the receiving of messages and input characters."""
        # Late decision that I might be able to fix easily - I had to use a list to push characters into on a earlier version of the code. It can be a string now, though.
        input_buffer = []

        sys.stdout.write(prompt)
        sys.stdout.flush()

        def clear_line():
            """Clear the current line.

            There might be an escape code that does this already. Eh, anyways...
            """
            sys.stdout.write("
")
            sys.stdout.write(" " * os.get_terminal_size().columns)
            sys.stdout.write("
")
            sys.stdout.flush()

        def redraw_input_buffer():
            """Redraw the input buffer.

            Shows the prompt and what has been typed until now.
            """
            sys.stdout.write(prompt + "".join(input_buffer))
            sys.stdout.flush()

        while True:
            # So, lemme explain what this format is.
            # Each item sent on the queue should be a tuple.
            # The first element is what should be done with the content (such as show message, add to input buffer), and the second element is the content itself.
            kind, content = await queue.get()

            if kind == "message":
                clear_line()
                sys.stdout.write(f"Message -- {content}
")
                sys.stdout.flush()
                redraw_input_buffer()
            elif kind == "input":
                sys.stdout.write(content)
                sys.stdout.flush()
                input_buffer += content
            elif kind == "finish":
                sys.stdout.write("
")

                sys.stdout.write(f"INPUT FINISHED :: {repr(''.join(input_buffer))}
")
                sys.stdout.flush()
                
                input_buffer.clear()
                redraw_input_buffer()
                # continue reading more input lines...
            else:
                raise ValueError(f"Unknown kind: {repr(kind)}")

            queue.task_done()
    
    async def main():
        queue = asyncio.Queue()
    
        senders = [terminput(queue), timedsender(queue)]
        recv = receiver(queue)
        await asyncio.gather(*senders, recv)
    
        await queue.join()
        recv.cancel()

    try:
        asyncio.run(main())
    finally:
        # reset to original termios
        termios.tcsetattr(fd, termios.TCSADRAIN, orig_termios)

readline()

The main problem at question here is that the queue is only read when a character is typed, and even then, if I don't wait enough time to read the next char with, say, asyncio.sleep(0.1), usually just one message is received in the meantime.

I am not sure if the problem is the queue or some inner workings of the stdin-stdout mechanism (maybe I can't write to stdout while stdin is blocked).

question from:https://stackoverflow.com/questions/66056513/async-io-reading-char-from-input-blocks-output

Answer 1

Just figured out a solution for this problem - setting a max wait time for a character to be input.

At the top of readline():

def readline(prompt: str = "Input: "):
    fd = sys.stdin.fileno()
    orig_termios = termios.tcgetattr(fd)
    new_termios = termios.tcgetattr(fd)
    new_termios[3] &= ~(termios.ICANON | termios.ECHO)
    
    # the following lines were added:
    new_termios[6][termios.VMIN] = 0 # minimal amount of characters to
    new_termios[6][termios.VTIME] = 1 # a max wait time of 1/10 second

When using this directly on C, on timeout the character returned would be of code 170, but here not even that seems to happen (the read operations from Python might already ignore them).