python - How does long-polling work in Tornado?

Question

In Tornado's chat demo, it has a method like this:

@tornado.web.asynchronous
def post(self):
    cursor = self.get_argument("cursor", None)
    global_message_buffer.wait_for_messages(self.on_new_messages,
                                            cursor=cursor)

I'm fairly new to this long polling thing, and I don't really understand exactly how the threading stuff works, though it states:

By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections...

My theory was that by making a simple app:

import tornado.ioloop
import tornado.web
import time

class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        print("Start request")
        time.sleep(4)
        print("Okay done now")
        self.write("Howdy howdy howdy")
        self.finish()

application =  tornado.web.Application([
    (r'/', MainHandler),
])

That if I made two requests in a row (i.e. I opened two browser windows and quickly refreshed both) I would see this:

Start request
Start request
Okay done now
Okay done now

Instead, I see

Start request
Okay done now
Start request
Okay done now

Which leads me to believe that it is, in fact, blocking in this case. Why is it that my code is blocking, and how do I get some code to do what I expect? I get the same output on Windows 7 with a core i7, and a linux Mint 13 box with I think two cores.

Edit:

I found one method - if someone can provide a method that works cross-platform (I'm not too worried about performance, only that it's non-blocking), I'll accept that answer.

Answer 1

The problem with the code in original question is that when you call time.sleep(4) you are effectively blocking the execution of event loop for 4 seconds. And accepted answer doesn't solve the problem either (IMHO).

Asynchronous serving in Tornado works on trust. Tornado will call your functions whenever something happens, but it trusts you that you will return control to it as soon as possible. If you block with time.sleep() then this trust is breached - Tornado can't handle new connections.

Using multiple threads only hides the mistake; running Tornado with thousands of threads (so you can serve 1000s of connections simultaneously) would be very inefficient. The appropriate way is running a single thread which only blocks inside Tornado (on select or whatever Tornado's way of listening for events is) - not on your code (to be exact: never on your code).

The proper solution is to just return from get(self) right before time.sleep() (without calling self.finish()), like this:

class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        print("Starting")

You must of course remember that this request is still open and call write() and finish() on it later.

I suggest you take a look at chat demo. Once you strip out the authentication you get a very nice example of async long polling server.