python - Why does Paramiko hang if you use it while loading a module?

Question

Put the following into a file hello.py (and easy_install paramiko if you haven't got it):

hostname,username,password='fill','these','in'
import paramiko
c = paramiko.SSHClient()
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
c.connect(hostname=hostname, username=username, password=password)
i,o,e = c.exec_command('ls /')
print(o.read())
c.close()

Fill in the first line appropriately.

Now type

python hello.py

and you'll see some ls output.

Now instead type

python

and then from within the interpreter type

import hello

and voila! It hangs! It will unhang if you wrap the code in a function foo and do import hello; hello.foo() instead.

Why does Paramiko hang when used within module initialization? How is Paramiko even aware that it's being used during module initialization in the first place?

Answer 1

Paramiko uses separate threads for the underlying transport. You should never have a module that spawns a thread as a side effect of importing. As I understand it, there is a single import lock available, so when a child thread from your module attempts another import, it can block indefinitely, because your main thread still holds the lock. (There are probably other gotchas that I'm not aware of too)

In general, modules shouldn't have side effects of any sort when importing, or you're going to get unpredictable results. Just hold off execution with the __name__ == '__main__' trick, and you'll be fine.

[EDIT] I can't seem to create a simple test case that reproduces this deadlock. I still assume it's a threading issue with import, because the auth code is waiting for an event that never fires. This may be a bug in paramiko, or python, but the good news is that you shouldn't ever see it if you do things correctly ;)

This is a good example why you always want to minimize side effects, and why functional programming techniques are becoming more prevalent.