python - Occasional ctypes error importing numpy from mod_wsgi django app

Question

Here's the setup:

• Django (1.2) app on mod_wsgi that imports ctypes
• Python 2.6.5
• Apache 2.2.3
• SELinux disabled
• RedHat EL 5 64bit
• some of the file system is mounted over nfs

Occasionally, when I restart apache I get an import error when it's trying to import ctypes. Every incoming request fails with a 500 error. If I restart apache usually everything just starts working again.

Here's the stack trace of the error:

Traceback (most recent call last):

 File "/home/appfirst/django/django/core/handlers/base.py", line 80, in get_response
   response = middleware_method(request)

     -------------- A BUNCH OF DJANGO MIDDLEWARE STUFF HERE -------------

 File "/home/appfirst/django/django/utils/importlib.py", line 35, in import_module
   __import__(name)

 File "/home/appfirst/backend/backend/streamer/views.py", line 6, in <module>
   import appfirst.main.models as FEmodels

 File "/home/appfirst/frontend/appfirst/main/models.py", line 27, in <module>
   import numpy, math, mpmath

 File "/usr/lib64/python2.6/site-packages/numpy/__init__.py", line 43, in <module>
   import ctypeslib

 File "/usr/lib64/python2.6/site-packages/numpy/ctypeslib.py", line 9, in <module>
   import ctypes

 File "/usr/lib64/python2.6/ctypes/__init__.py", line 546, in <module>
   CFUNCTYPE(c_int)(lambda: None)

MemoryError

I thought it might be related to this bug, but I have SELinux turned off which I thought would mean this case could never occur:

• http://bugs.python.org/issue5504

Any suggestions on how to reproduce it consistently and/or fix it? This is really stumping me!

Answer 1

I've run into this bug too. In my case it occurs when I exec a Python script from within a PHP script running under Apache on a 64-bit Linux system. [The Python code being run is the front-end to a pypy sandbox.] The same bit of code works fine on a 32-bit system and even works fine when the PHP script is executed directly from the command line. My "fix" has been simply to comment out that line "CFUNCTYPE(c_int)(lambda: None)" in ctypes/init.py. It's the last line of the file and is preceded by the following comment, showing that the programmer doesn't understand what's going on, either!

# XXX for whatever reasons, creating the first instance of a callback
# function is needed for the unittests on Win64 to succeed.  This MAY
# be a compiler bug, since the problem occurs only when _ctypes is
# compiled with the MS SDK compiler.  Or an uninitialized variable?
CFUNCTYPE(c_int)(lambda: None)

Clearly there's a deeper problem somewhere in cpython, but the fix works for me.