python - How to tell for which object attribute pickle fails?

Question

When you pickle an object that has some attributes which cannot be pickled it will fail with a generic error message like:

PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed

Is there any way to tell which attribute caused the exception? I am using Python 2.5.2.

Even though I understand in principle the root cause of the problem (e.g. in the above example having an instance method) it can still be very hard to exactly pinpoint it. In my case I already defined a custom __getstate__ method, but forgot about a critical attribute. This happened in a complicated structure of nested objects, so it took me a while to identify the bad attribute.

As requested, here is one simple example were pickle intentionally fails:

import cPickle as pickle
import new

class Test(object):
    pass

def test_func(self):
    pass

test = Test()
pickle.dumps(test)
print "now with instancemethod..."
test.test_meth = new.instancemethod(test_func, test)
pickle.dumps(test)

This is the output:

now with instancemethod...
Traceback (most recent call last):
  File "/home/wilbert/develop/workspace/Playground/src/misc/picklefail.py", line 15, in <module>
    pickle.dumps(test)
  File "/home/wilbert/lib/python2.5/copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects

Unfortunately there is no hint that the attribute test_meth causes the problem.

Answer 1

You could file a bug against Python for not including more helpful error messages. In the meantime, modify the _reduce_ex() function in copy_reg.py.

if base is self.__class__:
    print self # new   
    raise TypeError, "can't pickle %s objects" % base.__name__

Output:

<bound method ?.test_func of <__main__.Test object at 0xb7f4230c>>
Traceback (most recent call last):
  File "nopickle.py", line 14, in ?
    pickle.dumps(test)
  File "/usr/lib/python2.4/copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects