python - How to use dill to serialize a class definition?

Question

In the answer to Python pickle: dealing with updated class definitions, the author of the dill package writes:

"Ok, I have added this feature to dill in the latest revision on github. Implemented with far less trickery than I thought... just serialize the class definition with the pickle, and voila."

Having installed dill and tinkered with it, it's not obvious to me how to actually use this functionality in dill. Could someone provide an explicit example? I would like to pickle the class instance and also serialize the class definition.

(I am new to python and I this functionality seems extremely important, as since when pickling an object it would be great to get as close to a guarantee as possible that you could look at the object (could be the result of a simulation) in the future after the class definition may have changed and you haven't kept track of all the changes in an easily accessible way.)

Answer 1

I think you are looking for one of the following functionalities…

Here I build a class, and an instance, and then change the class definition. The pickled class and instance still are unpicklable because dill pickles the source code for the class by default… and manages having several classes with the same name in the namespace (it does this simply by managing the pointer references to the class definitions).

Python 2.7.8 (default, Jul 13 2014, 02:29:54) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>> 
>>> class Foo(object):
...   def bar(self, x):
...     return x+self.y       
...   y = 1
... 
>>> f = Foo()
>>> _Foo = dill.dumps(Foo)
>>> _f = dill.dumps(f)
>>> 
>>> class Foo(object):
...   def bar(self, x):
...     return x*self.z  
...   z = -1 
... 
>>> f_ = dill.loads(_f, ignore=True)
>>> f_.y
1
>>> f_.bar(1)
2
>>> Foo_ = dill.loads(_Foo)
>>> g = Foo_()
>>> g.bar(1)
2

Pickle would blow up on the above. If you don't want dill to serialize the class explicitly, and to do what pickle does, then you can ask dill to pickle by reference with dill.dumps(Foo, byref=True). Alternately, you can dynamically decide to ignore the newly-defined class by using ignore=False (the default).

Now, in the case below, we work with the new class definition, and extract the source from the object, then save it to a file. Additionally, we can dump the source to a file (here I use a temporary file) so it can be imported later.

>>> sFoo = dill.source.getsource(Foo)
>>> print sFoo
class Foo(object):
  def bar(self, x):
    return x*self.z
  z = -1

>>> open('myFoo.py', 'w').write(sFoo)    
>>>
>>> f = dill.temp.dump_source(Foo, dir='.')
>>> f.name
'/Users/mmckerns/dev/tmpM1dzYN.py'
>>> from tmpM1dzYN import Foo as _Foo_
>>> h = _Foo_()
>>> h.bar(2)
-2
>>> from myFoo import Foo as _SFoo_
>>> _SFoo_.z
>>> -1
>>> 

I hope that helps.