python - Can't pickle Function

Question

So I'm trying to speed up my computation time by doing a little bit multiprocessing

I'm trying to use the pool workers.

At the top of my code I have

import Singal as s
import multiprocessing as mp
def wrapper(Channel):
    Noise_Frequincies = []
    for i in range(1,125):
        Noise_Frequincies.append(60.0*float(i))
    Noise_Frequincies.append(180.0)
    filter1 = s.Noise_Reduction(Sample_Rate,Noise_Frequincies,Channel)
    return filter1

Then when the time comes I use

Both_Channels = [Chan1, Chan2]
results = mp.Pool(2).map(wrapper,Both_Channels)
filter1 = results[0]
filter2 = results[1]

I get the following error

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner
self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 761, in run
self.__target(*self.__args, **self.__kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 342, in _handle_tasks
put(task)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Chan1 and Chan2 are arrays of my signal, and I'm trying to filter out some noise out of each. I'm new to multiprocessing so I apologize if this is a dumb error

Answer 1

I marked this question as dup of that Q/A, and to sum up: you can't pickle functions, and that's what you're trying to do when you pass wrapper() to Pool().map(). To pickle functions, you need to use copy_reg, as shown by this example.

" Classes, functions, and methods cannot be pickled -- if you pickle an object, the object's class is not pickled, just a string that identifies what class it belongs to. " (cf doc)

• http://docs.python.org/2/library/copy_reg.html?highlight=copyreg
• http://bytes.com/topic/python/answers/552476-why-cant-you-pickle-instancemethods

I don't use custom classes, and that's where his issue is (at least as far as I can tell)

no, his problem that he was trying to pickle an instance method, which is relatively close to a function, as both can't be pickled. And the workaround for that A should work for you too.

Though I did not test it…

HTH