There are quite a few implementations for this problem, but none of them worked quite as well. picloud's cloudpickle is great!

The problem: You have a python function on one computer that you want to execute on the other. The naive way is rather simple, but you quickly run into problems.

On the sender:

send(marshal.dumps(func.func_code))

On the receiver:

func_code = marshal.loads(receive())
types.FunctionType(func_code)

But now you run into the problem that FunctionType takes a second argument, globals, which is to include anything the function you just received might attempt to use. If your function was alone in a module, you can likely just pass in the builtins, but if it has anything else, it gets trickier. What if it was a closure? Now it's just a lonely function that's going to barf when it attempts to access variables no longer in its scope.

Instead of writing the umpteenth shoddy implementation of this, I started looking around for existing ones. While I don't normally consider importing a huge, all-inclusive library like picloud, I thought this was a hard enough problem to warrant it, and I was right. They somehow manage to stick a function complete with its pointers to external information inside the pickle protocol, which is exactly what I was looking for.

Here's an example:

# sender.py
global_arg = 2
def closure_maker(closure_arg):
def closure_func(own_arg):
return own_arg + closure_arg + global_arg + 1
return closure_func

func = closure_maker(3)

from cloud.serialization import cloudpickle
result = cloudpickle.dumps(func)
with open('/tmp/pickled', 'w') as dest:
dest.write(result)

And the receiver

# receiver.py
from cloud.serialization import cloudpickle

with open('/tmp/pickled') as source:
result = source.read()
func = cloudpickle.loads(result)

assert func(4) == 10

If you run sender.py then receiver.py, you'll see that func(4) indeed returned 10, which uses all 3 variables available. Also, cloudpickle.loads is just pickle.loads, so your receiver doesn't even have to have the picloud library.

Normal warnings apply with using the pickle protocol here. Don't unpickle anything from an untrusted source, be that a user or an unauthenticated computer. The function could do anything, such as shutil.rmtree('/', True), so watch out. The mere act of unpickling can cause problems as pickle, refer to the pickle documentation for more.