Hi, unfortunately I cannot give a very clear bug description, because I get different errors every time.
However:

Using the simplest imaginable PythonLayer and multi-gpu with GPUs 0,1 selected, segfaults in boost Python library pop up. Sometimes I get errors from the garbage collector Fatal Python error: GC object already tracked

This seems to be because of parallelizing the Python code in multiple threads.
When I set share_in_parallel: true to serialize Python execution, everything works fine.

The Python layer I use:

import caffe
import numpy as np

class PythonLayer(caffe.Layer):

    def setup(self, bottom, top):
        if len(bottom) != 1:
            raise Exception("Need one input.")

    def reshape(self, bottom, top):
        top[0].reshape(bottom[0].num, bottom[0].channels, bottom[0].height, bottom[0].width)

    def forward(self, bottom, top):
        top[0].data[...] = bottom[0].data

    def backward(self, top, propagate_down, bottom):
        if propagate_down[0]:
            bottom[0].diff[...] = top[0].diff