python - TensorFlow: How to handle void labeled data in image segmentation?

Question

I was wondering how to handle not labeled parts of an image in image segmentation using TensorFlow. For example, my input is an image of height * width * channels. The labels are too of the size height * width, with one label for every pixel.

Some parts of the image are annotated, other parts are not. I would wish that those parts have no influence on the gradient computation whatsoever. Furthermore, I am not interested in the network predicting this “void” label.

Is there a label or a function for this? At the moment I am using tf.nn.sparse_softmax_cross_entropy_with_logits.

Answer 1

I'm not 100% familiar with TF. However, have you considered using the weights parameter of the loss?
Looking at tf.loses.sparse_softmax_cross_entropy it has a parameter weights

weights: Coefficients for the loss. This must be scalar or of same rank as labels

You can set weightof "void" pixels to zero, thus making the loss ignore them.

You can also remove the reduction from tf.nn.sparse_softmax_cross_entropy_with_logits and use tf.losses.compute_weighted_loss to perform the weighting.