python - How to visualize training process with output per patch/epoch?

Question

My neural network in Keras learns a representation of my original data. In order to see exactly how it learns I thought it would be interesting to plot the data for every training batch (or epoch alternatively) and convert the plots into a video.

I'm stuck on how to get the outputs of my model during the training phase.

I thought about doing something like this (pseudo code):

epochs = 200
plt_outputs = []
for i in range(epochs):
    model.fit(x_train,y_train, epochs = 1)
    plt_outputs.append(output_layer(x_test))

where output_layer is the layer in my neural network I'm interested in. Afterwards I would use plot_data to generate each plot and turn it into a video. (That part I'm not concerned about yet..)

But that doesn't strike me as a good solution, plus I don't know how get the output for every batch. Any thoughts on this?

question from:https://stackoverflow.com/questions/66048377/how-to-visualize-training-process-with-output-per-patch-epoch

Answer 1

You can customize what happens in the test step, much like this official tutorial:

import tensorflow as tf
import numpy as np

class CustomModel(tf.keras.Model):
    def test_step(self, data):
        # Unpack the data
        x, y = data
        # Compute predictions
        y_pred = self(x, training=False)

        test_outputs.append(y_pred) # ADD THIS HERE

        # Updates the metrics tracking the loss
        self.compiled_loss(y, y_pred, regularization_losses=self.losses)
        # Update the metrics.
        self.compiled_metrics.update_state(y, y_pred)
        # Return a dict mapping metric names to current value.
        # Note that it will include the loss (tracked in self.metrics).
        return {m.name: m.result() for m in self.metrics}


# Construct an instance of CustomModel
inputs = tf.keras.Input(shape=(8,))
x = tf.keras.layers.Dense(8, activation='relu')(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = CustomModel(inputs, outputs)
model.compile(loss="mse", metrics=["mae"], run_eagerly=True)

test_outputs = list() # ADD THIS HERE

# Evaluate with our custom test_step
x = np.random.random((1000, 8))
y = np.random.random((1000, 1))
model.evaluate(x, y)

I added a list, and now in the test step, it will append this list with the output. You will need to add run_eagerly=True in model.compile() for this to work. This will output a list of such outputs:

<tf.Tensor: shape=(32, 1), dtype=float32, numpy=
array([[ 0.10866462],
       [ 0.2749035 ],
       [ 0.08196291],
       [ 0.25862294],
       [ 0.30985728],
       [ 0.20230596],
            ...
       [ 0.17108777],
       [ 0.29692617],
       [-0.03684975],
       [ 0.03525433],
       [ 0.26774448],
       [ 0.21728781],
       [ 0.0840873 ]], dtype=float32)>