python - ValueError: logits and labels must have the same shape ((None, 10, 82) vs (None, 1))

Question

I am training an LSTM network but I have an error of

ValueError: logits and labels must have the same shape ((None, 10, 82) vs (None, 1))

I do not know where the error in the input shape is coming from. Any help would be highly appreciated. Thanks!

# The next step is to split training and testing data. For this we will use sklearn function train_test_split().
features_train, features_test, labels_train, labels_test = train_test_split(features, labels, test_size=0.2)

# features and labels shape
features_train = features_train.reshape(len(features_train), 1, features_train.shape[1])

features_train.shape

(180568, 1, 82)

model = Sequential()
model.add(LSTM(10, input_shape=(features_train.shape[1:])))
model.add(Embedding(180568, 82))
model.add(Dense(67, activation='softmax'))
model.add(Dropout(0.2))
model.add(Activation('sigmoid'))
model.build()

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()

Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lstm_3 (LSTM)                (None, 10)                3720      
_________________________________________________________________
embedding_3 (Embedding)      (None, 10, 82)            14806576  
_________________________________________________________________
dropout_3 (Dropout)          (None, 10, 82)            0         
_________________________________________________________________
activation_3 (Activation)    (None, 10, 82)            0         
=================================================================
Total params: 14,810,296
Trainable params: 14,810,296
Non-trainable params: 0

history = model.fit(features_train,
                    labels_train,
                    epochs=15,
                    batch_size=128,
                    validation_data=(features_test, labels_test))

Answer 1

1. Remove the reshaping operation
2. Put the embedding layer before the LSTM layer
3. Remove dropout after the output layer (this makes no sense to me)
4. Remove the sigmoid activation after softmax activation

import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import *

model = Sequential()
model.add(Embedding(180568, 82))
model.add(LSTM(10))
model.add(Dense(67, activation='softmax'))
# model.add(Dropout(0.2))
# model.add(Activation('sigmoid'))
model.build()

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()

model(tf.random.uniform((10, 82))).shape

TensorShape([10, 67])

Change your loss function to "sparse_categorical_crossentropy", as your labels appear to be integers.