Keras: Why is LSTM much faster than SimpleRNN during training - tensorflow

Question

When I tried using SimpleRNN vs LSTM, I found the SimpleRNN training had an ETA of 30 min, whereas the LSTM had ETA of 20 seconds. But SimpleRNN should have less operations than LSTM. What is causing this huge difference? Am I using SimpleRNN wrong?
import tensorflow as tf
SEQUENCE_LENGTH = 80
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.imdb.load_data(num_words = 2000)
x_train = tf.keras.preprocessing.sequence.pad_sequences(x_train, maxlen=SEQUENCE_LENGTH)
x_test = tf.keras.preprocessing.sequence.pad_sequences(x_test, maxlen=SEQUENCE_LENGTH)
model = tf.keras.models.Sequential([
tf.keras.layers.Embedding(2000, 128),
tf.keras.layers.SimpleRNN(8),
# tf.keras.layers.LSTM(8),
tf.keras.layers.Dense(1, activation="sigmoid"),
])
model.compile(
loss="binary_crossentropy",
optimizer="adam",
metrics=["accuracy"],
)
model.fit(x_train, y_train, batch_size=32, epochs=1)

Answer 1

Simple RNN :- There is simple multiplication of Input (xt) and Previous Output (ht-1),passed through Tanh activation function. No Gates present.
Recurrent neural networks(RNN) have a recurrent connection in which the output is transmitted back to the RNN neuron rather than only passing it to the next node.
Each node in the RNN model functions as a
memory cell, continuing calculation and operation implementation. An
RNN remembers every piece of information throughout time. RNNs have feedback loops in the recurrent layer. This lets them maintain information in ‘memory’ over time. But, it can be difficult to train standard RNNs to solve problems that require learning long-term temporal dependencies.This is
because the gradient of the loss function decays exponentially with
time (called the vanishing gradient problem).
LSTM : - LSTMs deal with vanishing and exploding gradient problem by introducing new gates, such as input(i) and forget(f) gates, which allow for a better control over the gradient flow that update and regulate the cell states in an LSTM network and enable better preservation of “long-range dependencies”.
LSTM tackles gradient vanishing by ignoring useless data/information
in the network. If there is no valuable data from other inputs
(previous words of the sentence), LSTM will forget that data and
produce the result “Cut down the budget.
It contains four networks activated by either the sigmoid function (σ)
or the tanh function, all with their own different set of parameters.
Forget gate layer (f):-Decides which information to forget from the cell state
Input gate layer (i):-This could also be a remember gate. It decides which of the new candidates are relevant for this time step
New candidate gate layer (n):- Creates a new set of candidates to be stored in the cell state
Output gate layer (o):- Determines which parts of the cell state are output.
Please check this link for better understanding in this.