How is this function programatically building a LSTM - tensorflow

Question

Here is the code:
def lstm(o, i, state):
#these are all calculated seperately, no overlap until....
#(input * input weights) + (output * weights for previous output) + bias
input_gate = tf.sigmoid(tf.matmul(i, w_ii) + tf.matmul(o,w_io) + b_i)
#(input * forget weights) + (output * weights for previous output) + bias
output_gate = tf.sigmoid(tf.matmul(i, w_oi) + tf.matmul(o,w_oo) + b_o)
#(input * forget weights) + (output * weights for previous output) + bias
forget_gate = tf.sigmoid(tf.matmul(i, w_fi) + tf.matmul(o,w_fo) + b_f)
memory_cell = tf.sigmoid(tf.matmul(i, w_ci) + tf.matmul(o,w_co) + b_c)
state = forget_gate * state + input_gate * memory_cell
output = output_gate * tf.tanh(state)
return output, state
And here is the drawing of the lstm:
I'm having trouble understanding how the two match up. Any help would be much appreciated.

Answer 1

This is an excellent blogpost on LSTMs. This code is directly implementing the LSTM; the code here is equivalent to the equations listed on Wikipedia:
The input & output weights reflect the state of the network. In a simple fully-connected (FC) layer, we'd only have one weight matrix, which is what we would use to calculate the output of the layer:
The advantage of a LSTM, however, is that it includes multiple sources of information, or state; this is what we refer to when we say that a LSTM has memory. We have the output gate, just like the FC layer, but we also have the forget gate, the input gate, the cell state, and the hidden state. These all combine to provide multiple, different, sources of information. The equations show how they come together to produce the output.
In the equations, x is the input, and f_t is the input gate. I would recommend reading the linked blogpost and the Wikipedia article to get an understanding of how the equations implement a LSTM.
The image depicts the input gate providing output to the cell based on the values from previous cells, and previous values of the input gate. The cell also incorporates the forget gate; the inputs are then fed into the output gate, which also takes previous values of the output gate as inputs.