Tensorflow- estimation of basic linear equation using neural net - tensorflow

Question

I am trying to learn Tensorflow. I am doing a basic example- model the equation y = x + 0.1, train it using a neural net, and then make predictions. I am actually taking a sigmoid approach (not ideal), so not using the standard softmax/relu way (which didn't work for me). The code runs, but the answer is wrong: all predictions in a batch give nearly identical answers, like y_true = [[0.356], [0.356], [0.356],[0.356]], for input= [[0.1, 0.2, 0.3, 0.4]]. What am I doing wrong? Code is below:
import tensorflow as tf
import numpy as np
epochs = 1000
# For equation y = b + 0.1, sample data below
myImportedDatax_np = np.array([[.1],[.2],[.3],[.4]],dtype=float)
myImportedDatay_np = np.array([[.2],[.3],[.4],[.5]],dtype=float)
c = tf.constant(0.1, name='c')
b = tf.placeholder(tf.float32, [None, 1], name='b')
y = tf.add(b, c, name='y')
y_true = tf.placeholder(tf.float32, [None, 1], name='y_true')
W1 = tf.Variable(tf.random_normal([1, 3], stddev=0.03), name='W1')
b1 = tf.Variable(tf.random_normal([3]), name='b1')
W2 = tf.Variable(tf.random_normal([3, 1], stddev=0.03), name='W2')
b2 = tf.Variable(tf.random_normal([1]), name='b2')
hidden_out = tf.add(tf.matmul(b, W1), b1)
hidden_out = tf.sigmoid(hidden_out)
y_ = tf.sigmoid(tf.add(tf.matmul(hidden_out, W2), b2))
cost = tf.reduce_mean(tf.square(y_ - y_true))
optimiser = tf.train.GradientDescentOptimizer(0.005).minimize(cost)
init_op = tf.initialize_all_variables()
with tf.Session() as sess:
# initialise the variables
sess.run(init_op)
for epoch in range(epochs):
_, cost_now = sess.run([optimiser, cost], {b: myImportedDatax_np, y_true: myImportedDatay_np})
print("Predicted values are:")
print(sess.run(y_, {b: myImportedDatax_np}))

Answer 1

There few things that are wrong with your code:
Yours is a regression problem, y = x + c, so remove the sigmoid output:
y_ = tf.add(tf.matmul(hidden_out, W2), b2)
You will be better served by a single hidden layer, your multiple hidden unit for such a simple task will require it to train it longer.
To handle 2, increase your epoch to higher value say, 10000 and your learning rate also higher, say 0.1
EDIT:
Adding the code:
#increased the number of epoch
epochs = 10000
# For equation y = b + 0.1, sample data below
myImportedDatax_np = np.array([[.1],[.2],[.3],[.4]],dtype=float)
myImportedDatay_np = np.array([[.2],[.3],[.4],[.5]],dtype=float)
c = tf.constant(0.1, name='c')
b = tf.placeholder(tf.float32, [None, 1], name='b')
y = tf.add(b, c, name='y')
y_true = tf.placeholder(tf.float32, [None, 1], name='y_true')
W1 = tf.Variable(tf.random_normal([1, 3], stddev=0.03), name='W1')
b1 = tf.Variable(tf.random_normal([3]), name='b1')
W2 = tf.Variable(tf.random_normal([3, 1], stddev=0.03), name='W2')
b2 = tf.Variable(tf.random_normal([1]), name='b2')
hidden_out = tf.add(tf.matmul(b, W1), b1)
hidden_out = tf.sigmoid(hidden_out)
# Removed the activation
y_ = tf.add(tf.matmul(hidden_out, W2), b2)
cost = tf.reduce_mean(tf.square(y_ - y_true)
#changed the learning rate
optimiser = tf.train.GradientDescentOptimizer(0.1).minimize(cost)
init_op = tf.global_variables_initializer()
#Predicted values are:
#[[ 0.19917184]
#[ 0.30153054]
#[ 0.40164429]
#[ 0.4976812 ]]