I am not sure if this is the right place to raise this. I was following https://www.tensorflow.org/get_started/get_started and came across the following sample code:
W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W * x + b
In the section on loss function it has the following:
y = tf.placeholder(tf.float32)
squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)
print(sess.run(loss, {x:[1,2,3,4], y:[0,-1,-2,-3]}))
Why is value of y [0,-1,-2,-3]? Based on
linear_model = W * x + b,
y would be 0.3x - 0.3. So for x of [1,2,3,4], y should be [0,0.3,0.6,0.9].
Or am I missing something?
Yes, you are missing something. The goal of this exercise is to show that you first build up a graph (W*x+b=y), and then supply the variables for the placeholders.
To do this you supply x and y, and see what the difference is between wat you expected (the y variable) and what you got (the linear_model variable).
You are confusing the result of the equation with what the tutorial wanted to get out of this equation. If you go on in the tutorial they will probably learn you how you can train your weights to get the solution you expect.
Good luck!
In this code snippet, 'x' is the input and 'y' serves the purpose of label as you seem to already understand.
'W' and 'b' are variables that the program should 'learn' such that when x=[1,2,3,4], y ends up as [0,-1,-2,-3].
The values of 'W' and 'b' you see are the initial values.
Not included in this code is the update step where you will update the weights after computing the gradient based on a loss function. And after a few iterations, you should get your 'W' and 'b' such that when x=[1,2,3,4], you will get y=[0,-1,-2,-3]
Related
X and y are observations and target correspodingly.
logreg = LogisticRegression(random_state=0)
parameters_logreg = {'C': [0.1, 1, 0.5], 'max_iter': [100, 102] }
gs_logreg = GridSearchCV(logreg, parameters_logreg, cv = 5)
gs_logreg.fit(X, y)
cv_logreg = KFold(n_splits=5, shuffle=True, random_state=9)
cross_val_score(gs_logreg, X, y, cv=cv_logreg, scoring='roc_auc')
I am doing a classification problem using logistic regression. Applying grid search I find the best hyperparamenters. After that I calculate scores on cross validation folds.
My first question is will gs_logreg.fit(X, y) effect the final scores on cross validation? How cross_val_scores works? It fits gs_logreg ones more but now on cv folds? Can it remember y after first fiting gs_logreg.fit(X, y) ?
The second question is this code correct? I surprisingly got high scores for my stupid model.
To answer your first question, why not comment out that statement (gs_logreg.fit(X, y)), rerun and check if the cross validation results have changed? If they do change, then that statement affects the final scores, otherwise it does not.
when two variables are being evaluated TensorFlow,it is not supposed to reuse the values from the previous evaluation.(This is what was mentioned in Hands on Machine learning with sklearn and tensorflow by Aurélien Géron)
w = tf.constant(5)
x = w + 5
y = x**2 + 5
z = x**2 + 5
Take the piece of code mentioned above for example.
y and z should have the same value, if x is not modified in between their evaluation.
But I tried modifying their vales in-between the evaluation and still they have the same results.
with tf.compat.v1.Session() as sess:
a = y.eval()
x = w + 3
b = z.eval()
I am sorry if this is a really dumb question, but I just wanted to get my basics clear, it would really helpful if someone took time to explain this. Thanks
print(a)
# 366
print(b)
# 366
To clarify your doubts, in your code,
w = tf.constant(5)
x = w + 5
y = x**2 + 5
z = x**2 + 5
just creates a computation graph until you create a session, even the variables are not initialized yet.
In the following code,
with tf.compat.v1.Session() as sess:
a = y.eval()
x = w + 3
b = z.eval()
Once the session opened, It takes care of placing all the operations into the device you are computing with ( CPU or GPU). And it holds all the variables inside it.
Inside the Session, once eval is called Tensorflow automatically determines the nodes that it depends, and evaluates these nodes first, to explain you with both the scenario you have taken.
1. a = y.eval()
Here y is dependent on x and x is dependent on w, so it evaluates w first and evaluates x to calculate y.
b = z.eval()
Here z is dependent on x and x is dependent on w, once again from start it evaluates w first and evaluates x to calculate z. It does not reuse the result of x and w from eval done on y.
In both the eval the nodes are selected by Tensorflow for value x is the first instance.
Below is the Graph representation of computation. Where you can see the second x declaration is not part of the graph(right corner).
if you want to run the computation in the order you can use sess.run() instead of eval().
When sess.run() called, this method completes one set of computations in our graph in the following manner: it starts at the requested output(s) and then works backward, computing nodes that must be executed according to the set of dependencies.
Below is the modified example with the desired result.
import tensorflow as tf
w = tf.constant(5)
x = w + 5
y = x**2 + 5
x = w + 3
z = x**2 + 5
with tf.compat.v1.Session() as sess:
# a = y.eval()
# init.run()
# b = z.eval()
print(sess.run(z))
print(sess.run(y))
Output:
69
105
You can notice when sess.run(z) is requested, while going backward immediate x value is x = w + 3 , similarly for sess.run(y), while doing backward computation immediate x value is x = w + 5.
Suppose I have a matrix A and two vectors x,y, of appropriate dimensions. I want to compute the dot product x' * A * y, where x' denotes the transpose. This should result in a scalar.
Is there a convenient API function in Tensorflow to do this?
(Note that I am using Tensorflow 2).
Use tf.linalg.tensordot(). See the documentation
As you have mentioned in the question that you are trying to find dot product. In this case tf.matmul() will not work, as it is only for cross product of metrices.
Demo code snippet
import tensorflow as tf
A = tf.constant([[1,4,6],[2,1,5],[3,2,4]])
x = tf.constant([3,2,7])
result = tf.linalg.tensordot(tf.transpose(x), A, axes=1)
result = tf.linalg.tensordot(result, x, axes=1)
print(result)
And the result will be
>>>tf.Tensor(532, shape=(), dtype=int32)
Few points I want to mention here
Don't forget the axes argument inside tf.linalg.tensordot()
When you create tf.zeros(5) it will create a list of shape 5 and it will be like [0,0,0,0,0], when you transpose this it will give you the same list. But if you create it like tf.zeros((5,1)), it would be a vector of shape (5,1) and the result will be
[
[0],[0],[0],[0],[0]
]
Now you can transpose this and the result will be different, but I recommend you do the code snippet I have mentioned. In case of dot product you don't have to bother much about this.
If you are still facing issues, will be very happy to help you.
Just do the following,
import tensorflow as tf
x = tf.constant([1,2])
a = tf.constant([[2,3],[3,4]])
y = tf.constant([2,3])
z = tf.reshape(tf.matmul(tf.matmul(x[tf.newaxis,:], a), y[:, tf.newaxis]),[])
print(z.numpy())
Returns
>>> 49
Just use tf.transpose and multiplication operator like this:
tf.transpose(x)* A * y .
Based on your example:
x = tf.zeros(5)
A = tf.zeros((5,5))
How about
x = tf.expand_dims(x, -1)
tf.matmul(tf.matmul(x, A, transpose_a=True), x)
Consider for example this example:
train_op = opt.minimize(loss)
gradients = tf.gradients(loss, tf.trainable_variables())
Are the gradients computed twice or just once?
Or this example:
a = y + z
b = y + z
Is the addition y + z computed twice or just once?
It is computed only once. See this post for more info about this and other optimizations tensorflow does at runtime.
I'm pursuing course 2 on this coursera course on linear regression (https://www.coursera.org/specializations/machine-learning)
I've solved the training using graphlab but wanted to try out sklearn for the experience and learning. I'm using sklearn and pandas for this.
The model overfits on the data. How can I fix this? This is the code.
These are the coefficients i'm getting.
[ -3.33628603e-13 1.00000000e+00]
poly1_data = polynomial_dataframe(sales["sqft_living"], 1)
poly1_data["price"] = sales["price"]
model1 = LinearRegression()
model1.fit(poly1_data, sales["price"])
print(model1.coef_)
plt.plot(poly1_data['power_1'], poly1_data['price'], '.',poly1_data['power_1'], model1.predict(poly1_data),'-')
plt.show()
The plotted line is like this. As you see it connects every data point.
and this is the plot of the input data
I wouldn't even call this overfit. I'd say you aren't doing what you think you should be doing. In particular, you forgot to add a column of 1's to your design matrix, X. For example:
# generate some univariate data
x = np.arange(100)
y = 2*x + x*np.random.normal(0,1,100)
df = pd.DataFrame([x,y]).T
df.columns = ['x','y']
You're doing the following:
model1 = LinearRegression()
X = df["x"].values.reshape(1,-1)[0] # reshaping data
y = df["y"].values.reshape(1,-1)[0]
model1.fit(X,y)
Which leads to:
plt.plot(df['x'].values, df['y'].values,'.')
plt.plot(X[0], model1.predict(X)[0],'-')
plt.show()
Instead, you want to add a column of 1's to your design matrix (X):
X = np.column_stack([np.ones(len(df['x'])),df["x"].values.reshape(1,-1)[0]])
y = df["y"].values.reshape(1,-1)
model1.fit(X,y)
And (after some reshaping) you get:
plt.plot(df['x'].values, df['y'].values,'.')
plt.plot(df['x'].values, model1.predict(X),'-')
plt.show()