I have code here that I've modified from this website. Basically what I have written is this:
#import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
#from tensorflow.examples.tutorials.mnist import input_data
from edward.models import Categorical, Normal
import edward as ed
#ed.set_seed(39)
import pandas as pd
import csv
# Use the TensorFlow method to download and/or load the data.
with open ("data_final.csv", "r") as csvfile:
reader1 = csv.reader(csvfile)
data1 = np.array(list(reader1)).astype(np.float)
#mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
N = data1.shape[0] -1 # number of images in a minibatch.
D = 4 # number of features.
K = 4 # number of classes.
# Create a placeholder to hold the data (in minibatches) in a TensorFlow graph.
x = tf.placeholder(tf.float32, [N, D])
# Normal(0,1) priors for the variables. Note that the syntax assumes TensorFlow 1.1.
w = Normal(loc=tf.zeros([D, K]), scale=tf.ones([D, K]))
b = Normal(loc=tf.zeros(K), scale=tf.ones(K))
# Categorical likelihood for classication.
y =tf.matmul(x,w)+b
# Contruct the q(w) and q(b). in this case we assume Normal distributions.
qw = Normal(loc=tf.Variable(tf.random_normal([D, K])),
scale=tf.nn.softplus(tf.Variable(tf.random_normal([D, K]))))
qb = Normal(loc=tf.Variable(tf.random_normal([K])),
scale=tf.nn.softplus(tf.Variable(tf.random_normal([K]))))
# We use a placeholder for the labels in anticipation of the traning data.
y_ph = tf.placeholder(tf.float32, [N, K])
# Define the VI inference technique, ie. minimise the KL divergence between q and p.
inference = ed.KLqp({w: qw, b: qb}, data={y:y_ph})
# Initialse the infernce variables
inference.initialize(n_iter=5000, n_print=100, scale={y: 1})
# We will use an interactive session.
sess = tf.InteractiveSession()
# Initialise all the vairables in the session.
tf.global_variables_initializer().run()
I use the data linked here, to run the code. I get an error after less than a second of running the code (so I have a hard time believing this actually happened) that said:
ValueError: GraphDef cannot be larger than 2GB.
I think there were other topics with the same error as mine, but those people had instantiated like 1 million parameters of something. I have on the order to 20 parameters, so unsure why I'm getting this error.
In my case, there were still variables (and likely a graphs) that were not garbage collected from a previous Edward runs. Garbage collecting/resetting the console fixed the problem.
Related
I have tried the code below
import pandas as pd
from sklearn.linear_model import LinearRegression
import numpy as np
# Assign the dataframe to this variable.
# TODO: Load the data
bmi_life_data = pd.read_csv("bmi_and_life_expectancy.csv")
X= bmi_life_data['BMI'].values.reshape(-1,1)
y = bmi_life_data['Life expectancy'].values.reshape(-1,1)
# Make and fit the linear regression model
#TODO: Fit the model and Assign it to bmi_life_model
bmi_life_model = LinearRegression()
bmi_life_model.fit(X,y)
# Mak a prediction using the model
# TODO: Predict life expectancy for a BMI value of 21.07931
laos_life_exp = bmi_life_model.predict(21.07931)
but it gives me the error
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.
Even after reshaping it. I have tried to not reshape it but it still gives me the same error.
The error was in the prediction line
laos_life_exp = bmi_life_model.predict(21.07931)
should be
laos_life_exp = bmi_life_model.predict([[21.07931]])
to be of appropriate dimension
Thanks to #onyambu
I am trying to preprocess the infamous Titanic data (from Kaggle) by following this tutorial.
Everything was okay until I get to run the titanic_processing Model on the data (titanic_features) and I get this error:
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float).
In the tutorial it is mentioned that one should transform the data into a dict of tensors, but:
I don't see how the code (see HERE1 tag in my code below) makes a dict of tensors (there is no tf.convert_to_tensor for example)
I don't understand why one should retransform all the data as the previous code was suppose to do just that (when one create preprocessed_inputs etc.)
Here is my code, but you can also execute it on Google Colab here.
import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.layers.experimental import preprocessing
url = "https://raw.githubusercontent.com/aymeric75/IA/master/train.csv"
titanic = pd.read_csv(url)
titanic_features = titanic.copy()
titanic_labels = titanic_features.pop('Survived')
inputs = {}
for name, column in titanic_features.items():
dtype = column.dtype
if dtype == object:
dtype = tf.string
else:
dtype = tf.float32
inputs[name] = tf.keras.Input(shape=(1,), name=name, dtype=dtype)
numeric_inputs = {name:input for name,input in inputs.items()
if input.dtype==tf.float32}
x = layers.Concatenate()(list(numeric_inputs.values()))
norm = preprocessing.Normalization()
norm.adapt(np.array(titanic[numeric_inputs.keys()]))
all_numeric_inputs = norm(x)
preprocessed_inputs = [all_numeric_inputs]
for name, input in inputs.items():
if input.dtype == tf.float32:
continue
lookup = preprocessing.StringLookup(vocabulary=np.unique(titanic_features[name].dropna()))
one_hot = preprocessing.CategoryEncoding(max_tokens=lookup.vocab_size())
x = lookup(input)
x = one_hot(x)
preprocessed_inputs.append(x)
preprocessed_inputs_cat = layers.Concatenate()(preprocessed_inputs)
titanic_preprocessing = tf.keras.Model(inputs, preprocessed_inputs_cat)
titanic_features_dict = {}
# This model just contains the input preprocessing. You can run it to see what it does to your data.
# Keras models don't automatically convert Pandas DataFrames because
# it's not clear if it should be converted to one tensor or to a dictionary of tensors. So convert it to a dictionary of tensors:
# HERE1
titanic_features_dict = {name: np.array(value)
for name, value in titanic_features.items()}
features_dict = {name:values[:1] for name, values in titanic_features_dict.items()}
titanic_preprocessing(features_dict)
Thanks a lot for you support!
Aymeric
[UPDATE] if you can answer question 2 ("I don't understand why one should retransform all the data as the previous code was suppose to do just that (when one create preprocessed_inputs etc.") then I will validate your answer, because I think I need to reformat the input indeed (but I don't see what it the point of doing all the code before...)
In your case, the problem is caused by the fact that your feature "Cabin" contains some nan (Not a Number) values. Tensorflow is fine with nan in floating point and integer data types, but not for strings.
You can replace all those nan values with an empty strings in your pandas dataframe :
titanic_features["Cabin"] = titanic_features["Cabin"].fillna("")
The previous code simply declares a preprocessing function as a keras model. You don't actually preprocess any data until your call to the titanic_preprocessing model.
I understand you can assign a batch size to a Dataset and return a new dataset object. Is there an API to interrogate the batch size given a dataset object?
I am trying to find the calls at:
https://www.tensorflow.org/api_docs/python/tf/data/Dataset
when you call the .batch(32) method , it returns an tensorflow.python.data.ops.dataset_ops.BatchDataset object. As documented in Tensorflow Documentation This kind of object has private attribute called ._batch_size which contain a tensor of batch_size.
In tensorflow 2.X you need just call .numpy() method of this tensor to convert it to numpy.int64 type.
In tensorflow 1.X you need to cal .eval() method.
I do not know if you can just get it as an attribute, but you could just iterate through the dataset once and print the shape:
# create a simple tf.data.Dataset with batchsize 3
import tensorflow as tf
f = tf.data.Dataset.range(10).batch(3) # Dataset with batch_size 3
# iterating once
for one_batch in f:
print('batch size:', one_batch.shape[0])
break
If you know your dataset has targets/labels as well, you have to iterate as follows:
# iterating once
for one_batch_x, one_batch_y in f:
print('batch size:', one_batch_x.shape[0])
break
In both cases, it will print:
batch size: 3
In Tensorflow 1.* access batch_size via dataset._dataset._batch_size:
import tensorflow as tf
import numpy as np
print(tf.__version__) # 1.14.0
dataset = tf.data.Dataset.from_tensor_slices(np.random.randint(0, 2, 100)).batch(10)
with tf.compat.v1.Session() as sess:
batch_size = sess.run(dataset._dataset._batch_size)
print(batch_size) # 10
In Tensorflow 2 you can access via dataset._batch_size:
import tensorflow as tf
import numpy as np
print(tf.__version__) # 2.0.1
dataset = tf.data.Dataset.from_tensor_slices(np.random.randint(0, 2, 100)).batch(10)
batch_size = dataset._batch_size.numpy()
print(batch_size) # 10
In the following example, every time I run sess.run([image, label]), a different sample from the queue is returned, thus a different np_image is returned.
Is there a way that I can let the slim.queues.QueueRunners know that I want to use (run) the same sample multiples before a dequeue operation takes place?
The reason I ask is that I have a large op that doesn't fit in my VRAM. I have to break the large op into several small ops and feed a different feed_dict every time a small ops is runned. However, when I run the small op, image changes which break the code. Putting all the small ops in a list and run the list at the same time doesn't work for me because the VRAM size is the limitation.
Thanks!
import tensorflow as tf
import numpy as np
slim = tf.contrib.slim
from datasets import dataset_utils
from tensorflow.python.ops import control_flow_ops
from datasets import dataset_factory
from deployment import model_deploy
from nets import nets_factory
from preprocessing import preprocessing_factory
with tf.Graph().as_default():
dataset = dataset_factory.get_dataset('cifar10', 'train','/home/user/dataset/cifar10')
provider = slim.dataset_data_provider.DatasetDataProvider(
dataset,
num_readers=1,
common_queue_capacity=256,
common_queue_min=128)
[image, label] = provider.get(['image', 'label'])
image_preprocessing_fn = preprocessing_factory.get_preprocessing(
'cifarnet',
is_training=True)
images, labels = tf.train.batch([image, label],
batch_size=32,
num_threads=1,
capacity=64)
with tf.Session() as sess:
with slim.queues.QueueRunners(sess):
for i in range(3):
#in every iteration, the tensor 'image' will be different
#the np_image value will be different as well
np_image, np_label = sess.run([image, label])
Peek operation for queues is currently not supported, for discussion see
https://github.com/tensorflow/tensorflow/issues/7880
A work-around is to restructure your code to take values from tf.Variable objects rather than from tf.dequeue. IE something like this
x = tf.Variable(queue.dequeue())
y = x+2
sess.run(x.initializer)
sess.run(y)
sess.run(y) # same value
sess.run(x.initializer)
sess.run(y) # new value
So here is an example of using batch normalization over a 1-D input vector. Batch normalization is performed over 100 training examples xTr. I then want to test on say just 1 example later on xTe.
import tensorflow as tf
import numpy as np
from tensorflow.contrib.layers import layers
if __name__ == "__main__":
bn = layers.batch_norm
nFeats = 3
nObs = 100
xTr = np.random.rand(nObs,nFeats) # Train
xTe = np.random.rand(1,nFeats) # Test
bnTrain = tf.placeholder(tf.bool)
X = tf.placeholder(tf.float32,[None,nFeats])
Y = bn(X,nFeats,is_training=bnTrain) # want to be able to change is_training via a feed_dict.
init_op = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init_op)
yTr_ = Y.eval(feed_dict={X:xTr,bnTrain:True})
yTe_ = Y.eval(feed_dict={X:xTe,bnTrain:False})
But I can't pass a tf.Tensor to a function expecting a normal python bool. What is the best way of going about this so I can change a bool during a session.
The current implementation of the tf.contrib.layers.batch_norm() function is designed to accept a tf.Tensor as the is_training argument (although this fact doesn't appear to be documented), and looking at the revision history, it was added in the TensorFlow 0.10 release. If you are using an older version, please try upgrading to the latest release (currently 0.12), and your existing code should work. Among other improvements, it contains a fused implementation of batch normalization that should make a significant performance improvement.