Tensorflow Tensor out of CSV has no size? - tensorflow

I just can't get any dimensions (size, lenght) out of this damn tensor "datatens". here is the code and the error message:
import tensorflow as tf
import numpy as np
import tflearn
import pandas as pd
from tensorflow import keras
file = 'some.csv'
record_defaults = [tf.float64]*18
from tflearn.data_utils import load_csv
data , label = load_csv(file, target_column=0,has_header=True,
categorical_labels=True, n_classes=50)
datatens = tf.data.Dataset.from_tensor_slices((data,label))
print(datatens.get_shape().as_list())
ERROR:
<TensorSliceDataset shapes: ((17,), (50,)), types: (tf.string, tf.float64)>
Traceback (most recent call last):
File "basic_class.m", line 44, in <module>
print(datatens.get_shape().as_list())
AttributeError: 'TensorSliceDataset' object has no attribute 'get_shape'
FOLLOWUP:
after getting eager execution running im curious, why my tensor is integer instead of float. here is the output of the advised code.
CODE:
print(tf.shape(data))
print(tf.shape(label))
OUTPUT:
Tensor("Shape:0", shape=(2,), dtype=int32)
Tensor("Shape_1:0", shape=(2,), dtype=int32)

When you call tf.data.Dataset.from_tensor_slices, you get a dataset, not a tensor. A dataset is essentially a container of tensors, and you can access its tensors in a few ways.
The simplest way is to call the dataset's make_one_shot_iterator method. This returns an iterator that cycles through the tensors. The best documentation on datasets and iterators is here.
Are you sure you want to call tf.data.Dataset.from_tensor_slices? Aren't data and label already tensors?
EDIT:
If you want to validate the tensor containing labels, try this code:
import tensorflow as tf
import numpy as np
import tflearn
import pandas as pd
from tensorflow import keras
from tflearn.data_utils import load_csv
tf.enable_eager_execution()
file = 'some.csv'
record_defaults = [tf.float64]*18
data, label = load_csv(file, target_column=0,has_header=True,
categorical_labels=True, n_classes=50)
print(tf.shape(label))
Enabling eager execution is important because it lets you access the tensor without having to create and run a session.

Related

Keras EfficientNet transfer learning code example not working

My code were working perfectly for months but today I realized it's not working anymore. In my code, I just copy-pasted this keras code example : https://keras.io/examples/vision/image_classification_efficientnet_fine_tuning/#example-efficientnetb0-for-stanford-dogs
So "my" code looks like this :
import tensorflow as tf
import keras
from keras.layers import *
from keras import Sequential
from keras.layers.experimental import preprocessing
from keras import layers
from tensorflow.keras.applications import EfficientNetB0
img_augmentation = Sequential(
[
preprocessing.RandomRotation(factor=0.15),
preprocessing.RandomTranslation(height_factor=0.1, width_factor=0.1),
preprocessing.RandomFlip(),
preprocessing.RandomContrast(factor=0.1),
],
name="img_augmentation",
)
inputs = layers.Input(shape=(224, 224, 3))
x = img_augmentation(inputs)
outputs = EfficientNetB0(include_top=True, weights=None, classes=5)(x)
model = tf.keras.Model(inputs, outputs)
model.compile(
optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"]
)
However, today when I run this cell in my colab, I get a lot of warnings like this :
WARNING:tensorflow:
The following Variables were used a Lambda layer's call (tf.compat.v1.nn.fused_batch_norm_422), but
are not present in its tracked objects:
<tf.Variable 'top_bn/gamma:0' shape=(1280,) dtype=float32>
<tf.Variable 'top_bn/beta:0' shape=(1280,) dtype=float32>
It is possible that this is intended behavior, but it is more likely
an omission. This is a strong indication that this layer should be
formulated as a subclassed Layer rather than a Lambda layer
And this error :
TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.
I think google colab updated keras and tensorflow, now they are both version 2.5.0
How can I make my code works again ?
You should not mix tf 2.x and standalone keras. You should import your libraries as follows, thus you won't get any issue.
import tensorflow as tf
from tensorflow.keras.layers import *
from tensorflow.keras import Sequential
from tensorflow.keras.layers.experimental import preprocessing
from tensorflow.keras import layers
from tensorflow.keras.applications import EfficientNetB0
img_augmentation = Sequential(
[
preprocessing.RandomRotation(factor=0.15),
preprocessing.RandomTranslation(height_factor=0.1, width_factor=0.1),
preprocessing.RandomFlip(),
preprocessing.RandomContrast(factor=0.1),
],
name="img_augmentation",
)
inputs = layers.Input(shape=(224, 224, 3))
x = img_augmentation(inputs)
outputs = EfficientNetB0(include_top=True, weights=None, classes=5)(x)
model = tf.keras.Model(inputs, outputs)
So after some research, I realized it comes from the imports :
from tensorflow.keras.applications import EfficientNetB0
I don't know why but it does not throw any error, but breaks the entire code. Instead, I have to import :
from keras.applications.efficientnet import EfficientNetB0
And it works perfectly.

numpy method for tensors in TensorFlow 2.x and eager execution

Using TensorFlow 2.4.1 on colab
Running this code below:
import tensorflow as tf
from tensorflow.keras.datasets import cifar100
import numpy as np
(train_data, train_labels), (test_data, test_labels) = cifar100.load_data(label_mode='fine')
train_dataset = tf.data.Dataset.from_tensor_slices((train_data, train_labels))
for (train, label) in train_dataset.take(1):
print(label)
print(label.numpy()[0])
# tf.Tensor([19], shape=(1,), dtype=int64)
# 19
This is all fine but when trying this with the filter method for keras.Dataset objects in the code below, the numpy method does not work:
def filter_classes(dataset, classes):
def match_class(data, label):
print(label)
print(label.numpy()[0])
return label.numpy()[0] in classes
return dataset.filter(match_class)
cifar_classes = [0, 29, 99]
train_dataset = filter_classes(train_dataset, cifar_classes)
# Tensor("args_1:0", shape=(1,), dtype=int64)
# AttributeError: 'Tensor' object has no attribute 'numpy'
From reading some of the related questions, the error seems to be due to the latter tensor not being eager executed.
Does the attribute in the tensor "arg_1:0" rather than being a numpy array signify that the tensor has not been evaluated?
With the filter method, is it by design that the tensors within the dataset objects do not get evaluated eagerly?
Thanks.
tf.data.Dataset functions do not run in EagerMode for performance reasons. You can't use the .numpy method in a function used by a tf.data.Dataset.
In your case, you can use a combination of tf.math.equal and tf.math.reduce_any to filter your dataset and keep only the desired classes:
ds_filtered = train_dataset.filter(lambda x:tf.math.reduce_any(tf.equal(x,cifar_classes)))

what's the meaning of 'input_length'?

the data have 4 timestamps,but the embedding's input_length=3,so what's the meaning of input_length?
from tensorflow import keras
import numpy as np
data = np.array([[0,0,0,0]])
emb = keras.layers.Embedding(input_dim=2, output_dim=3, input_length=3)
emb(data)
As per the official documentation here,
input_length: Length of input sequences, when it is constant. This
argument is required if you are going to connect Flatten then Dense
layers upstream (without it, the shape of the dense outputs cannot be
computed).
from tensorflow import keras
import numpy as np
model = keras.models.Sequential()
model.add(keras.layers.Embedding(input_dim=2, output_dim=3, input_length=4))
# the model will take as input an integer matrix of size (batch, input_length).
input_array = np.array([[0,0,0,0]])
model.compile('rmsprop', 'mse')
output_array = model.predict(input_array)
print(output_array)
Above works fine, but if you change input_length to 3, then you will get below error:
ValueError: Error when checking input: expected embedding_input to
have shape (3,) but got array with shape (4,)

multi_gpu_model : object of type 'NoneType' has no len()

I am getting this error while using keras multi_gpu_model. The code run fines if I eliminate this line. Also, with CNN model it works fines, it's just that while dense network it gives the error. Could you please help me to solve this issue. Thanks.
import numpy as np
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.layers import LSTM, BatchNormalization,Flatten
from keras.utils.vis_utils import model_to_dot
from keras.optimizers import adam
from keras.models import load_model
import pylab
from sklearn.model_selection import train_test_split
from keras.utils import multi_gpu_model
from scipy.io import wavfile
X=np.ones(10000)
y=np.zeros(100)
x_train=X
y_train=y
x_train=np.array(x_train)
y_train=np.array(y_train)
x_train.shape=(1,10000)
y_train.shape=(1,100)
model = Sequential()
model.add(Dense(500,activation = 'tanh'))
model.add(Dense(450, activation = 'tanh'))
model.add(Dense(412, activation = 'tanh'))
model.add(Dense(100, activation = 'tanh'))
opt = adam(lr=0.002, decay=1e-6)
model = multi_gpu_model(model, gpus=4)
model.compile(loss='mae', optimizer=opt, metrics=['accuracy'])
model.fit(x_train,y_train,epochs=50, batch_size = 40000)
Error: Traceback (most recent call last):
File "p.py", line 37, in <module>
model = multi_gpu_model(model, gpus=4)
File "/home/ENG/benipas1/anaconda3/envs/new/lib/python3.7/site-packages/keras/utils/multi_gpu_utils.py", line 203, in multi_gpu_model
for i in range(len(model.outputs)):
TypeError: object of type 'NoneType' has no len()
The problem is here:
model = Sequential()
model.add(Dense(500,activation = 'tanh'))
You are not giving an input shape to the first layer, so the outputs of the model are completely undefined and model.outputs is None. If you provide the input shape to the first layer, then the outputs are defined and it should work fine. You are probably providing the input shape to your CNN models and that is why it works:
model.add(Dense(500,activation = 'tanh', input_shape=(something,)))

Possible compatibility issue with Keras, TensorFlow and scikit (tf.global_variables())

I'm trying to do a small test with my dataset on Keras Regressor (using TensorFlow), but I'm having a small issue. The error seems to be on the function cross_val_score from scikit. It starts on it and the last error message is:
File "/usr/local/lib/python2.7/dist-packages/Keras-2.0.2-py2.7.egg/keras/backend/tensorflow_backend.py", line 298, in _initialize_variables
variables = tf.global_variables()
AttributeError: 'module' object has no attribute 'global_variables'
My full code is basically the example found in http://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/ with small changes.
I've looked upon the " 'module' object has no attribute 'global_variables' " error and it seems to be about the Tensorflow version, but I'm using the most recent one (1.0) and there is no function in the code that works directly with tf that I can change. Below is my full code, is there anyway i can change it so it works? Thanks for the help
import numpy
import pandas
import sys
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_svmlight_file
# define base mode
def baseline_model():
# create model
model = Sequential()
model.add(Dense(68, activation="relu", kernel_initializer="normal", input_dim=68))
model.add(Dense(1, kernel_initializer="normal"))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam')
return model
X, y, query_id = load_svmlight_file(str(sys.argv[1]), query_id=True)
scaler = StandardScaler()
X = scaler.fit_transform(X.toarray())
# fix random seed for reproducibility
seed = 1
numpy.random.seed(seed)
# evaluate model with standardized dataset
estimator = KerasRegressor(build_fn=baseline_model, nb_epoch=100, batch_size=5, verbose=0)
kfold = KFold(n_splits=5, random_state=seed)
results = cross_val_score(estimator, X, y, cv=kfold)
print("Results: %.2f (%.2f) MSE" % (results.mean(), results.std()))
You are probably using an older Tensorflow version install tensorflow 1.2.0rc2 and you should be fine.