My data source - https://www.kaggle.com/vbookshelf/respiratory-sound-database
Tensroflow version - 2.4.0
After a bit of Data Cleaning my pandas Dataframe looked like this:
My objective is to make Deep Learning Model for a Classification task, so I read the audio files with scipy.io wavefile and put the array as a feature in the data frame.
All the values of the audio have a shape of (882000,)
My problem is that I want to convert my Pandas Dataframe into a Tensorflow Dataset.
I get this error: ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type NumPy.ndarray).
I tried using tf.convert_to_tensor and still get the same error. What should I do?
Related
I have a pandas dataframe containing np.arrays in each cell (see the photo to understand better). Each array is 1000 samples long. However, when trying to use this as a training data in LSTM, it won't go through. enter image description here
model.fit(x_train, y_train, epochs = 15)
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type numpy.ndarray).
How do I tackle this? Can't find an answer elsewhere, tried this:
x_train=np.asarray(x_train).astype(np.float32)
but it failed due ValueError: setting an array element with a sequence.
Is there another way to use this sort of numpy arrays as input?
I was trying to train LSTM with my pandas dataframe data containing 1000 sample long np.arrays
I have a class 'tensorflow.python.framework.ops.Tensor as output and need to convert this to a numpy array.
.numpy() doesn't work because it isn't a eagerTensor.
.eval doesn't work as well, because i'm using tensorflow >2.0
Is there any other way to fix this?
img_height=330
img_width=600
img_depth=23
save_model="saved_Models/wheatModel"
prediction_data_path=["data/stacked/MOD13Q1.A2017.2738.tif","data/stacked/MOD13Q1.A2017.889.tif","data/stacked/MOD13Q1.A2017.923.tif"]
prediction_data=dataConv.preparePredictionData(prediction_data_path)
prediction_reshaped=dataConv.reshapeFiles(prediction_data,img_width,img_height,img_depth)
x_ds =tf.stack(prediction_reshaped)
model = tf.keras.models.load_model(save_model)
model.predict(x_ds)
image=model.get_layer(name='prediction_image').output
n,output_width,output_height,output_depth,output_channels=image.shape
print(type(image))
image=tf.reshape(image,(output_width,output_height,output_depth))
print(type(image))
image.numpy()
So in the code above.
I load my trained model
predict the given images
get the output from the next to last layer
reshape this data
Now i want to convert this tensor to an numpyarray
I am working with Tensorflow and Tensorboard version 1.14.
I would like to perform some off-line analysis starting from Data I have saved during training using the tf.summary.tensor_summary()
I am not able to recover the data saved with the method described here, using the tf.train.summary_iterator which does recover scalar data but not the data I saved with the tensor_summary method.
Though with the EventAccumulator object I am able to recover the data I have saved, that it is returned as a TensorEvent Object which has the following attributes:
step
wall_time
tensor_proto
tensor_content
Thing is that I would like to convert this data into numpy array, the TensorEvent object sure has all the information needed (tensor_proto for type and shape, tensor_content for values), but not being a Tensor does not have a .value or a .numpy() method. So I do I trasform a TensorEvent Object into a numpy array? or equivalently into a Tensor object then into a numpy array?
You can use tf.make_ndarray to convert a TensorProto into a NumPy array:
tensor_np = tf.make_ndarray(tensor_event.tensor_proto)
I am trying to use what if tool on my xgboost model.
But on the link I am only able to find examples of xgboost used through google AI Platform. Is there any way we can use whatif tool on XGboost without Google AI platform
I tried the functions that were used in examples for tensorflow and keras and used functions set_estimator_and_feature_spec and set_compare_custom_predict_fn
bst = xgb.XGBClassifier(
objective='reg:logistic'
)
bst.fit(x_train, y_train)
test_examples = df_to_examples(df_test)
config_builder = WitConfigBuilder(test_examples).set_custom_predict_fn(xg.predict)
WitWidget(config_builder)
When trying to perform run inference, an error msg is displayed cannot initialize DMatrix from a list and I am not unable to do it
After a lot of trial and error I finally got it to work with XGBoost using the following code:
# first argument in my case is a Pandas dataframe of all features and the target 'label';
# extract just the numpy array with 'values', then convert that to a list
# second argument is the column name as a list
# I use a sklearn pipeline, so here I am just accessing the classifier model which is an XGBClassifier instance
# Wit tool expects a 2D array, where the 1st dimension is each sample, and 2nd dimension is probabilities of
# each class; so use 'predict_proba' over 'predict'
config_builder = (WitConfigBuilder(df_sample.values.tolist(), df_sample.columns.tolist())
.set_custom_predict_fn(clf['classifier'].predict_proba)
.set_target_feature('label')
.set_label_vocab(['No Churn', 'Churn']))
This eliminated the need to use their suggested helper functions and works out-of-the-box with Pandas DataFrames and Sklearn ML Models
My model learns W:
W = tf.Variable(tf.truncated_normal([pixels,h1],stddev=np.sqrt(2.0 / (pixels))))
I return W from a function that runs my TF graph / session.
In my notebook, I checked type of W:
type(W)
out: tensorflow.python.ops.variables.Variable
I also checked dimensionality of W:
W.get_shape()
out: TensorShape([Dimension(3072), Dimension(1024)])
I'd like to convert W into a Pandas dataframe (for examination, etc.).
How can I do this?
(Saw this answer on converting tensor to numpy with eval(), which could then be written to pandas of course. But that operation only seemed to work within the TF session.)
variables only exist within a session. they are defined in the graph, as operations, but dont actually store any values as such, in the graph . they only have values when a session is created from the graph, and initialize operation called (or load is called).
Of course, once you've loaded the value from the varaible, in a session, using eval, you're free to dispose of the session, and use the resulting numpy tensor jsut as any normal numpy tensor.