I am using tensorflow 2.1. And using keras from tensorflow :
import tensorflow as tf
from tensorflow import keras
I just want to find where is the function load_data() is stored.
fashion_mnist = keras.datasets.fashion_mnist
(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()
for the above statements, I juse wonder:
what is the mechanism of keras.datasets.fashion_mnist? As datasets is the subdirectory of keras, and fashion_mnist is the subdirectory of datasets. How can they be called as method/function type in python3?
Where/which python file under tensorflow stores load_data() function?
it seems keras is located under c:\Users\myusername\.keras, where C:\Users\myusername\.conda\envs\tensorflow2\Lib\site-packages stores like keras_applications, kreas_preprocessing. I can not find which location and python file is read when load_data() is called.
Many thanks if someone can help give some suggestions.
Regards.
Wei
Related
I'm learning TensorFlow pretty much from scratch, and finding it hard to replicate examples on latest versions of TF.
In a typical VAE example of MNIST I have a segment of code like:
(...)
def vae_loss(y_true, y_pred):
kl_loss = vae_kl_loss(y_true, y_pred)
rc_loss = vae_rc_loss(y_true, y_pred)
kl_weight_const = 1e-2
return kl_weight_const*kl_loss + rc_loss
vae.compile(
loss = [vae_loss],
optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3),
metrics=[vae_kl_loss,vae_rc_loss]
)
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.astype('float32')/255
x_test = x_test.astype('float32')/255
x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:])))
x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))
vae.fit(x_train, x_train, shuffle=True, epochs=nb_epoch, batch_size = batch_size, validation_data=(x_test, x_test))
With TensorFlow 2.4 and no special config, receive the error:
Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.
Through some search, I found references that this could be a symptom of eager of execution in TF2 and
indeed with
tf.compat.v1.disable_eager_execution()
The code runs. Now I'm trying to learn what would be the way to do this example with eager execution.
Interestingly enough on TensorFlow 2.3 with
tf.config.run_functions_eagerly(True)
it runs, but not with TensorFlow 2.4.
In another similar example I have where x_train is loaded with tensorflow_datasets, but otherwise same TF version and loss function, it also works correctly without any config settings. I'd assume that the behaviour is because the input is now a tf.data.Dataset rather than numpy array ?
Thanks in advance for any help you may provide explaining how I can my example working with eager mode.
Thanks
If I want to implement a classifier using the sklearn library. Is there a way to save the model or convert the file into a saved tensorflow file in order to convert it to tensorflow lite later?
If you replicate the architecture in TensorFlow, which will be pretty easy given that scikit-learn models are usually rather simple, you can explicitly assign the parameters from the learned scikit-learn models to TensorFlow layers.
Here is an example with logistic regression turned into a single dense layer:
import tensorflow as tf
import numpy as np
from sklearn.linear_model import LogisticRegression
# some random data to train and test on
x = np.random.normal(size=(60, 21))
y = np.random.uniform(size=(60,)) > 0.5
# fit the sklearn model on the data
sklearn_model = LogisticRegression().fit(x, y)
# create a TF model with the same architecture
tf_model = tf.keras.models.Sequential()
tf_model.add(tf.keras.Input(shape=(21,)))
tf_model.add(tf.keras.layers.Dense(1))
# assign the parameters from sklearn to the TF model
tf_model.layers[0].weights[0].assign(sklearn_model.coef_.transpose())
tf_model.layers[0].bias.assign(sklearn_model.intercept_)
# verify the models do the same prediction
assert np.all((tf_model(x) > 0)[:, 0].numpy() == sklearn_model.predict(x))
It is not always easy to replicate a scikit model in tensorflow. For instance scitik has a lot of on the fly imputation libraries which will be a bit tricky to implement in tensorflow
Is there a guide anywhere for serializing and restoring Estimator models in TF2? The documentation is very spotty, and much of it not updated to TF2. I've yet to see a clear ands complete example anywhere of an Estimator being saved, loaded from disk and used to predict from new inputs.
TBH, I'm a bit baffled by how complicated this appears to be. Estimators are billed as simple, relatively high-level ways of fitting standard models, yet the process for using them in production seems very arcane. For example, when I load a model from disk via tf.saved_model.load(export_path) I get an AutoTrackable object:
<tensorflow.python.training.tracking.tracking.AutoTrackable at 0x7fc42e779f60>
Its not clear why I don't get my Estimator back. It looks like there used to be a useful-sounding function tf.contrib.predictor.from_saved_model, but since contrib is gone, it does not appear to be in play anymore (except, it appears, in TFLite).
Any pointers would be very helpful. As you can see, I'm a bit lost.
maybe the author doesn't need the answer anymore but I was able to save and load a DNNClassifier using TensorFlow 2.1
# training.py
from pathlib import Path
import tensorflow as tf
....
# Creating the estimator
estimator = tf.estimator.DNNClassifier(
model_dir = <model_dir>,
hidden_units = [1000, 500],
feature_columns = feature_columns, # this is a list defined earlier
n_classes = 2,
optimizer = 'adam')
feature_spec = tf.feature_column.make_parse_example_spec(feature_columns)
export_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
servable_model_path = Path(estimator.export_saved_model(<model_dir>, export_input_fn).decode('utf8'))
print(f'Model saved at {servable_model_path}')
For loading, you found the correct method, you just need to retrieve the predict_fn
# testing.py
import tensorflow as tf
import pandas as pd
def predict_input_fn(test_df):
'''Convert your dataframe using tf.train.Example() and tf.train.Features()'''
examples = []
....
return tf.constant(examples)
test_df = pd.read_csv('test.csv', ...)
# Loading the estimator
predict_fn = tf.saved_model.load(<model_dir>).signatures['predict']
# Predict
predictions = predict_fn(examples=predict_input_fn(test_df))
Hope that this can help other people too (:
I just successfully finished training a tf.keras sequential model and wrote a separate "Flask" script where I load the saved model to an app I uploaded to Heroku. Everything worked. But, when I was playing, I realized for my requirements I only had to import Flask, request from flask as well as pandas, numpy and from tensorflow.python.keras.models import load_model:
from tensorflow.python.keras.models import load_model
from flask import Flask, request
import pandas as pd
import numpy as np
With all that model.predict works (after I run load_model() and the necessary pandas and numpy transformations to get the Flask/Post data in the right format (processed_features):
preds = flask_model.predict(processed_features).flatten()
What is allowing me to run model.predict()? Is model.predict() available when load_model is imported (I obviously didn't have to use load_model to run model.predict to run against my test data after training)? Or is model.predict() a generic function in python, numpy or pandas that knows how to execute a prediction through the keras model (somehow just leverages the weights, biases, model shape)?
While everything works, I think I am not understanding how it works.
The saved model to disk has both the model architecture and the weights. load_model API deserializes this file, builds and returns a Keras Model object. So, you're essentially invoking predict() on the Keras Model object. You can inspect the model object by invoking the following methods:
type(flask_model) #check the type
dir(flask_model) #list the attributes/methods available
Is it possible to convert a keras model (h5 file of network architecture and weights) into a tensorflow model? Or is there an equivalent function to model.save of keras in tensorflow?
Yes, it is possible, because Keras, since it uses Tensorflow as backend, also builds computational graph. You just need to get this graph from your Keras model.
"Keras only uses one graph and one session. You can access the session
via: K.get_session(). The graph associated with it would then be:
K.get_session().graph."
(from fchollet: https://github.com/keras-team/keras/issues/3223#issuecomment-232745857)
Or you can save this graph in checkpoint format (https://www.tensorflow.org/api_docs/python/tf/train/Saver):
import tensorflow as tf
from keras import backend as K
saver = tf.train.Saver()
sess = K.get_session()
retval = saver.save(sess, ckpt_model_name)
By the way, since tensorflow 13 you can use keras right from it:
from tensorflow.python.keras import models, layers