module 'tensorflow' has no attribute 'Session', sparkdl errors - tensorflow

I am trying to use deep learning in pyspark but it's not working.
here is my code :
featurizer = DeepImageFeaturizer(inputCol="image",
outputCol="features",
modelName="InceptionV3")
lr = LogisticRegression(maxIter=5, regParam=0.03,
elasticNetParam=0.5, labelCol="label")
sparkdn = Pipeline(stages=[featurizer, lr])
spark_model = sparkdn.fit(train)
The last part of The error:
File ~/anaconda3/lib/python3.9/site-packages/sparkdl/transformers/keras_applications.py:45, in KerasApplicationModel.getModelData(self, featurize)
44 def getModelData(self, featurize):
---> 45 sess = tf.Session()
46 with sess.as_default():
47 K.set_learning_phase(0)
AttributeError: module 'tensorflow' has no attribute 'Session'
I tried to uninstall and install tf.
also I am running pyspark with deep-learning package:
pyspark --packages databricks:spark-deep-learning:0.1.0-spark2.1-s_2.11

Looks like you are trying to use the DeepImageFeaturizer transformer from the spark-deep-learning package. This error usually occurs if you have an old version of TensorFlow installed on your system, or if your using a version of TensorFlow that is not compatible with the spark-deep-learning package.
Try the following:
Uninstall the current version of TensorFlow using pip uninstall tensorflow.
Then install the latest version of TensorFlow: pip install tensorflow.
Make sure that the version of TensorFlow that you are using is compatible with the spark-deep-learning package. According to the documentation it requires TensorFlow 1.6 or later.
Try setting the environment variable TF_CPP_MIN_LOG_LEVEL to 3 before running your code. This can help suppress some TensorFlow warnings and errors.

Related

AttributeError: module 'tensorflow._api.v1.config' has no attribute 'set_visible_devices'

First of all, I apologize that my English is not good for you to understand.
Currently, I am doing computer vision using tensorflow version 1.14. In the process, the following problem ocurred in the process of rotating the model using GPU.
AttributeError: module 'tensorflow._api.v1.config' has no attribute 'set_visible_devices'
The current development environment is as follows.
Python: 3.7.9
conda: 4.8.3
tensorflow: 1.14.0
keras: 2.3.1
In addition, I currently have 4 gpu, and i want to use 2 gpu as if it were 1 gpu. Can you give me a good idea for this?
thank you.
It seems you need to upgrade the tensorflow because tf.config.set_visible_devices() function is available in latest version of tensorflow, you can use below code to upgrade the tensorflow:
!pip install --upgrade pip.
!pip install --upgrade tensorflow
You can follow the link to install the CPU/GPU version of tensorflow as per requirement and for tf.config.set_visible_devices() function related details, check here

TensorFlow problem i have the required TF but still get an error

I created an env and imported TensorFlow but I get the error:
Keras requires TensorFlow 2.2 or higher
TensorFlow at this env is 2.3, so I don't know why I get this error.
the way to solve the problem is to create a new virtual environment
python 3.7
tesorflow 1.15.2
or uninstall any previously installed tensorflow and install 1.15.2
i'm not sure why this worked though.

Tutorial tf-nightly not woring

I am pretty new to tensorflow and right now I'm following the tutorials on their website to learn more about it. While going through the text classification one, whenever I get to step that wants me to do:
**batch_size = 32
seed = 42
raw_train_ds = tf.keras.preprocessing.text_dataset_from_directory(
'aclImdb/train',
batch_size=batch_size,
validation_split=0.2,
subset='training',
seed=seed)**
I get an error saying AttributeError: module 'tensorflow.keras.preprocessing' has no attribute 'text_dataset_from_directory'. I looked online and it seems that the problem is that tf-nightly isn't installed. But I already installed tf-nightly on my anaconda prompt. Is there another issue I'm missing?
Thank you
I suspect that the package named keras-preprocessing is missing in Anaconda in your Machine.
You could install it by running the below command in the cell of Jupyter Notebook,
!pip install keras_preprocessing
As I could successfully execute the code in my Jupyter Notebook, I will share the packages which I have, that may be useful to you in executing Tensorflow Code.
Please find the screenshots below:

Module 'tensorflow' has no attribute 'contrib'

I am trying to train my own custom object detector using Tensorflow Object-Detection-API
I installed the tensorflow using "pip install tensorflow" in my google compute engine. Then I followed all the instructions on this site: https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html
When I try to use train.py I am getting this error message:
Traceback (most recent call last):
File "train.py", line 49, in
from object_detection.builders import dataset_builder
File "/usr/local/lib/python3.6/dist-packages/object_detection-0.1->py3.6.egg/object_detection/builders/dataset_builder.py", line 27, in
from object_detection.data_decoders import tf_example_decoder
File "/usr/local/lib/python3.6/dist-packages/object_detection-0.1-py3.6.egg/object_detection/data_decoders/tf_example_decoder.py", line 27, in
slim_example_decoder = tf.contrib.slim.tfexample_decoder
AttributeError: module 'tensorflow' has no attribute 'contrib'
Also I am getting different results when I try to learn version of tensorflow.
python3 -c 'import tensorflow as tf; print(tf.version)' : 2.0.0-dev20190422
and when I use
pip3 show tensorflow:
Name: tensorflow
Version: 1.13.1
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: opensource#google.com
License: Apache 2.0
Location: /usr/local/lib/python3.6/dist-packages
Requires: gast, astor, absl-py, tensorflow-estimator, keras-preprocessing, grpcio, six, keras-applications, wheel, numpy, tensorboard, protobuf, termcolor
Required-by:
sudo python3 train.py --logtostderr --train_dir=training/ --
pipeline_config_path=training/ssd_inception_v2_coco.config
What should I do to solve this problem? I couldn't find anything about this error message except this: tensorflow 'module' object has no attribute 'contrib'
tf.contrib has moved out of TF starting TF 2.0 alpha.
Take a look at these tf 2.0 release notes https://github.com/tensorflow/tensorflow/releases/tag/v2.0.0-alpha0
You can upgrade your TF 1.x code to TF 2.x using the tf_upgrade_v2 script
https://www.tensorflow.org/alpha/guide/upgrade
This issue might be helpful for you, it explains how to achieve TPUStrategy, a popular functionality of tf.contrib in TF<2.0.
So, in TF 1.X you could do the following:
resolver = tf.contrib.cluster_resolver.TPUClusterResolver('grpc://' + os.environ['COLAB_TPU_ADDR'])
tf.contrib.distribute.initialize_tpu_system(resolver)
strategy = tf.contrib.distribute.TPUStrategy(resolver)
And in TF>2.0, where tf.contrib is deprecated, you achieve the same by:
tf.config.experimental_connect_to_host('grpc://' + os.environ['COLAB_TPU_ADDR'])
resolver = tf.distribute.cluster_resolver.TPUClusterResolver('grpc://' + os.environ['COLAB_TPU_ADDR'])
tf.tpu.experimental.initialize_tpu_system(resolver)
strategy = tf.distribute.experimental.TPUStrategy(resolver)
One easy way is you can pass your code written in TensorFlow 1.x to the below code to automatically upgrade it to TensorFlow 2.x.
$tf_upgrade_v2 \
--intree my_project/ \
--outtree my_project_v2/ \
--reportfile report.txt
The above code will replace all the commands which are deprecated in 2.x with the onces that are actually working in 2.x. And then you can run your code in TensorFlow 2.x.
In case if it throws an error and is unable to convert the complete code and then don't panic. Please open the "report.txt" file that is generated by the above code. In this file, you will find commands that are deprecated and their alternative commands that can be used in TensorFlow 2.x.
Taadaa, just replace the commands that are throwing errors with the new ones.
Example:
If the command in TensorFlow 1.x is:
tf.contrib
Then the same command in Tensorflow 2.x is:
tf.compat.v1.estimator
In the above example replace "tf.contrib" with "tf.compat.v1.estimator" and that should solve the problem.
I used google colab to run my models and everything was perfect untill i used inline tesorboard. With tensorboard inline, I had the same issue of "Module 'tensorflow' has no attribute 'contrib'".
It was able to run training when rebuild and reinstall the model using setup.py(research folder) after initialising tensorboard.
I used tensorflow 1.8 to train my model and there is no problem for now. Tensorflow 2.0 alpha is not suitable with object detection API
I'm using Google Colab as well. A comment suggested to put
%tensorflow_version 1.x
in the first (code) cell, and it worked!
If you want to use tf.contrib, you need to now copy and paste the source code from github into your script/notebook. It's annoying and doesn't always work. But that's the only workaround I've found. For example, if you wanted to use tf.contrib.opt.AdamWOptimizer, you have to copy and paste from here. https://github.com/tensorflow/tensorflow/blob/590d6eef7e91a6a7392c8ffffb7b58f2e0c8bc6b/tensorflow/contrib/opt/python/training/weight_decay_optimizers.py#L32
I face the same error and solve it by install python version 3.7 then i can install tensorflow 1.15 and it work.
I used tensorflow==2.9 but tensorflow-probability==0.6.0 so I met this error too. tensorflow-probability==0.6.0 seems to be compatible with tf 1
this is solution: pip install tensorflow_probability==0.12.2
This version of TensorFlow Probability requires TensorFlow version >= 2.3
if there are still some errors pip install tensorflow_probability==0.17.0
For me it worked using the latest release of tensorflow: pip install tensorflow==2.2.0

Why keras always says using theano backend?

I installed keras using conda in my virtual environment and checked $HOME/.keras/config.json file.
{
"image_data_format": "channels_last",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "tensorflow"
}
I already set backend to tensorflow but when I run this in the python console
import keras
It is showing me that keras is using theano backend. Why?
Using Theano backend.
WARNING (theano.configdefaults): install mkl with `conda install mkl-service`: No module named mkl
I added export KERAS_BACKEND=tensorflow at the end of my .bashrc and restart the command line and activate my source. Still seeing the above error again. Can anyone help me with this?
We had also faced the same issue when installed keras using conda environment. Since we already had keras installed using pip, where the backend was set as theano, it was taking that keras. The problem got fixed when we removed the pip version of keras using the command pip uninstall keras
Well you can start your editor with line:
KERAS_BACKEND=tensorflow
KERAS_BACKEND=tensorflow spyder
This would force use the Tensorflow backend.
But before using this ensure that you have tensorflow installed with all the required dependencies.
Source