I am trying to use deep learning in pyspark but it's not working.
here is my code :
featurizer = DeepImageFeaturizer(inputCol="image",
outputCol="features",
modelName="InceptionV3")
lr = LogisticRegression(maxIter=5, regParam=0.03,
elasticNetParam=0.5, labelCol="label")
sparkdn = Pipeline(stages=[featurizer, lr])
spark_model = sparkdn.fit(train)
The last part of The error:
File ~/anaconda3/lib/python3.9/site-packages/sparkdl/transformers/keras_applications.py:45, in KerasApplicationModel.getModelData(self, featurize)
44 def getModelData(self, featurize):
---> 45 sess = tf.Session()
46 with sess.as_default():
47 K.set_learning_phase(0)
AttributeError: module 'tensorflow' has no attribute 'Session'
I tried to uninstall and install tf.
also I am running pyspark with deep-learning package:
pyspark --packages databricks:spark-deep-learning:0.1.0-spark2.1-s_2.11
Looks like you are trying to use the DeepImageFeaturizer transformer from the spark-deep-learning package. This error usually occurs if you have an old version of TensorFlow installed on your system, or if your using a version of TensorFlow that is not compatible with the spark-deep-learning package.
Try the following:
Uninstall the current version of TensorFlow using pip uninstall tensorflow.
Then install the latest version of TensorFlow: pip install tensorflow.
Make sure that the version of TensorFlow that you are using is compatible with the spark-deep-learning package. According to the documentation it requires TensorFlow 1.6 or later.
Try setting the environment variable TF_CPP_MIN_LOG_LEVEL to 3 before running your code. This can help suppress some TensorFlow warnings and errors.
First of all, I apologize that my English is not good for you to understand.
Currently, I am doing computer vision using tensorflow version 1.14. In the process, the following problem ocurred in the process of rotating the model using GPU.
AttributeError: module 'tensorflow._api.v1.config' has no attribute 'set_visible_devices'
The current development environment is as follows.
Python: 3.7.9
conda: 4.8.3
tensorflow: 1.14.0
keras: 2.3.1
In addition, I currently have 4 gpu, and i want to use 2 gpu as if it were 1 gpu. Can you give me a good idea for this?
thank you.
It seems you need to upgrade the tensorflow because tf.config.set_visible_devices() function is available in latest version of tensorflow, you can use below code to upgrade the tensorflow:
!pip install --upgrade pip.
!pip install --upgrade tensorflow
You can follow the link to install the CPU/GPU version of tensorflow as per requirement and for tf.config.set_visible_devices() function related details, check here
I wanted to use LMS for my deep learning project, and I used the code for this:
import tensorflow as tf
tf.config.experimental.set_lms_enabled(True)
from this IBM GitHub link. But I get the mentioned error.
AttributeError: module 'tensorflow._api.v2.config.experimental' has no attribute 'set_lms_enabled'
I tried it in colab notebook, which uses tensorflow 2.2. Should I downgrade it? How else do I solve this issue?
according to TensorFlow Large Model Support github page:
TFLMS is built into the tensorflow-gpu conda package so it is installed by default when you install the GPU enabled TensorFlow from WML CE.
so you need to install anaconda and install TensorFlow from WML CE by conda package manager.
to install anaconda follow anaconda official document page by this link:
https://docs.anaconda.com/anaconda/install/
I am trying to train my own custom object detector using Tensorflow Object-Detection-API
I installed the tensorflow using "pip install tensorflow" in my google compute engine. Then I followed all the instructions on this site: https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html
When I try to use train.py I am getting this error message:
Traceback (most recent call last):
File "train.py", line 49, in
from object_detection.builders import dataset_builder
File "/usr/local/lib/python3.6/dist-packages/object_detection-0.1->py3.6.egg/object_detection/builders/dataset_builder.py", line 27, in
from object_detection.data_decoders import tf_example_decoder
File "/usr/local/lib/python3.6/dist-packages/object_detection-0.1-py3.6.egg/object_detection/data_decoders/tf_example_decoder.py", line 27, in
slim_example_decoder = tf.contrib.slim.tfexample_decoder
AttributeError: module 'tensorflow' has no attribute 'contrib'
Also I am getting different results when I try to learn version of tensorflow.
python3 -c 'import tensorflow as tf; print(tf.version)' : 2.0.0-dev20190422
and when I use
pip3 show tensorflow:
Name: tensorflow
Version: 1.13.1
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: opensource#google.com
License: Apache 2.0
Location: /usr/local/lib/python3.6/dist-packages
Requires: gast, astor, absl-py, tensorflow-estimator, keras-preprocessing, grpcio, six, keras-applications, wheel, numpy, tensorboard, protobuf, termcolor
Required-by:
sudo python3 train.py --logtostderr --train_dir=training/ --
pipeline_config_path=training/ssd_inception_v2_coco.config
What should I do to solve this problem? I couldn't find anything about this error message except this: tensorflow 'module' object has no attribute 'contrib'
tf.contrib has moved out of TF starting TF 2.0 alpha.
Take a look at these tf 2.0 release notes https://github.com/tensorflow/tensorflow/releases/tag/v2.0.0-alpha0
You can upgrade your TF 1.x code to TF 2.x using the tf_upgrade_v2 script
https://www.tensorflow.org/alpha/guide/upgrade
This issue might be helpful for you, it explains how to achieve TPUStrategy, a popular functionality of tf.contrib in TF<2.0.
So, in TF 1.X you could do the following:
resolver = tf.contrib.cluster_resolver.TPUClusterResolver('grpc://' + os.environ['COLAB_TPU_ADDR'])
tf.contrib.distribute.initialize_tpu_system(resolver)
strategy = tf.contrib.distribute.TPUStrategy(resolver)
And in TF>2.0, where tf.contrib is deprecated, you achieve the same by:
tf.config.experimental_connect_to_host('grpc://' + os.environ['COLAB_TPU_ADDR'])
resolver = tf.distribute.cluster_resolver.TPUClusterResolver('grpc://' + os.environ['COLAB_TPU_ADDR'])
tf.tpu.experimental.initialize_tpu_system(resolver)
strategy = tf.distribute.experimental.TPUStrategy(resolver)
One easy way is you can pass your code written in TensorFlow 1.x to the below code to automatically upgrade it to TensorFlow 2.x.
$tf_upgrade_v2 \
--intree my_project/ \
--outtree my_project_v2/ \
--reportfile report.txt
The above code will replace all the commands which are deprecated in 2.x with the onces that are actually working in 2.x. And then you can run your code in TensorFlow 2.x.
In case if it throws an error and is unable to convert the complete code and then don't panic. Please open the "report.txt" file that is generated by the above code. In this file, you will find commands that are deprecated and their alternative commands that can be used in TensorFlow 2.x.
Taadaa, just replace the commands that are throwing errors with the new ones.
Example:
If the command in TensorFlow 1.x is:
tf.contrib
Then the same command in Tensorflow 2.x is:
tf.compat.v1.estimator
In the above example replace "tf.contrib" with "tf.compat.v1.estimator" and that should solve the problem.
I used google colab to run my models and everything was perfect untill i used inline tesorboard. With tensorboard inline, I had the same issue of "Module 'tensorflow' has no attribute 'contrib'".
It was able to run training when rebuild and reinstall the model using setup.py(research folder) after initialising tensorboard.
I used tensorflow 1.8 to train my model and there is no problem for now. Tensorflow 2.0 alpha is not suitable with object detection API
I'm using Google Colab as well. A comment suggested to put
%tensorflow_version 1.x
in the first (code) cell, and it worked!
If you want to use tf.contrib, you need to now copy and paste the source code from github into your script/notebook. It's annoying and doesn't always work. But that's the only workaround I've found. For example, if you wanted to use tf.contrib.opt.AdamWOptimizer, you have to copy and paste from here. https://github.com/tensorflow/tensorflow/blob/590d6eef7e91a6a7392c8ffffb7b58f2e0c8bc6b/tensorflow/contrib/opt/python/training/weight_decay_optimizers.py#L32
I face the same error and solve it by install python version 3.7 then i can install tensorflow 1.15 and it work.
I used tensorflow==2.9 but tensorflow-probability==0.6.0 so I met this error too. tensorflow-probability==0.6.0 seems to be compatible with tf 1
this is solution: pip install tensorflow_probability==0.12.2
This version of TensorFlow Probability requires TensorFlow version >= 2.3
if there are still some errors pip install tensorflow_probability==0.17.0
For me it worked using the latest release of tensorflow: pip install tensorflow==2.2.0
You can install the nightly build via:
pip install tf-nightly
How to figure out what code commit this is coming from? I have looked int he tags from the github tensorflow branch and do not see any corresponding to the tf.__version__ that is showing in the REPL.
python -c "import tensorflow as tf; print(tf.__git_version__, tf.__version__)"
printed the following in my experiment:
v1.12.1-25814-g2e81bc66c5 2.2.0-dev20200225
The SHA of the commit is the string at the end, after -g. In this case it was 2e81bc66c5.
The accepted answer did not work for me with tf_nightly-2.2.0.dev20200225. All I got was AttributeError: module 'tensorflow' has no attribute 'VERSION' and AttributeError: module 'tensorflow' has no attribute 'GIT_VERSION'
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
From the tensorflow new issue template:
https://github.com/tensorflow/tensorflow/issues/new?template=00-bug-performance-issue.md