Error on gcloud with tensorflow.python.keras.applications for machine learning - tensorflow

Im working with gcloud and I got this error:
The replica master 0 exited with a non-zero status of 1.
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/root/.local/lib/python3.7/site-packages/object_detection/model_main.py", line 25, in <module>
from object_detection import model_lib
File "/root/.local/lib/python3.7/site-packages/object_detection/model_lib.py", line 30, in <module>
from object_detection import exporter as exporter_lib
File "/root/.local/lib/python3.7/site-packages/object_detection/exporter.py", line 24, in <module>
from object_detection.builders import model_builder
File "/root/.local/lib/python3.7/site-packages/object_detection/builders/model_builder.py", line 37, in <module>
from object_detection.meta_architectures import deepmac_meta_arch
File "/root/.local/lib/python3.7/site-packages/object_detection/meta_architectures/deepmac_meta_arch.py", line 19, in <module>
from object_detection.models.keras_models import resnet_v1
File "/root/.local/lib/python3.7/site-packages/object_detection/models/keras_models/resnet_v1.py", line 22, in <module>
from tensorflow.python.keras.applications import resnet
ModuleNotFoundError: No module named 'tensorflow.python.keras.applications'
My requeriments.txt file is:
six
google-api-python-client>=1.6.7
kaggle>=1.3.9
Keras-Applications
numpy>=1.15.4
oauth2client
pandas>=0.22.0
psutil>=5.4.3
py-cpuinfo>=3.3.0
scipy>=0.19.1
tensorflow-hub>=0.6.0
tensorflow-model-optimization
tensorflow-model-optimization>=0.4.1
tensorflow-addons
dataclasses;python_version<"3.7"
gin-config
tf_slim>=1.1.0
Cython
matplotlib
# Loader becomes a required positional argument in 6.0 in yaml.load
pyyaml>=5.1,<6.0
# CV related dependencies
opencv-python-headless
Pillow
pycocotools
# NLP related dependencies
seqeval
sentencepiece
sacrebleu
From my terminal I used this code to run the machine learning:
gcloud ai-platform jobs submit training segmentation_maskrcnn_img_53 ^
--runtime-version 2.5 ^
--python-version 3.7 ^
--job-dir=gs://meat-segmentation-img/training_process ^
--package-path ./object_detection ^
--module-name object_detection.model_main ^
--region us-central1 ^
--scale-tier CUSTOM ^
--master-machine-type n1-standard-16 ^
--master-accelerator count=1,type=nvidia-tesla-t4 ^
-- ^
--model_dir=gs://meat-segmentation-img/training_process ^
--pipeline_config_path=gs://meat-segmentation-img/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8.config
I used runtime version 2.7 but I got this error:
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /opt/conda/lib/python3.7/site-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)
How I fix this?
I'm using this from Windows by conda environment.

You may need to modify the code for importing keras applications as
from tensorflow.keras import applications
in place of
from tensorflow.python.keras import applications.
Please check this link for more details.

Related

Conda with Python3.9 using numpy in Python3.10

I'm trying to install statsmodels in Oracle Machine Learning in Conda enviroment.
My conda version is:
%conda
info
active environment : None
shell level : 0
user config file : /u01/.condarc
populated config files : /usr/share/conda/condarc.d/defaults.yaml
/u01/.condarc
conda version : 4.6.14
conda-build version : not installed
python version : 3.6.8.final.0
base environment : /usr (read only)
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/linux-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /u01/.conda/pkgs
/var/cache/conda/pkgs
envs directories : /u01/.conda/envs
/usr/envs
platform : linux-64
user-agent : conda/4.6.14 requests/2.27.1 CPython/3.6.8 Linux/5.4.17-2136.314.6.3.el7uek.x86_64 oracle/7.9 glibc/2.17
UID:GID : 65000:65000
netrc file : None
offline mode : False
I created the conda enviroment with the next command:
%conda
create -n arima_enviroment python=3.9 xz sqlite libuuid statsmodels numpy
I activated the enviroment with:
%conda
activate arima_enviroment
Test the enviroment with:
%python
import sys
import platform
print("sys.version:", sys.version)
print("sys.version_info:", sys.version_info)
print("platform.python_version:", platform.python_version())
sys.version: 3.9.12 (main, Jun 1 2022, 11:38:51) [GCC 7.5.0]
sys.version_info: sys.version_info(major=3, minor=9, micro=12, releaselevel='final', serial=0)
platform.python_version: 3.9.12
Then I execute the next command for import the ARIMA model.
%python
from statsmodels.tsa.arima_model import arima
But give me the next error:
Fail to execute line 2: from statsmodels.tsa.arima_model import arima
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/numpy/core/__init__.py", line 23, in <module>
from . import multiarray
File "/usr/local/lib/python3.10/site-packages/numpy/core/multiarray.py", line 10, in <module>
from . import overrides
File "/usr/local/lib/python3.10/site-packages/numpy/core/overrides.py", line 6, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/1675189382222-0/zeppelin_python.py", line 206, in <module>
exec(code, _zcUserQueryNameSpace)
File "<stdin>", line 2, in <module>
File "/u01/.conda/active_env/lib/python3.9/site-packages/statsmodels/tsa/__init__.py", line 1, in <module>
from statsmodels.tools._testing import PytestTester
File "/u01/.conda/active_env/lib/python3.9/site-packages/statsmodels/tools/__init__.py", line 1, in <module>
from .tools import add_constant, categorical
File "/u01/.conda/active_env/lib/python3.9/site-packages/statsmodels/tools/tools.py", line 4, in <module>
import numpy as np
File "/usr/local/lib/python3.10/site-packages/numpy/__init__.py", line 144, in <module>
from . import core
File "/usr/local/lib/python3.10/site-packages/numpy/core/__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.9 from "/u01/.conda/active_env/bin/python3"
* The NumPy version is: "1.22.1"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: No module named 'numpy.core._multiarray_umath'
Why conda is using Numpy in Python 3.10 folder and not the Numpy version installed in Python 3.9? How can repair it?

Tensorflow cannot be imported on ubuntu

System information
Linux Ubuntu 16.04
TensorFlow version: tensorflow-gpu 1.5
Python version: 2.7
Installed using virtualenv: pip
CUDA/cuDNN version: CUDA9.0 & CUDNN7.0
Enviroment
Using anaconda to create a virtual environment named tf.
Error Msg
(tf) ➜ ~ python
Python 2.7.18 |Anaconda, Inc.| (default, Jun 4 2021, 14:47:46)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/xander/anaconda3/envs/tf/lib/python2.7/site-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import *
File "/home/xander/anaconda3/envs/tf/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 52, in <module>
from tensorflow.core.framework.graph_pb2 import *
File "/home/xander/anaconda3/envs/tf/lib/python2.7/site-packages/tensorflow/core/framework/graph_pb2.py", line 6, in <module>
from google.protobuf import descriptor as _descriptor
File "/home/xander/anaconda3/envs/tf/lib/python2.7/site-packages/google/protobuf/descriptor.py", line 113
class DescriptorBase(metaclass=DescriptorMetaclass):
^
SyntaxError: invalid syntax
>>> import tensorflow as tf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/xander/anaconda3/envs/tf/lib/python2.7/site-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import *
File "/home/xander/anaconda3/envs/tf/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
ImportError: cannot import name pywrap_tensorflow
As mentioned in this link, You're running the script in the same directory as TensorFlow. Try to move it out and you should be fine.
For example, Open the Linux command line and execute the script that you've just copied to desktop
Restarting the conda (using miniconda + conda-forge channel) works for me. But I don't know why, though...

Tensorflow installation error (ERROR: Could not find a version that satisfies the requirement tensorflow)

pip3 install --upgrade tensorflowI tried installing tensorflow in my virtual environment and kept on running into this error. I have the necessary prerequisites for pip3(pip 21.0.1) and python3(Python 3.7.9) version and cannot understand why this error is occurring. Any inputs to solve this error would be very much appreciated.
pip3 install --upgrade tensorflow
ERROR: Could not find a version that satisfies the requirement tensorflow
ERROR: No matching distribution found for tensorflow
Error message
pip3 install https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.8.0-py3-none-any.whl
This line worked for me but then this error popped up while checking for the tensorflow version
python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/nvidia/projects/hand_gesture/venv/lib/python3.7/site-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/home/nvidia/projects/hand_gesture/venv/lib/python3.7/site-packages/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "/home/nvidia/projects/hand_gesture/venv/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/home/nvidia/projects/hand_gesture/venv/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 114
def TFE_ContextOptionsSetAsync(arg1, async):
^
SyntaxError: invalid syntax
error terminal description

After installing Tensorflow-gpu i got error Failed to load the native TensorFlow runtime

I have installed redhat 6.6, cuda 9.0, cudnn 7.3.1 and tesnorflow-gpu. but after installation getting following error. My graphics card is nvidia titan v
Traceback (most recent call last):
File "<ipython-input-1-88d96843a926>", line 1, in <module>
import keras
File "/root/anaconda3/lib/python3.7/site-packages/keras/__init__.py", line 3, in <module>
from . import utils
File "/root/anaconda3/lib/python3.7/site-packages/keras/utils/__init__.py", line 6, in <module>
from . import conv_utils
File "/root/anaconda3/lib/python3.7/site-packages/keras/utils/conv_utils.py", line 9, in <module>
from .. import backend as K
File "/root/anaconda3/lib/python3.7/site-packages/keras/backend/__init__.py", line 89, in <module>
from .tensorflow_backend import *
File "/root/anaconda3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 5, in <module>
import tensorflow as tf
File "/root/anaconda3/lib/python3.7/site-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/root/anaconda3/lib/python3.7/site-packages/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "/root/anaconda3/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/root/anaconda3/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/root/anaconda3/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/root/anaconda3/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/root/anaconda3/lib/python3.7/imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "/root/anaconda3/lib/python3.7/imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory
Did you tested your installations of cuda and cudnn:
Nvidia drivers:
$ nvidia-smi
This should display you an overview of the metrics of your GPU
Test cuda:
Check first if:
$ nvcc -V
display the right version of your cuda toolkit
Then you can test it with the following process:
First:
$ cd ~/NVIDIA_CUDA-9.0_Samples
$ make
And then:
$ cd ~/NVIDIA_CUDA-10.0_Samples/bin/x86_64/linux/release
$./deviceQuery
if you have a 'Result: pass' at the end, you're all good!
To test cudnn:
$ cp -r /usr/src/cudnn_samples_v7/ $HOME
$ cd $HOME/cudnn_samples_v7/mnistCUDNN
$ make clean && make
$ ./mnistCUDNN
You should have as result: 'Test passed!'
Tensorflow:
If cuda and cudnn are working, i advice you to install tensorflow in a conda environment using:
conda create --name tf_gpu tensorflow-gpu
For me (and after a lot of problems) it was working very well.
To test it:
from tensorflow.python.client import device_lib
device_lib.list_local_devices()
main sources:
install tensorflow GPU
tensorflow for GPU made easy

ModuleNotFoundError: No module named 'nets'

Hi guys I'm having an error when I'm trying to run the training using this command:
> `python train.py --logtostderr --train_dir=training/
> --pipeline_config_path=training/ssd_mobilenet_v1_coco.config`
Traceback (most recent call last): File "train.py", line 51, in
<module> from object_detection.builders import model_builder File
"C:\tensorflow1\models\research\object_detection\builders\model_builder.py",
line 35, in <module> from object_detection.models import
faster_rcnn_inception_resnet_v2_feature_extractor as frcnn_inc_res
File
"C:\tensorflow1\models\research\object_detection\models \faster_rcnn_inception_resnet_v2_feature_extractor.py",
line 28, in <module> from nets import inception_resnet_v2
ModuleNotFoundError: No module named 'nets
The nets module is in slim folder, you need to add the slim library to PYTHONPATH:
# From tensorflow/models/research/
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
Try Installing pip install nets and also check the site package and environment package for installed folders (It worked for me)