The code below used to work last year, but updates in keras/tensorflow/numpy broke it. It now outputs the exception below. Does anyone know how to make it work again?
I'm using:
Tensorflow 2.4.1
Keras 2.4.3
Numpy 1.20.1
Python 3.9.1
import numpy as np
from keras.layers import LSTM, Embedding, Input, Bidirectional
dim = 30
max_seq_length = 40
vecs = np.random.rand(45,dim)
input_layer = Input(shape=(max_seq_length,))
embedding_layer = Embedding(len(vecs), dim, weights=[vecs], input_length=max_seq_length, trainable=False, name="layerA")(input_layer)
lstm_nobi = LSTM(max_seq_length, return_sequences=True, activation="linear", name="layerB")
lstm = Bidirectional(lstm_nobi, name="layerC")(embedding_layer)
Complete output of the script above: https://pastebin.com/DsQNWVwz
Shortened output:
2021-02-10 17:51:13.037468: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-02-10 17:51:13.037899: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-02-10 17:51:13.038418: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
Traceback (most recent call last):
File "/run/media/volker/DATA/configruns/load/./test.py", line 13, in <module>
lstm = Bidirectional(lstm_nobi, name="layerC")(embedding_layer)
... omitted, see pastebin ...
File "/usr/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 852, in __array__
raise NotImplementedError(
NotImplementedError: Cannot convert a symbolic Tensor (layerC/forward_layerB/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported
Installing numpy 1.19.5 works for me even with Python 3.9
pip install -U numpy==1.19.5
My environment is Windows and since I do not have Visual C++ Compiler installed, I ride on third party whl file installation
pip install -U https://mirrors.aliyun.com/pypi/packages/bc/40/d6f7ba9ce5406b578e538325828ea43849a3dfd8db63d1147a257d19c8d1/numpy-1.19.5-cp39-cp39-win_amd64.whl#sha256=0eef32ca3132a48e43f6a0f5a82cb508f22ce5a3d6f67a8329c81c8e226d3f6e
Solution: Use Python 3.8, because Python 3.9 is not supported by Tensorflow.
Related
sequence_input = Input(shape=(max_len,), dtype="int32")
embedded_sequences = Embedding(vocab_size, 128, input_length=max_len,
mask_zero=True)(sequence_input)
lstm = Bidirectional(LSTM(64, dropout=0.5, return_sequences=True))(embedded_sequences)
The third line of code gives the following error:
Cannot convert a symbolic Tensor (bidirectional/forward_lstm/strided_slice:0) to a numpy array.
This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported
When I was looking for a solution to the same error as me, I saw a lot of answers on stackoverflow telling me to lower the numpy version to less than 1.20.
But since I use featuretools, I need to set the numpy version to 1.2 or higher.
So, my question is, is there currently no way to fix this error without downgrading the numpy version?
(my tensorflow version is 2.3.0, numpy version is 1.23)
I solved this error by changing the cuda version and installing the latest tensorflow. (2.3 -> 2.10)
During running LSTM model , I'm getting this error when my code is calling the following function:
model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))
error is:
" a NumPy call, which is not supported".format(self.name))
NotImplementedError: Cannot convert a symbolic Tensor
(lstm/strided_slice:0) to a numpy array. This error may indicate that
you're trying to pass a Tensor to a NumPy call, which is not
supported"
Im using-
tensorflow 2.4.0
Numpy 1.20.0
pandas=1.3.4
python 3.7.3
platform: Pi 4 + 32bit OS
Try to use the lower version of Numpy i.e 1.19.5 or 1.19 using
pip install numpy == 1.19.5
Or
pip install numpy == 1.19
This problem usually occurs when you try to evaluate something using a Symbolic Tensor ( For Example Tensor array etc) with Non-Symbolic Types like NumPy, It is quite difficult to avoid because we might use symbolic tensors like tf.zeros() or tf.ones() as the parameters of your model, which use NumPy internally. Try for a Lower version of NumPy or the Latest Version of Tensorflow version i.e. Tensorflow Core v2.6.0
I solved this problem by changing my OS from 32 bit OS to 64 bit OS. I tried with 32 bit OS+ numpy 1.19.5 + tf 2.4.0; I was receiving this error constantly. later I changed the OS to 64 bit with numpy 1.19.5 and tf 2.4.0 and its working now.
I am trying to convert a TF 2.0 saved_model to tensorRT on the Jetson Nano.
The model was saved in TF 2.0.0. The nano has Jetpack 4.2.2 w/ TensorRT __ and Tensorflow 1.14 (that is the latest Tensorflow release for Jetson).
I have been following the instuctions from here which describe how to convert a TF 2.0.0 saved_model into TensorRT.
Below is my code:
import tensorflow as tf
from tensorflow.python.compiler.tensorrt import trt_convert as trt
tf.enable_eager_execution()
converter = trt.TrtGraphConverterV2(input_saved_model_dir=input_saved_model_dir)
converter.convert()
converter.save(output_saved_model_dir)
saved_model_loaded = tf.saved_model.load(
output_saved_model_dir, tags=[tag_constants.SERVING])
graph_func = saved_model_loaded.signatures[
signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
frozen_func = convert_to_constants.convert_variables_to_constants_v2(
graph_func)
def wrap_func(*args, **kwargs):
# Assumes frozen_func has one output tensor
return frozen_func(*args, **kwargs)[0]
output = wrap_func(input_data).numpy()
It seems to start converting successfully. However I get an KeyError: 'serving_default' error when it reaches the convert_to_tensor line. My complete printout is below found here (too long for SO), but the python traceback appears below. How can I fix this?
Thanks!
printout summary (complete printout here):
Traceback (most recent call last):
File "tst.py", line 38, in <module>
convert_savedmodel()
File "tst.py", line 24, in convert_savedmodel
converter.convert()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/compiler/tensorrt/trt_convert.py", line 956, in convert
func = self._saved_model.signatures[self._input_saved_model_signature_key]
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/signature_serialization.py", line 196, in __getitem__
return self._signatures[key]
KeyError: 'serving_default'
I can see two problems in your experiment:
You are using TF-TRT 2.0 API while having TF 1.14 installed. That is not supported. If you have TF 1.14 installed on your system, then you would need to use TF-TRT 1.x API.
TF Models saved in TF2.0 are not compatible with TF1.14 according to https://www.tensorflow.org/guide/versions
If you only have access to TF1.14, I suggest to re-generate the graph in TF1.14 and save the model there before applying TF-TRT, and then use TF-TRT 1.x API.
The code is like this:
import tensorflow as tf
from keras_contrib.layers import CRF
from tensorflow import keras
def create_model(max_seq_len, adapter_size=64):
"""Creates a classification model."""
# adapter_size = 64 # see - arXiv:1902.00751
# create the bert layer
with tf.io.gfile.GFile(bert_config_file, "r") as reader:
bc = StockBertConfig.from_json_string(reader.read())
bert_params = map_stock_config_to_params(bc)
bert_params.adapter_size = adapter_size
bert = BertModelLayer.from_params(bert_params, name="bert")
input_ids = keras.layers.Input(shape=(max_seq_len,), dtype='int32', name="input_ids")
# token_type_ids = keras.layers.Input(shape=(max_seq_len,), dtype='int32', name="token_type_ids")
# output = bert([input_ids, token_type_ids])
bert_output = bert(input_ids)
print("bert_output.shape: {}".format(bert_output.shape)) # (?, 100, 768)
crf = CRF(len(tag2idx))
logits = crf(bert_output)
model = keras.Model(inputs=input_ids, outputs=logits)
model.build(input_shape=(None, max_seq_len))
# load the pre-trained model weights
load_stock_weights(bert, bert_ckpt_file)
# freeze weights if adapter-BERT is used
if adapter_size is not None:
freeze_bert_layers(bert)
model.compile('adam', loss=crf.loss_function, metrics=[crf.accuracy])
model.summary()
return model
I am using tensorflow keras and also use keras_contrib package, to do NER. it seems the tensorflow keras package does not work well with keras_contrib package.
The Traceback information is listed below:
Traceback (most recent call last):
File "F:/_gitclone3/bert_examples/bert_ner_example_eval.py", line 120, in <module>
model = create_model(max_seq_len, adapter_size=adapter_size)
File "F:/_gitclone3/bert_examples/bert_ner_example_eval.py", line 101, in create_model
logits = crf(bert_output)
File "C:\Users\yuexiang\Anaconda3\lib\site-packages\keras\engine\base_layer.py", line 443, in __call__
previous_mask = _collect_previous_mask(inputs)
File "C:\Users\yuexiang\Anaconda3\lib\site-packages\keras\engine\base_layer.py", line 1311, in _collect_previous_mask
mask = node.output_masks[tensor_index]
AttributeError: 'Node' object has no attribute 'output_masks'
How do I use CRF with tensorflow keras?
I run into a similar problem and spent a lot of time trying to get things to work. Here's what worked for me using python 3.6.5:
Seqeval:
pip install seqeval==0.0.5
Keras:
pip install keras==2.2.4
Keras-contrib (2.0.8):
git clone https://www.github.com/keras-team/keras-contrib.git
cd keras-contrib
python setup.py install
TensorFlow:
pip install tensorflow==1.14.0
Do pip list to make sure you have actually installed those versions (eg pip seqeval may automatically update your keras)
Then in your code import like so:
from keras.models import *
from keras.layers import LSTM, Embedding, Dense, TimeDistributed, Dropout, Bidirectional, Input
from keras_contrib.layers import CRF
#etc.
Hope this helps, good luck!
You can try tensorflow add-ons.(If you are using tensorflow version 2).
You can try tf-crf-layer (if you are using tensorflow==1.15.0)
They have it mentioned on their README.
git clone https://www.github.com/keras-team/keras-contrib.git
cd keras-contrib
python convert_to_tf_keras.py
USE_TF_KERAS=1 python setup.py install
I have gone through the possible solutions, mentioning which worked for me:
Install tf2crf (https://pypi.org/project/tf2crf/): It provides a simple CRF layer for TensorFlow 2 keras.
Use TensorFlow SIG Addons: ( https://www.tensorflow.org/addons/api_docs/python/tfa/layers/CRF): It provides the functionality that is not available in core TensorFlow.
I am trying to run the code:
from keras.datasets import imdb as im
from keras.preprocessing import sequence as seq
from keras.models import Sequential
from keras.layers import Embedding
from keras.layers import LSTM
from keras.layers import Dense
train_set, test_set = im.load_data(num_words = 10000)
X_train, y_train = train_set
X_test, y_test = test_set
X_train_padded = seq.pad_sequences(X_train, maxlen = 100)
X_test_padded = seq.pad_sequences(X_test, maxlen = 100)
model = Sequential()
model.add(Embedding(input_dim=10000, output_dim=128))
model.add(LSTM(units=128))
model.add(Dense(units=1, activation='sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='sgd',
metrics=['accuracy'])
scores = model.fit(X_train_padded,y_train)
When I run the code, it gives me a message:
I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations: SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.
I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 4. Tune using inter_op_parallelism_threads for best performance.
I don't understand what the issue is and what I am supposed to do next. I installed the "tenserflow" package (1.14.0) but that doesn't solve the issue.
I have looked at this reference but I don't know what I am looking for:
https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions
Can someone please help me. Thanks.
my config: osx-64, MacOS Mojave v.10.14.6, Python 3.7 with Spyder with Anaconda, conda version : 4.7.12
You can ignore the message and everything will work fine.
As far as I can gather from https://github.com/tensorflow/tensorflow/pull/24782/commits/7faefa4bb665e115cc744d7895a407338624993f, when TensorFlow is compiled with MKL-DNN support (which it is, according to your message), MKL-DNN will take care of using all available CPU performance features. So it doesn't matter that TensorFlow wasn't compiled to use them.
This might not be answering the exact question you have put, but I had a very similar error message when running a similar task.
In addition to the error message above, I also had the following error message:
OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
Error was solved with:
conda install nomkl
This is as per this stackoverflow post