I am using arabic Bert and passing my training datasets as batches, the error message that I get is that Tensor can not be treated as numpy array but could not detach to numpy using detach().numpy()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function *
outputs = self.distribute_strategy.run(
<ipython-input-9-271b41658d5b>:46 call *
x = self.embed_with_bert(inputs)
<ipython-input-9-271b41658d5b>:39 embed_with_bert *
embds = self.bert_layer(all_tokens[:,0,:].detach().numpy(),
AttributeError: 'Tensor' object has no attribute 'detach'
also when I change the tokenizer to 'tf'
I get the following error:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function *
outputs = self.distribute_strategy.run(
<ipython-input-17-b2127b5212bc>:46 call *
x = self.embed_with_bert(inputs)
<ipython-input-53-b99c90611f94>:39 embed_with_bert *
embds = self.bert_layer([all_tokens[:,0,:],
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py:550 __call__ *
result = self.forward(*input, **kwargs)
/usr/local/lib/python3.6/dist-packages/transformers/modeling_bert.py:691 forward *
input_shape = input_ids.size()
AttributeError: 'list' object has no attribute 'size'
tried this also
embds = self.bert_layer(tf.unstack(all_tokens[:,0,:]),
tf.unstack(all_tokens[:,1,:]),
tf.unstack(all_tokens[:,2,:]))
but it generated:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function *
outputs = self.distribute_strategy.run(
<ipython-input-64-e1ff853b33fb>:55 call *
x = self.embed_with_bert(inputs)
<ipython-input-29-d3665857b399>:44 embed_with_bert *
embds = self.bert_layer(tf.unstack(all_tokens[:,0,:]),tf.unstack(all_tokens[:,1,:]),tf.unstack(all_tokens[:,2,:])) #[:,0,:],all_tokens[:,1,:],all_tokens[:,2,:])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py:1510 unstack **
raise ValueError("Cannot infer num from shape %s" % value_shape)
ValueError: Cannot infer num from shape (None, None)
Example: --> all_tokens are tensor sent as batch to the model but now I want to use it as numpy array to fix the error?
embds = self.bert_layer(all_tokens[:,0,:],
all_tokens[:,1,:],
all_tokens[:,2,:])
though:
this work fine [t is generated from the tokenizer of hugging-face:
a = bert_layer(t["input_ids"], t["attention_mask"], t["token_type_ids"])
Related
I'm having some trouble with a pretty basic model. Am unable to create a pre-processing layer that simply normalizes all features. It is likely that my conceptual understanding of the situation is problematic. My thinking was that the input layer is a list or a dictionary of tf.keras.Input objects, which refer to the input tensors by "name", and indicates their shape and datatypes. Normalizer layers are built by first adapting them over the training dataset, and those layers can be accrued in a list and concatenated. After the input layer is defined, the preprocessing layer takes as input the input layer, and passes its results downstream. Each item in an input layer list is a symbolic representation of the tensors that will flow, and each normalizer will get the right tensors by virtue of having been adapted on that feature.
The error I get is as follows:
TypeError Traceback (most recent call last)
Input In [11], in <cell line: 63>()
59 concatenated_preprocessing_layer = tf.keras.layers.Concatenate(preprocessing_layers)
61 #outputs = concatenated_preprocessing_layer(input_layer.values())
---> 63 outputs = concatenated_preprocessing_layer(all_inputs)
File ~/.pyenv/versions/3.8.5/lib/python3.8/site-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
File ~/.pyenv/versions/3.8.5/lib/python3.8/site-packages/keras/layers/merge.py:509, in Concatenate.build(self, input_shape)
507 shape_set = set()
508 for i in range(len(reduced_inputs_shapes)):
--> 509 del reduced_inputs_shapes[i][self.axis]
510 shape_set.add(tuple(reduced_inputs_shapes[i]))
512 if len(shape_set) != 1:
TypeError: list indices must be integers or slices, not ListWrapper
And the code is as follows:
import tensorflow as tf
filepath='./taxi_data.csv'
CSV_COLUMNS = [
'fare_amount',
'pickup_datetime',
'pickup_longitude',
'pickup_latitude',
'dropoff_longitude',
'dropoff_latitude',
'passenger_count',
'key',
]
LABEL_COLUMN = 'fare_amount'
STRING_COLS = ['pickup_datetime']
NUMERIC_COLS = ['pickup_longitude', 'pickup_latitude',
'dropoff_longitude', 'dropoff_latitude',
'passenger_count']
DEFAULTS = [[0.0], ['na'], [0.0], [0.0], [0.0], [0.0], [0.0], ['na']]
DAYS = ['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat']
def map_features_and_labels(row, target_name):
label = row.pop(target_name)
row.pop('key')
row.pop('pickup_datetime')
return row, label
def create_dataset(filepath, target_name, batch_size=1, mode=tf.estimator.ModeKeys.EVAL, CSV_COLUMNS=None, column_defaults=None ):
dataset = tf.data.experimental.make_csv_dataset(file_pattern=filepath, column_names=CSV_COLUMNS, column_defaults=DEFAULTS, num_epochs=1, batch_size=1)
dataset = dataset.map(lambda X: map_features_and_labels(X, target_name))
if mode == tf.estimator.ModeKeys.TRAIN:
dataset = dataset.shuffle(buffer_size=1000).repeat()
return dataset
train_ds = create_dataset(filepath, target_name=LABEL_COLUMN, batch_size=1, CSV_COLUMNS=CSV_COLUMNS, column_defaults=DEFAULTS )
#The input layer is usually a dictionary of feature_name: Input object
input_layer = {
'pickup_longitude': tf.keras.Input(shape=(0,), name='pickup_longitude', dtype=tf.dtypes.float32),
'pickup_latitude': tf.keras.Input(shape=(0,), name='pickup_latitude', dtype=tf.dtypes.float32),
'dropoff_longitude': tf.keras.Input(shape=(0,), name='dropoff_longitude', dtype=tf.dtypes.float32),
'dropoff_latitude': tf.keras.Input(shape=(0,), name='dropoff_latitude', dtype=tf.dtypes.float32),
'passenger_count': tf.keras.Input(shape=(0,), name='passenger_count', dtype=tf.dtypes.float32),
}
preprocessing_layers = []
all_inputs = []
for column in NUMERIC_COLS:
feature_ds = train_ds.map(lambda X, y: X[column])
normalizer = tf.keras.layers.Normalization(axis=None)
normalizer.adapt(feature_ds)
preprocessing_layers.append(normalizer)
all_inputs.append(tf.keras.Input(shape=(0,), name=column, dtype=tf.dtypes.float32, ))
concatenated_preprocessing_layer = tf.keras.layers.Concatenate(preprocessing_layers)
#outputs = concatenated_preprocessing_layer(input_layer.values())
outputs = concatenated_preprocessing_layer(all_inputs)
And here is some of the data in the taxi_data.csv file
17,2014-10-25 21:39:42 UTC,-73.978713,40.78303,-74.008102,40.73881,2,unused
14.9,2012-08-22 12:01:00 UTC,-73.987667,40.728747,-74.003272,40.715202,2,unused
21.5,2013-12-18 23:26:12 UTC,-74.008969,40.716853,-73.97688,40.780289,2,unused
23.5,2014-10-04 21:58:00 UTC,-73.954153,40.806257,-74.00343,40.731867,2,unused
34.3,2012-12-17 15:23:00 UTC,-73.866917,40.770342,-73.968872,40.757482,2,unused
16.1,2009-09-24 17:37:31 UTC,-73.967549,40.762828,-73.97961,40.723133,2,unused
17.3,2010-04-26 20:52:36 UTC,-73.981381,40.749913,-73.966612,40.691132,2,unused
35,2014-08-13 20:16:00 UTC,-73.866107,40.771245,-74.013987,40.676437,2,unused
17.3,2010-12-30 17:55:00 UTC,-73.997803,40.725982,-73.982382,40.772225,2,unused
I was able to get this to work. Like I suspected, it was my conceptual understanding that was the issue. Specifically, I wasn't correctly hooking up the Input (input_placeholder) to the normalizer. The modified code is below:
preprocessing_layers = []
all_inputs = []
for column in NUMERIC_COLS:
normalizer = get_normalization_layer(column, train_ds)
input_placeholder = tf.keras.Input(shape=(1,), name=column, dtype=tf.dtypes.float32, )
encoded_feature = normalizer(input_placeholder)
preprocessing_layers.append(encoded_feature)
all_inputs.append(input_placeholder)
concatenated_preprocessing_layer = tf.keras.layers.concatenate(preprocessing_layers)
#outputs = concatenated_preprocessing_layer(input_layer.values())
preprocessing_new_model = tf.keras.Model(inputs=all_inputs, outputs=concatenated_preprocessing_layer)
preprocessing_new_model(train_features)
You need to concatenate preprocessing_layers and all_inputs by using the code below:
concatenated_preprocessing_layer = tf.keras.layers.Concatenate((preprocessing_layers,all_inputs))
As you have used
concatenated_preprocessing_layer = tf.keras.layers.Concatenate(preprocessing_layers)
You can concatenate all_inputs by using:
outputs =tf.keras.layers.Concatenate((concatenated_preprocessing_layer,all_inputs))
Please refer to this working gist for your reference.
I've implemented an accumulated gradient optimizer but when I want to train model it gives me this error.So what is the problem?
The idea behind gradient accumulation is that it calculates the loss and gradients after each mini-batch, but instead of updating the model parameters, it waits and accumulates the gradients over consecutive batches. And then ultimately updates the parameters based on the cumulative gradient after a specified number of batches. It serves the same purpose as having a mini-batch with higher number of images.
class AccumAdamOptimizer(keras.optimizers.Optimizer):
def __init__(self,learning_rate=0.001,steps=1,beta_1=0.9,beta_2=0.999,epsilon=1e-
7,amsgrad=False,name='AccumAdamOptimizer',**kwargs):
super(AccumAdamOptimizer, self).__init__(name, **kwargs)
self._set_hyper('learning_rate', kwargs.get('lr', learning_rate))
self._set_hyper('decay', self._initial_decay)
self._set_hyper('beta_1', beta_1)
self._set_hyper('beta_2', beta_2)
self.epsilon = epsilon
self.amsgrad = amsgrad
self.iterations = tf.Variable(1, dtype='int64', name='iterations')
self.steps = steps
self.condition = tf.math.equal(self.iterations % self.steps , 0)
def _create_slots(self, var_list):
for var in var_list:
self.add_slot(var, 'm')
for var in var_list:
self.add_slot(var, 'v')
# if self.amsgrad:
# for var in var_list:
# self.add_slot(var, 'vhat')
for var in var_list:
self.add_slot(var, "ag") #accumulated gradient
def _resource_apply_dense(self, grad, var, apply_state=None):
var_device, var_dtype = var.device, var.dtype.base_dtype
m = self.get_slot(var, 'm')
v = self.get_slot(var, 'v')
ag = self.get_slot(var, 'ag')
lr=self._get_hyper('learning_rate', var_dtype)
beta1= self._get_hyper('beta_1', var_dtype)
beta2=self._get_hyper('beta_2', var_dtype)
t = tf.cast(self.iterations, tf.float32)
beta1_power=tf.math.pow(beta1, t )
beta2_power=tf.math.pow(beta2, t)
if self.condition:
new_m = beta1 * m + (1-beta1) * ag
new_v = beta2 * v + (1-beta2) * tf.math.square(ag)
m_corrected = new_m/(1-beta1_power)
v_corrected = new_v/(1-beta2_power)
new_var = var - lr * m_corrected/(tf.math.sqrt(v_corrected)+self.epsilon))
var.assign(new_var) # update weights
shape_var = tf.shape(var)
ag.assign(tf.zeros(shape_var, dtype=var.dtype))
m.assign(m_corrected)
v.assign(v_corrected)
self.iterations.assign_add(1)
else:
ag.assign_add(grad)
self.iterations.assign_add(1)
def _resource_apply_sparse(self, grad, var):
raise NotImplementedError
def get_config(self):
config = super(AccumAdamOptimizer, self).get_config()
config.update({
'learning_rate': self._serialize_hyperparameter('learning_rate'),
'decay': self._initial_decay,
'beta_1': self._serialize_hyperparameter('beta_1'),
'beta_2': self._serialize_hyperparameter('beta_2'),
'epsilon': self.epsilon,
'amsgrad': self.amsgrad,
})
return config
This is my complete error :
TypeError: in user code:
/usr/local/lib/python3.7/dist-packages/keras/engine/training.py:830 train_function *
return step_function(self, iterator)
/usr/local/lib/python3.7/dist-packages/keras/engine/training.py:813 run_step *
outputs = model.train_step(data)
/usr/local/lib/python3.7/dist-packages/keras/engine/training.py:774 train_step *
self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:530 minimize **
return self.apply_gradients(grads_and_vars, name=name)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:668 apply_gradients
apply_state)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:732 _distributed_apply
with ops.control_dependencies([control_flow_ops.group(update_ops)]):
/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/control_flow_ops.py:2966 group
"'%s' with type '%s'" % (inp, type(inp)))
TypeError: Expected tf.group() expected Tensor arguments not 'None' with type '<class 'NoneType'>'
I am trying to convert facebooks' fast-text model to tensorflow-hub format. I have attached two main files for the purpose.
def _compute_ngrams(word, min_n=1, max_n=3):
BOW, EOW = ('<', '>') # Used by FastText to attach to all words as prefix and suffix
ngrams = [] # batch_size, n_words, maxlen
shape = word.shape # batch_size, n_sentenes, n_words
maxlen = 0
for b in range(shape[0]): # batch
ngram_b = []
for w in word[b]:
ngram = []
extended_word = BOW + "".join( chr(x) for x in bytearray(w)) + EOW
if w.decode("utf-8") not in global_vocab:
for ngram_length in range(min_n, min(len(extended_word), max_n) + 1):
for i in range(0, len(extended_word) - ngram_length + 1):
ngram.append(extended_word[i:i + ngram_length])
else:
ngram.append(w.decode("utf-8") )
ngram_b.append(ngram)
maxlen = max(maxlen, len(ngram))
ngrams.append(ngram_b)
for batches in ngrams:
for words in batches:
temp = maxlen
r = []
while temp > len(words):
r.append("UNK")
temp = temp - 1
words.extend(r)
return ngrams
def make_module_spec(vocabulary_file, vocab_size, embeddings_dim=300,
num_oov_buckets=1):
def module_fn():
"""Spec function for a token embedding module."""
words = tf.placeholder(shape=[None, None], dtype=tf.string, name="tokens")
tokens = tf.py_func(_compute_ngrams, [words], tf.string)
embeddings_var = tf.get_variable(
initializer=tf.zeros([vocab_size + num_oov_buckets, embeddings_dim]),
name=EMBEDDINGS_VAR_NAME,
dtype=tf.float32
)
lookup_table = tf.contrib.lookup.index_table_from_file(
vocabulary_file=vocabulary_file,
num_oov_buckets=num_oov_buckets,
)
ids = lookup_table.lookup(tokens)
#combined_embedding = tf.reduce_mean(tf.nn.embedding_lookup(params=embeddings_var, ids=ids), axis=2)
combined_embedding = tf.nn.embedding_lookup(params=embeddings_var, ids=ids)
hub.add_signature("default", {"tokens": words},
{"default": combined_embedding})
return hub.create_module_spec(module_fn)
The model is created as expected with tf-hub format.
But when I try to use the above created model, I get this error.
The sample testing code to use tf-hub model created above is attached below.
with tf.Graph().as_default():
module_url = "/home/sahil_wadhwa/tf-hub/tf_sent"
embed = hub.Module(module_url)
embeddings = embed([["Indian", "American"], ["Hello", "World"]])
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())
result = sess.run(embeddings)
print(result)
print(result.shape)
The error I get is here.
Traceback (most recent call last):
File "/home/sahil_wadhwa/.local/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 195, in __call__
raise ValueError("callback %s is not found" % token)
ValueError: callback pyfunc_0 is not found
[[{{node module_apply_default/PyFunc}} = PyFunc[Tin=[DT_STRING], Tout=[DT_STRING], token="pyfunc_0", _device="/job:localhost/replica:0/task:0/device:CPU:0"](Const)]]
Been stuck with this for a long time, any help here would be useful.
Thanks in advance.
Answered on https://github.com/tensorflow/hub/issues/222:
Hi Sahil,
the issue here is that tf.py_func cannot be serialized. Serializing
arbitrary Python functions is not supported (for multiple reasons).
I see you are creating ngrams from a token if not present in the vocabulary
(btw, are the ngrams actually in the FastText vocabulary to be looked up or
does it contain only full words?).
One way of solving this could be to rewrite your _compute_ngrams function
in TensorFlow (maybe you could use this directly or at least get some
inspiration:
https://www.tensorflow.org/tfx/transform/api_docs/python/tft/ngrams).
I'm implementing a Seq2Seq model with TensorFlow. My code works using the Greedy Decoder, but when I was using BeamSearchDecoder to improve the performance, I encountered this error:
Traceback (most recent call last):
File "/Users/MichaelChen/Projects/CN-isA-Relation-Extraction/isa_seq2seq/predict.py", line 83, in <module>
out_file='result/result_wc_4.out', checkpoint=checkpoint)
File "/Users/MichaelChen/Projects/CN-isA-Relation-Extraction/isa_seq2seq/predict.py", line 48, in predict
loader = tf.train.import_meta_graph(checkpoint + '.meta')
File "/usr/local/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1686, in import_meta_graph
**kwargs)
File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/meta_graph.py", line 504, in import_scoped_meta_graph
producer_op_list=producer_op_list)
File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 283, in import_graph_def
raise ValueError('No op named %s in defined operations.' % node.op)
ValueError: No op named GatherTree in defined operations.
This error occurred when I used the infer module to generate
outputs:
with tf.Session(graph=loaded_graph) as sess:
loader = tf.train.import_meta_graph(checkpoint + '.meta')
loader.restore(sess, checkpoint)
input_data = loaded_graph.get_tensor_by_name('inputs:0')
logits = loaded_graph.get_tensor_by_name('predictions:0')
src_seq_len = loaded_graph.get_tensor_by_name('source_sequence_length:0')
tgt_seq_len = loaded_graph.get_tensor_by_name('target_sequence_length:0')
for i in range(len(text)):
if len(text[i].strip()) < 1:
continue
text_seq = src2seq_word(text[i], True)
answer_logits = sess.run(logits, {input_data: [text_seq] * batch_size,
tgt_seq_len: [len(text_seq)] * batch_size,
src_seq_len: [len(text_seq)] * batch_size}
)[0]
pred_res = "".join([pp.id2c[i] for i in answer_logits if i != pad and i != eos])
Program failed at loader = tf.train.import_meta_graph(checkpoint + '.meta')
I don't know if I handle the outputs of the decoder right, so here is the corresponding code:
# 5. Predicting decoder
# Share params with Training Deocder
tiled_dec_start_state = tf.contrib.seq2seq.tile_batch(encoder_state, beam_width)
tiled_encoder_outputs = tf.contrib.seq2seq.tile_batch(encoder_outputs, beam_width)
tiled_src_seq_len = tf.contrib.seq2seq.tile_batch(src_seq_len, beam_width)
with tf.variable_scope('decode', reuse=True):
batch_size_tensor = tf.constant(batch_size)
beam_decoder_cell = get_decoder_cell(tiled_encoder_outputs, tiled_src_seq_len, 2 * num_units)
beam_initial_state = beam_decoder_cell.zero_state(batch_size_tensor * beam_width, tf.float32)
beam_initial_state = beam_initial_state.clone(cell_state=tiled_dec_start_state)
start_tokens = tf.tile(tf.constant([c2id['<GO>']], dtype=tf.int32), [batch_size], name='start_tokens')
predicting_decoder = tf.contrib.seq2seq.BeamSearchDecoder(
cell=beam_decoder_cell,
embedding=decoder_embeddings,
start_tokens=start_tokens,
end_token=c2id['<EOS>'],
initial_state=beam_initial_state,
beam_width=beam_width,
output_layer=output_layer
)
predicting_decoder_output, _, _ = tf.contrib.seq2seq.dynamic_decode(decoder=predicting_decoder, maximum_iterations=max_tgt_seq_len)
Handling outputs:
training_decoder_output, predicting_decoder_output = seq2seq_model(params...)
training_logits = tf.identity(training_decoder_output.rnn_output, name='logits')
predicting_logits = tf.identity(predicting_decoder_output.predicted_ids[:,:,0], name='predictions')
Also, I found something in the nmt model in
https://github.com/tensorflow/nmt/blob/77e6c55052ba31a8d733c94bb820d091c8156d35/nmt/model.py (line 391)
if beam_width > 0:
logits = tf.no_op()
sample_id = outputs.predicted_ids
else:
logits = outputs.rnn_output
sample_id = outputs.sample_id
Is this has something to do with my error?
Thanks in advance!
I've written some code in Tensorflow to compute the edit-distance between one string and a set of strings. I can't figure out the error.
import tensorflow as tf
sess = tf.Session()
# Create input data
test_string = ['foo']
ref_strings = ['food', 'bar']
def create_sparse_vec(word_list):
num_words = len(word_list)
indices = [[xi, 0, yi] for xi,x in enumerate(word_list) for yi,y in enumerate(x)]
chars = list(''.join(word_list))
return(tf.SparseTensor(indices, chars, [num_words,1,1]))
test_string_sparse = create_sparse_vec(test_string*len(ref_strings))
ref_string_sparse = create_sparse_vec(ref_strings)
sess.run(tf.edit_distance(test_string_sparse, ref_string_sparse, normalize=True))
This code works and when run, it produces the output:
array([[ 0.25],
[ 1. ]], dtype=float32)
But when I attempt to do this by feeding the sparse tensors in through sparse placeholders, I get an error.
test_input = tf.sparse_placeholder(dtype=tf.string)
ref_input = tf.sparse_placeholder(dtype=tf.string)
edit_distances = tf.edit_distance(test_input, ref_input, normalize=True)
feed_dict = {test_input: test_string_sparse,
ref_input: ref_string_sparse}
sess.run(edit_distances, feed_dict=feed_dict)
Here is the error traceback:
Traceback (most recent call last):
File "<ipython-input-29-4e06de0b7af3>", line 1, in <module>
sess.run(edit_distances, feed_dict=feed_dict)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 372, in run
run_metadata_ptr)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 597, in _run
for subfeed, subfeed_val in _feed_fn(feed, feed_val):
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 558, in _feed_fn
return feed_fn(feed, feed_val)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/client/session.py", line 268, in <lambda>
[feed.indices, feed.values, feed.shape], feed_val)),
TypeError: zip argument #2 must support iteration
Any idea what is going on here?
TL;DR: For the return type of create_sparse_vec(), use tf.SparseTensorValue instead of tf.SparseTensor.
The problem here comes from the return type of create_sparse_vec(), which is tf.SparseTensor, and is not understood as a feed value in the call to sess.run().
When you feed a (dense) tf.Tensor, the expected value type is a NumPy array (or certain objects that can be converted to an array). When you feed a tf.SparseTensor, the expected value type is a tf.SparseTensorValue, which is similar to a tf.SparseTensor but its indices, values, and shape properties are NumPy arrays (or certain objects that can be converted to arrays, like the lists in your example.
The following code should work:
def create_sparse_vec(word_list):
num_words = len(word_list)
indices = [[xi, 0, yi] for xi,x in enumerate(word_list) for yi,y in enumerate(x)]
chars = list(''.join(word_list))
return tf.SparseTensorValue(indices, chars, [num_words,1,1])