ValueError: Exception encountered when calling layer "tf.__operators__.getitem_20" (type SlicingOpLambda) - dataframe

Question

Followed tensorflow's tutorial and tried to recreate the code myself with multi-label input features and encountered this error. I've recreated the sample code as follow.
DataFrame Creation:
sample_df = pd.DataFrame({"feature_1": [['aa', 'bb','cc'], ['cc', 'dd', 'ee'], ['cc', 'aa', 'ee']], "feature_2": [['aa', 'bb','cc'], ['cc', 'dd', 'ee'], ['cc', 'aa', 'ee']]})
Output:
feature_1 feature_2
0 [aa, bb, cc] [aa, bb, cc]
1 [cc, dd, ee] [cc, dd, ee]
2 [cc, aa, ee] [cc, aa, ee]
Input Layer:
inputs = {}
inputs['feature_1'] = tf.keras.Input(shape=(), name='feature_1', dtype=tf.string)
inputs['feature_2'] = tf.keras.Input(shape=(), name='feature_2', dtype=tf.string)
Output:
{'feature_1': <KerasTensor: shape=(None,) dtype=string (created by layer 'feature_1')>,
'feature_2': <KerasTensor: shape=(None,) dtype=string (created by layer 'feature_2')>}
Preprocessing Layer:
preprocessed = []
for name, column in sample_df.items():
vocab = ['aa', 'bb', 'cc', 'dd', 'ee']
lookup = tf.keras.layers.StringLookup(vocabulary=vocab, output_mode='multi_hot')
print(f'name: {name}')
print(f'vocab: {vocab}\n')
x = inputs[name][:, tf.newaxis]
x = lookup(x)
preprocessed.append(x)
Output:
name: feature_1
vocab: ['aa', 'bb', 'cc', 'dd', 'ee']
name: feature_2
vocab: ['aa', 'bb', 'cc', 'dd', 'ee']
[<KerasTensor: shape=(None, 6) dtype=float32 (created by layer 'string_lookup_27')>,
<KerasTensor: shape=(None, 6) dtype=float32 (created by layer 'string_lookup_28')>]
Model Creation:
preprocessed_result = tf.concat(preprocessed, axis=-1)
preprocessor = tf.keras.Model(inputs, preprocessed_result)
tf.keras.utils.plot_model(preprocessor, rankdir="LR", show_shapes=True)
Output:
<KerasTensor: shape=(None, 12) dtype=float32 (created by layer 'tf.concat_4')>
Error:
preprocessor(dict(sample_df.iloc[:1]))
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
.../sample.ipynb Cell 63' in <cell line: 1>()
----> 1 preprocessor(dict(sample_df.iloc[:1]))
File ~/.local/lib/python3.10/site-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
File ~/.local/lib/python3.10/site-packages/tensorflow/python/framework/constant_op.py:102, in convert_to_eager_tensor(value, ctx, dtype)
100 dtype = dtypes.as_dtype(dtype).as_datatype_enum
101 ctx.ensure_initialized()
--> 102 return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Exception encountered when calling layer "tf.__operators__.getitem_20" (type SlicingOpLambda).
Failed to convert a NumPy array to a Tensor (Unsupported object type list).
Call arguments received:
• tensor=0 [aa, bb, cc]
Name: feature_2, dtype: object
• slice_spec=({'start': 'None', 'stop': 'None', 'step': 'None'}, 'None')
• var=None
Guide Link
Any help with the error or further understand the error will be greatly appreciated.
Thank you very much in advance.

Answer 1

I have created a workaround for anyone who is interested / facing similar issue. This is by means just a workaround and not a solution.
Workaround: Since my multi-hot encodings are binary in nature, I just broke them down each into a feature on its own.
Sample code:
sample_df = pd.DataFrame({"feature_1": [['aa', 'bb','cc'], ['cc', 'dd', 'ee'], ['cc', 'aa', 'ee']]})
feature_1_labels = set()
for i in range(sample_df.shape[0]):
feature_1_labels.update(sample_df.iloc[i]['feature_1'])
for label in sorted(feature_1_labels):
sample_df[label] = 0
for i in range(sample_df.shape[0]):
for label in sample_df.iloc[i]['feature_1']:
sample_df.iloc[i, sample_df.columns.get_loc(label)] = 1
sample_df
Output:
feature_1 aa bb cc dd ee
0 [aa, bb, cc] 1 1 1 0 0
1 [cc, dd, ee] 0 0 1 1 1
2 [cc, aa, ee] 1 0 1 0 1
Note: By doing so will significantly increase the number of input features. Something to keep in mind.
Feel free to let me know a better workaround / if I'm wrong in any way :)