Getting ValueError: Data Params Error when using masterful.trainer.train - tensorflow

Question

I followed the tutorial here to try to train my model using CIFAR-100. But I'm getting this error. What do I do?
ValueError: Data Params Error: The dataset label shape (100,) does not match the
number of classes (10) in the dataset. Please ensure the dataset
labels have 10 classes, or change the number of classes
to match the dataset.
Here is my setup, adapted from the tutorial but for CIFAR-100.
import tensorflow as tf
import tensorflow_datasets as tfds
import masterful
masterful = masterful.register()
TRAINING_PERCENTAGE = 5
(training_dataset,
test_dataset) = tfds.load('cifar100',
as_supervised=True,
split=[f'train[:{TRAINING_PERCENTAGE}%]', 'test'],
with_info=False)
def sparse_to_dense(image, label):
label = tf.cast(label, tf.int32)
one_hot_label = tf.one_hot(label, depth=100)
return image, one_hot_label
training_dataset = training_dataset.map(sparse_to_dense,
num_parallel_calls=tf.data.AUTOTUNE)
test_dataset = test_dataset.map(sparse_to_dense,
num_parallel_calls=tf.data.AUTOTUNE)
def get_model():
model = tf.keras.models.Sequential()
model.add(
tf.keras.layers.experimental.preprocessing.Rescaling(1. / 255,
input_shape=(32, 32,
3)))
model.add(tf.keras.layers.Conv2D(
16,
(3, 3),
activation='relu',
))
model.add(tf.keras.layers.GlobalAveragePooling2D())
model.add(tf.keras.layers.Dense(10))
return model
model = get_model()
model_params = masterful.architecture.learn_architecture_params(
model=model,
task=masterful.enums.Task.CLASSIFICATION,
input_range=masterful.enums.ImageRange.CIFAR10_TORCH,
num_classes=10,
prediction_logits=True,
)
training_dataset_params = masterful.data.learn_data_params(
dataset=training_dataset,
task=masterful.enums.Task.CLASSIFICATION,
image_range=masterful.enums.ImageRange.CIFAR10_TORCH,
num_classes=10,
sparse_labels=False,
)
optimization_params = masterful.optimization.learn_optimization_params(
model,
model_params,
training_dataset,
training_dataset_params,
)
# This is a set of parameters learned on CIFAR10 for
# small sized models.
regularization_params = masterful.regularization.parameters.CIFAR10_SMALL
training_report = masterful.training.train(
model,
model_params,
optimization_params,
regularization_params,
None,
training_dataset,
training_dataset_params,
)

Answer 1

Both of you are correct, regularization and classification is support 10 classes you may consider multiple steps approaches.
[ Sample ]:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: DataSet
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
TRAINING_PERCENTAGE = 5
(training_dataset, test_dataset) = tfds.load('cifar100', as_supervised=True, split=[f'train[:{TRAINING_PERCENTAGE}%]', 'test'], with_info=False)
training_dataset = training_dataset.map(sparse_to_dense, num_parallel_calls=tf.data.AUTOTUNE)
test_dataset = test_dataset.map(sparse_to_dense, num_parallel_calls=tf.data.AUTOTUNE)
task = masterful.enums.Task.CLASSIFICATION
[ ENUM ]:
# Task.CLASSIFICATION
# Task.CLASSIFICATION
# Task.BINARY_CLASSIFICATION
# Task.MULTILABEL_CLASSIFICATION
# Task.DETECTION
# Task.LOCALIZATION
# Task.SEMANTIC_SEGMENTATION
# Task.INSTANCE_SEGMENTATION
# Task.KEYPOINT_DETECTION
[ Statistics ]:
# #enum.unique
# class DatasetStatistics(enum.Enum):
# """An enum for datasets statistics required for Masterful's normalization."""
# CIFAR_10_MEAN = [0.4914, 0.4822, 0.4465]
# CIFAR_10_STD = [0.247, 0.243, 0.261]
# CIFAR_10_BGR_255 = [113.8575, 122.961, 125.307]
# CIFAR_100_MEAN = [0.5071, 0.4867, 0.4408]
# CIFAR_100_STD = [0.2675, 0.2565, 0.2761]
# CIFAR_100_BGR_255 = [129.3105, 124.1085, 112.404]
# IMAGENET_MEAN = [0.485, 0.456, 0.406]
# IMAGENET_STD = [0.229, 0.224, 0.225]
# IMAGENET_MEAN_BGR_255 = [103.939, 116.779, 123.68]
[ Params ]:
# ArchitectureParams(task=<Task.CLASSIFICATION: 'classification'>, num_classes=10, ensemble_multiplier=1, custom_objects={}, model_config=None,
# backbone_only=False, input_shape=(32, 32, 3), input_range=<ImageRange.CIFAR100_TORCH: 'CIFAR100_TORCH'>, input_dtype=tf.float32, input_channels_last=True,
# prediction_logits=True, prediction_dtype=tf.float32, prediction_structure=<TensorStructure.SINGLE_TENSOR: 'single_tensor'>, prediction_shape=TensorShape([10]))

Answer 2

This Value Error is explaining the problem its the message.
The dataset labels, being CIFAR-100, are one hot vectors of length 100.
But the call to "masterful.data.learn_data_params" gets passed the value 10.
Update your model architecture with:
model.add(tf.keras.layers.Dense(100))
and your call to learn_data_params with:
num_classes=100,