Tensorflow Object detection API: printing the detected objects label name - tensorflow

Question

I am following Nicollas renotte's tutorial on Realtime hand-sign detection with TensorFlow and OpenCV and finished the code.
import cv2
import numpy as np
import time
category_index = label_map_util.create_category_index_from_labelmap(ANNOTATION_PATH+'/label_map.pbtxt')
cap = cv2.VideoCapture(0)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
while True:
ret, frame = cap.read()
image_np = np.array(frame)
input_tensor = tf.convert_to_tensor(np.expand_dims(image_np, 0), dtype=tf.float32)
detections = detect_fn(input_tensor)
num_detections = int(detections.pop('num_detections'))
detections = {key: value[0, :num_detections].numpy()
for key, value in detections.items()}
detections['num_detections'] = num_detections
# detection_classes should be ints.
detections['detection_classes'] = detections['detection_classes'].astype(np.int64)
label_id_offset = 1
image_np_with_detections = image_np.copy()
viz_utils.visualize_boxes_and_labels_on_image_array(
image_np_with_detections,
detections['detection_boxes'],
detections['detection_classes']+label_id_offset,
detections['detection_scores'],
category_index,
use_normalized_coordinates=True,
max_boxes_to_draw=5,
min_score_thresh=.5,
agnostic_mode=False)
cv2.imshow('object detection', cv2.resize(image_np_with_detections, (800, 600)))
if cv2.waitKey(1) & 0xFF == ord('q'):
cap.release()
break
cap.release()
detections = detect_fn(input_tensor)
so this code is running fine and recognize the hand sign and draw a box around the hand-sign and labels it but I want to print the name of the recognized hand-sign in the terminal itself ( for using it with pyttx3 to speak out the sign that's detected)
I tried just printing the detections['detection_classes'] but that only give some sort of array as output can anyone explain how I can print the name of the object detected with the score?
Thanks in advance, first post on Stack Overflow so please go easy on me

Answer 1

detections['detection_classes'] returns the category id of each bouding box detected.
A category index is a dictionary that maps integer ids to dicts
containing categories, e.g. {1: {'id': 1, 'name': 'dog'}, 2: {'id': 2, 'name': 'cat'}, ...}.
So if you print category_index, you will get something like this:
{1: {'id': 1, 'name': 'Aa'}, 2: {'id': 2, 'name': 'Bb'}, ...}
assuming you are dealing with hand signs of alphabets.
With this knowledge, it is easy to print the label for the hand-sign detected.
# flatten the category_index to a single dictionary
category_dict = {value.get('id'):value.get('name') for _,value in category_index.items()}
detected_signs = []
for sign_index in detections['detection_classes']:
sign_label = category_dict.get(sign_index)
detected_signs.append(sign_label)
print(detected_signs)
# Feed detected_signs to downstream system like pyttx3 to speak out the sign