how to decode png with tensorflow 2.x API without using the palette contained in the png file - tensorflow2.0

Question

png files are usually index values assosiated with a default palette. by default, index values can be read by PIL image, for example:
import tensorflow as tf
from PIL import Image
import numpy as np
path = '1.png'
image1 = Image.open(path)
print(image1.mode)
array1 = np.array(image1)
print(array1.shape)
print(set(list(array1.reshape(-1))))
the results:
P
(1024, 543)
{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}
index can be converted to palette colors:
image2 = image1.convert('RGB')
print(image2.mode)
array2 = np.array(image2)
print(array2.shape)
print(set([tuple(x) for x in list(array2.reshape(-1, 3))]))
the results are
RGB
(1024, 543, 3)
{(0, 255, 255), (255, 255, 0), (0, 0, 255), (85, 255, 170), (170, 255, 85), (255, 0, 0), (255, 170, 0), (0, 170, 255), (0, 85, 255), (255, 85, 0), (0, 0, 170)}
the question is, by default, tensorflow 2.x API read palette colors, is there a way to read the index values?
tensor = tf.io.read_file(path)
tensor = tf.image.decode_png(tensor, channels=3)
array3 = tensor.numpy()
print(array3.shape)
print(set([tuple(x) for x in list(array3.reshape(-1, 3))]))
(1024, 543, 3)
{(0, 255, 255), (255, 255, 0), (0, 0, 255), (85, 255, 170), (170, 255, 85), (255, 0, 0), (255, 170, 0), (0, 170, 255), (0, 85, 255), (255, 85, 0), (0, 0, 170)}

Answer 1

my temporary solution is to remove all the palette assosiated with png files. Still looking for more efficient solutions
from PIL import Image
import numpy as np
img = Image.open('1.png')
img = np.array(img)
img = Image.fromarray(img)
img.save('2.png')

Answer 2

Ideal solution
A clean solution would be to re-implement a custom op to decode a PNG without palette conversion.
Currently, the palette conversion in TF for the decode_png-op is done at core level:
// convert palette to rgb(a) if needs be.
if (context->color_type == PNG_COLOR_TYPE_PALETTE)
png_set_palette_to_rgb(context->png_ptr);
Work-around solution
(TF2.X)
As you've mentioned PIL in your sample code, you could wrap the PIL-call with tf.py_function in order to get the desired behavior, like:
def read_png(mask):
def read_fn(p):
return np.asarray(Image.open(p.numpy().decode()))
return tf.py_function(read_fn, [mask], tf.uint8)
and then read a paletted PNG-image without automatic palette application, like:
img = read_png(path)
img= img.numpy()
print(img.shape)
print(set(list(img.reshape(-1))))
Sample output
(1024, 543)
{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}
Note: in TF1.X this works only in graph-mode, and you would have to use tf.py_func to wrap the TF around PIL-function (+ some minor changes).