I just want to get a list of bidirectional approaches to
flatten 3 channel image
reconstruct from the flattened array
Can you input it, thank you.
Here is one example from me, I am not sure if it is correct or not:
image = PIL.......
image = np.array(image)
origin_shape = image.shape
image_flatten = image.ravel()
image_flatten = tf.cast(image_flatten, dtype=tf.float32)
image_flatten = image_flatten[np.newaxis][:]
.....
............
rebuilt_image= np.reshape(image_flatten, origin_shape)
Related
My code is:
randomScale = random.uniform(0.08, 1.0)
CPtransform = transforms.RandomResizedCrop((self.height, self.width), scale=(randomScale, randomScale), ratio=(1,1), interpolation=2)
toImage = T.ToPILImage()
padImage= CPtransform(toImage(image).convert("L"))
padMask = CPtransform(toImage(mask).convert("L"))
return TF.to_tensor(padImage), TF.to_tensor(padMask)
But the mask are not correspond to its image after augmentation as the graph shows. The function that I used on them are all the same but the result are different.
Can anybody help? Thanks!
You can concat image and mask and convert it to a single tensor and do the transformation.
image = T.PILToTensor()(pil_image)
mask = T.PILToTensor()(pil_mask)
# concatenate the images and apply transform:
both_images = torch.cat((image, mask),0)
# Apply the transformations to both images simultaneously
transformed_images = CPtransform(both_images)
#get transformed image and mask separately
image_trans = transformed_images[:image.shape[0]]
mask_trans = transformed_images[image.shape[0]:]
I am making a deep multimodal autoencoder model which takes two inputs and produces a two outputs (which are the reconstructed inputs). The two inputs are with shape of (1000, 50) and (1000,60) respectively and the model has 3 hidden layers and aim to concatenate the two latent layer of input1 and input2.
I would like to extract the encoded part of my model and save the data as a numpy array.
here is the complete code of the model :
input_X = Input(shape=(X[0].shape))
dense_X = Dense(40,activation='relu')(input_X)
dense1_X = Dense(20,activation='relu')(dense_X)
latent_X= Dense(2,activation='relu')(dense1_X)
input_X1 = Input(shape=(X1[0].shape))
dense_X1 = Dense(40,activation='relu')(input_X1)
dense1_X1 = Dense(20,activation='relu')(dense_X1)
latent_X1= Dense(2,activation='relu')(dense1_X1)
Concat_X_X1 = concatenate([latent_X, latent_X1])
decoding_X = Dense(20,activation='relu')(Concat_X_X1)
decoding1_X = Dense(40,activation='relu')(decoding_X)
output_X = Dense(X[0].shape[0],activation='sigmoid')(decoding1_X)
decoding_X1 = Dense(20,activation='relu')(Concat_X_X1)
decoding1_X1 = Dense(40,activation='relu')(decoding_X1)
output_X1 = Dense(X1[0].shape[0],activation='sigmoid')(decoding1_X1)
multi_modal_autoencoder = Model([input_X, input_X1], [output_X, output_X1], name='multi_modal_autoencoder')
encoder = Model([input_X, input_X1], Concat_X_X1)
encoder.save('encoder.h5')
multi_modal_autoencoder.compile(optimizer=keras.optimizers.Adam(lr=0.001),loss='mse')
model = multi_modal_autoencoder.fit([X,X1], [X, X1], epochs=70, batch_size=150)
With h5py package you can get into your .h5 file and extract exactly what you want:
f = h5py.File('encoder.h5', 'r')
keys = list(f.keys())
values = f.get('some_key')
You can hierarchically use .get many times to go deeper into your .h5 file to extract what you need.
I am trying to follow this blog https://brunolopezgarcia.github.io/2018/05/09/Crafting-adversarial-faces.html to generate adversarial face images against Facenet. The code is here https://github.com/tensorflow/cleverhans/tree/master/examples/facenet_adversarial_faces and works fine! My question is how can I export these adversarial images. Is this question too straightforward, so the blog didn't mention it, but only shows some sample pictures.
I was thinking it is not a hard problem, since I know the generated adversarial samples are in the "adv". But this adv (float32) came from faces1, after being prewhiten and normalized. To restore the int8 images from adv(float32), I have to reverse the normalization and prewhiten process. It seems like if we want output some images from facenet, we have to do this process.
I am new to Facenet and Cleverhans, I am not sure whether this is the best way to do that, or is that common way(such as functions) for people to export images from Facenet.
In facenet_fgsm.py, we finally got the adversarial samples. I need to export adv to plain int images.
adv = sess.run(adv_x, feed_dict=feed_dict)
In set_loader.py. There are some kinda of normalization.
def load_testset(size):
# Load images paths and labels
pairs = lfw.read_pairs(pairs_path)
paths, labels = lfw.get_paths(testset_path, pairs, file_extension)
# Random choice
permutation = np.random.choice(len(labels), size, replace=False)
paths_batch_1 = []
paths_batch_2 = []
for index in permutation:
paths_batch_1.append(paths[index * 2])
paths_batch_2.append(paths[index * 2 + 1])
labels = np.asarray(labels)[permutation]
paths_batch_1 = np.asarray(paths_batch_1)
paths_batch_2 = np.asarray(paths_batch_2)
# Load images
faces1 = facenet.load_data(paths_batch_1, False, False, image_size)
faces2 = facenet.load_data(paths_batch_2, False, False, image_size)
# Change pixel values to 0 to 1 values
min_pixel = min(np.min(faces1), np.min(faces2))
max_pixel = max(np.max(faces1), np.max(faces2))
faces1 = (faces1 - min_pixel) / (max_pixel - min_pixel)
faces2 = (faces2 - min_pixel) / (max_pixel - min_pixel)
In the facenet.py load_data function, there is a prewhiten process.
nrof_samples = len(image_paths)
images = np.zeros((nrof_samples, image_size, image_size, 3))
for i in range(nrof_samples):
img = misc.imread(image_paths[i])
if img.ndim == 2:
img = to_rgb(img)
if do_prewhiten:
img = prewhiten(img)
img = crop(img, do_random_crop, image_size)
img = flip(img, do_random_flip)
images[i,:,:,:] = img
return images
I hope some expert can point me some hidden function in facenet or cleverhans that can directly export the adv images, otherwise reversing normalization and prewhiten process seems akward. Thank you very much.
I don't know much about the Facenet code. From your discussion, it seems like you will have to save the values of min_pixel,max_pixelto reverse the normalization, and then look at theprewhiten` function to see how you can reverse it. I'll email Bruno to see if he has any further comments to help you out.
EDIT: Now image exporting is included in the Facenet example of Cleverhans: https://github.com/tensorflow/cleverhans/commit/08f6fb9cf2a7f199467d5ed60179fc3ae9140458
I'm trying to set up TensorFlow to accept one image at a time but I believe I'm getting incorrect results because I pass a regular array without first performing tf.image.per_image_whitening() beforehand. Is there an easy way to do this in Python to an individual image without using the image queue?
Here's my code so far:
im = Image.open(request.FILES.values()[0])
im = im.convert('RGB')
im = im.crop((0, 0, cifar10.IMAGE_SIZE, cifar10.IMAGE_SIZE))
(width, height) = im.size
image_array = list(im.getdata())
image_array = np.array(image_array)
image_array = image_array.reshape((1, height, width, 3))
# tf.image.per_image_whitening() should be done here
#mean = numpy.mean(image_array)
#stddev = numpy.std(image_array)
#adjusted_stddev = max(stddev, 1.0/len(image_array.flatten())))
feed_dict = {"shuffle_batch:0": image_array}
# predictions always returns something close to [1, 0]
predictions = sess.run(tf.nn.softmax(logits), feed_dict=feed_dict)
If you want to avoid the image queue and do the predictions one by one, I think
image_array = (image_array - mean) / adjusted_stddev
should be able to do the trick.
If you want to do the prediction by batches, it's a little bit complicated as per_image_whitening (now per_image_standardization) only works with single images. So you need to do it before you form the batch like the way above or setup a preprocess procedure.
This question is based on: Tensorflow image reading & display
Following their code we have the following:
string = ['/home/user/test.jpg']
filepath_queue = tf.train.string_input_producer(string)
self.reader = tf.WholeFileReader()
key, value = self.reader.read(filepath_queue)
print(value)
# Output: Tensor("ReaderRead:1", shape=TensorShape([]), dtype=string)
my_img = tf.image.decode_jpeg(value, channels=3)
print(my_img)
# Output: Tensor("DecodeJpeg:0", shape=TensorShape([Dimension(None), Dimension(None), Dimension(3)]), dtype=uint8)
Why does my_img have no dimensions? (Dimension(3) is only because of the argument channels=3)
Does this mean that the image is not properly loaded? (img = misc.imread('/home/user/test.jpg') does load that image).
The image will be properly loaded, but TensorFlow doesn't have enough information to infer the image's shape until the op is run. This arises because tf.image.decode_jpeg() can produce tensors of different shapes (heights and widths), depending on the contents of the string tensor value. This enables you to build input pipelines using a collection of images with different sizes.
The Dimension(None) in the shape means "unknown" rather than "empty".
If you happen to know that all images read by this operation will have the same size, you can use Tensor.set_shape() to provide this information, and doing so will help to validate the shapes of later parts of the graph:
my_img = tf.image.decode_jpeg(value, channels=3)
KNOWN_HEIGHT = 28
KNOWN_WIDTH = 28
my_img.set_shape([KNOWN_HEIGHT, KNOWN_WIDTH, 3])
print(my_img)
# Output: Tensor("DecodeJpeg:0", shape=TensorShape([Dimension(28), Dimension(28), Dimension(3)]), dtype=uint8)