I have a folder in which I have 100+ .npy files.
The path to this folder is '/content/drive/MyDrive/lung_cancer/subset0/trainImages'.
This folder has the .npy files as shown in the image the .npy files
The shape of each of these .npy files is (3,512,512)
I want to combine all of these files into one single file with the name trainImages.npy so that I can train my unet model with it.
My unet model takes input of the shape (1,512,512).
I will load the above trainImages.npy file into imgs_train as below to pass it as input into unet model
imgs_train = np.load(working_path+"trainImages.npy").astype(np.float32)
Can someone please tell me how do i concatenate all those .npy files into one single .npy file??
Thanks.
So I found the answer out by myself and I am attaching the code below if anyone needs it. Change it according to your needs..
import os
import numpy as np
path = '/content/drive/MyDrive/lung_cancer/subset0/trainImages/'
trainImages = []
for i in os.listdir(path):
data = np.load(path+i)
trainImages.append(data)
Related
I have a dataset of images which has the images in .tfrecord format and the labels in .pbtxt. I can read the first as:
filenames = [filename]
raw_dataset = tf.data.TFRecordDataset(filenames)
raw_dataset
But once I have this dataset and the labels in a .pbtxt file, how can I add the labels to the dataset or use both for training the model?
Thanks
I too have faced the same problem and found something really helpful to tackle the tasks. Here is the link of solution for the issue:
Link
I have successfully run Keras on BMP RGB files but now I need to increase the channels in my data so I've switched to NPY files.
In the process I discovered that the ImageDataGenerator only works with image files...
So I've decided to assemble my train-test data and train-test labels in a mnist-type file because there are many keras scripts out there that read directly from mnist.npz
But I don't understand how to get from my directories of data to a mnist.npz file...??!?
My data is organized as follows:
a train directory full of npy files
a test directory full of npy files
a txt file with labels (one hot encoding) for the train npy files
a txt file with labels (one hot encoding) for the test npy files
Each line in the label files looks this: aaa.npy 100000000000000000000
If you have any suggestions, you're welcome.
Cheers!
I am a newbie on the TensorFlow object detection library. I have a specific data set what I have to produce myself and labeled it with thousands of jpg. I have run the file to detect the object from these images.. any way.The end of the process i have gotten frozen_graph and from it I exported model.ckpl file to inference graph folder everything goes fine, and I have tested model.ckpl model on the object_detection.ipynb file it works fine. Until this step, there is no problem.
However,Am not able to understand how could convert that model.ckpl file to model.tflite file to use on android studio app.
I have see many things like but I am no idea what is the input_tensors = [...]
output_tensors = [...]
I may already know but what it was actually...
Could you show me how could I convert it?
Use tensorboard to find out your input and output layer. For reference follow these links -
https://heartbeat.fritz.ai/intro-to-machine-learning-on-android-how-to-convert-a-custom-model-to-tensorflow-lite-e07d2d9d50e3
Tensorflow Convert pb file to TFLITE using python
If you don't know your inputs and outputs, use summarize_graph tool and feed it your frozen model.
See command here
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms#inspecting-graphs
If you have trained your model from scratch you must be having the .meta file. Also you need to specify the output node names using which you can create a .pb file. Please refer to below link on steps to create this file:
Tensorflow: How to convert .meta, .data and .index model files into one graph.pb file
Once this is created you can further convert your .pb to tflite as below:
import tensorflow as tf
graph_def_file = "model.pb"
input_arrays = ["model_inputs"]
output_arrays = ["model_outputs"]
converter = tf.lite.TFLiteConverter.from_frozen_graph(
graph_def_file, input_arrays, output_arrays)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
I have a list named t_list = [a.tfrecords, b.tfrecords, c.tfrecords, d.tfrecords], which contains 4 TFRecords paths named a.tfrecords, b.tfrecords, c.tfrecords and d.tfrecords.
I'm reading these TFRecords with dataset = tf.data.TFRecordDataset(t_list). And I set a epoch_num by dataset = dataset.repeat(epoch_num)
I have two questions about this function:
How can I know which TFRecords file is been reading now during training?
How can I know which epoch is now during training?
Thanks!
I have an image data set of size 600 x 400 each and I have converted each of the images to TFRecord format. But I am unable to figure out how to use this data? I have seen the imagenet dataset and found only one single binary file (when extracted it form here).
Is it that for an image dataset there will be only one TFRecord or each individual images will have their own TFRecord files?
Tensorflow doesnt look for single tfrecord file. So feel free and point your "data directory" and "train directory" to the location which has set of tfrecord files.
Also, keep in mind files should be in respective directories based on their names like TRAIN-*.tfrecord files in "train directory".
Answer can be more specific if you mentioned what model of TF you are targeting to run on these TF record files.
Hope it helps.