I recreated a deep learning network (Yolov3) and extracted a feature map after the prediction. This has the following dimensions (1, 13, 13, 3, 50). The dimensions 13x13 stand for the grid and the 3 for the RGB values. The 50 stand for the 50 different classes my model can predict.
Currently I am trying to reformat the feature maps for each class individually. That means, I try to create 50 arrays from the structure described above, which contain 3 arrays (each for RGB features) and should each contain the grid of 13x13.
What you have to consider is that the feature map contains the values of the 50 classes for each cell of the 13x13 grid.
Currently I have solved the problem with a for-loop that can only extract one class. So I have to ask myself if I can use Numpy for example with resize, transpose, reshape to set a better previous one.
def extract_feature_maps(model_output, class_index):
for row in model_output:
feature_maps= [[], [], []]
for column in row:
tmp = [[], [], []]
for three_dim in column:
counter = 0
for feature_map_tmp in three_dim:
feature_map_tmp_0 = feature_map_tmp[5:]
feature_number = feature_map_tmp_0[class_index]
tmp[counter].append(feature_number)
counter += 1
feature_maps[0].append(tmp[0])
feature_maps[1].append(tmp[1])
feature_maps[2].append(tmp[2])
return np.array(feature_maps[0]), np.array(feature_maps[1]), np.array(feature_maps[2])
As I said, I can currently only extract the feature map from one class in a very time consuming way. Is there a way to do this more clever?
Related
When using GridSearchCV() to perform a k-fold cross validation analysis on some data is there a way to know which data was used for each split?
For example, assumed the goal is to build a binary classifier of your choosing, named 'model'. There are 100 data points (rows) with 5 features each and an associated 1 or 0 target. 20 of the 100 data points are held out for testing after training and hyperparameter tuning, GridSearchCV will never see those 20 data points. The other 80 data rows are put into the estimator as X and Y, so GridSearchCV will only see 80 rows of data. Various hyper parameters are tuned and laid out in the param_grid variable. For this case the cross validation parameter of cv is assigned a value of 3, as shown:
grid = GridSearchCV(estimator=model, param_grid=param_grid, cv=3) grid_result = grid.fit(X, Y)
Is there a way to see which data was used as the training data and as the cross validation data for each fold? Maybe seeing which indices were used for the split?
I am looking to train a model with a cycle loss (similar to CycleGAN) on a different x/y paired dataset in each epoch. The aim is that, across many epochs, the model would be trained on many if not all of the admissible pairings of the elements of x with y.
E.g., suppose 2 tf.data datasets: x_tf_data and y_tf_data. Each element of x_tf_data can be paired with 1 or more elements of y_tf_data. E.g., the first element of x_tf_data can be paired with the first 10 elements of y_tf_data. This is given by a list of vectors denoted list_vectors such that list_vectors[0] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] and list_vectors[i-1] are the y_tf_data elements that can be paired with the i'th element of x_tf_data.
In each epoch, the x/y pair presented to the model should be (potentially) different. E.g., in each epoch, the first element of x_tf_data can be paired with any of the first 10 elements of y_tf_data. This can be achieved by randomly selecting 1 element of list_vectors[i], for all i, in each epoch.
What may be a scalable solution?
After a lot of experimentation, what worked best was to create set of N tf.data Datasets in which each element of x was paired with a randomly chosen element of y, and then to sequentially concatenate the set of N Datasets to form one humungous Dataset. This Dataset was then saved to file and read into Keras. This achieved two goals. It helped the model converge more quickly because all the data did not change each epoch and it helped to ensure that a sufficient number of pairings were used for each element of x so as to get robust results.
I've got a TFRecordDataset and I'm trying to preprocess the features of two subsequent elements by means of the map() API.
dataset_ext = dataset.map(lambda x: tf.py_function(parse_data, [x], [tf.float32]))
As map applies the function parse_data to every dataset element, I don't know what parse_data should look like in order to keep track of the feature extracted from the previous dataset element.
Can anyone help? Thank you
EDIT: I'm working on the Waymo dataset, so each element is a frame. You can refer to https://github.com/Jossome/Waymo-open-dataset-document for its structure.
This is my parse function parse_data:
from waymo_open_dataset import dataset_pb2 as open_dataset
def parse_data(input_data):
frame = open_dataset.Frame()
frame.ParseFromString(bytearray(input_data.numpy()))
av_speed = (frame.images[0].velocity.v_x, frame.images[0].velocity.v_y, frame.images[0].velocity.v_z)
return av_speed
I'd like to build a dataset whose features are the car speed and acceleration, defined as the speed variation between subsequent frames (the first value can be 0).
One way I thought about is to give the map function dataset and dataset.skip(1) as inputs but I'm not sure about it yet.
I am not sure but it might be unnecessary to make your mapped function a tf.py_function. How parse_data is supposed to look like depends on your dataset dataset_ext. If it has for example two file paths (1 instace of input data and 1 instance of output data), the mapping function should have 2 arguments and should return 2 arguments.
For example: if your dataset contains images and you want them to be randomly cropped each time an example of your dataset is drawn the mapping function looks like this:
def process_img_random_crop(img_in, img_out, output_shape):
merged = tf.stack([img_in, img_out])
mergedCrop = tf.image.random_crop(merged, size=(2,) + output_shape)
img_in_cropped, img_out_cropped = tf.unstack(mergedCrop, 2, 0)
return img_in_cropped, img_out_cropped
I call it as follows:
image_ds_test = image_ds_test.map(lambda i, o: process_img_random_crop(i, o, output_shape=(64, 64, 1)), num_parallel_calls=tf.data.experimental.AUTOTUNE)
What exactly is your plan with dataset_ext and what does it contain?
Edit:
Okay, got what you meant with you the two frames. So the map function is applied to each entry of your dataset separatly. If you need cross-entry information, a single entry of your dataset needs to contain two frames. With this more complicated set-up, I would suggest you to use a tensorflow Sequence: The explanation from the tensorflow team is pretty straigth forward. Hope this help!
I have some pretty large arrays to deal with. By describing them big, I mean like in the scale of (514, 514, 374). I want to randomly get an index base on its pixel value. For example, I need the 3-d index of a pixl with value equal to 1. So, I list all the possibilities by
indices = np.asarray(np.where(img_arr == 1)).T
This works perfect, except that it runs very slow, to an intolerable extent, since the array is so big. So my question is is there a better way to do that? It would be nicer if I can input a list of pixel values, and I get back a list of corresponding indices. For example, I want to sample the indices of these pixel values [0, 1, 2], and I get back list of indices [[1,2,3], [53, 215, 11], [223, 42, 113]]
Since I am working with medical images, solutions with SimpleITK is also welcomed. So feel free to leave your opinions, thanks.
import numpy as np
value = 1
# value_list = [1, 3, 5] you can also use a list of values -> *
n_samples = 3
n_subset = 500
# Create a example array
img_arr = np.random.randint(low=0, high=5, size=(10, 30, 20))
# Choose randomly indices for the array
idx_subset = np.array([np.random.randint(high=s, size=n_subset) for s in x.shape]).T
# Get the values at the sampled positions
values_subset = img_arr[[idx_subset[:, i] for i in range(img_arr.ndim)]]
# Check which values match
idx_subset_matching_temp = np.where(values_subset == value)[0]
# idx_subset_matching_temp = np.argwhere(np.isin(values_subset, value_list)).ravel() -> *
# Get all the indices of the subset with the correct value(s)
idx_subset_matching = idx_subset[idx_subset_matching_temp, :]
# Shuffle the array of indices
np.random.shuffle(idx_subset_matching)
# Only keep as much as you need
idx_subset_matching = idx_subset_matching[:n_samples, :]
This gives you the desired samples. The distribution of those samples should be the same as if you are using your method of looking at all matches in the array. In both cases you get a uniform distribution along all the positions with matching values.
You have to be careful when choosing the size of the subset and the number of samples you want. The subset must be large enough that there are enough matches for the values, otherwise it won't work.
A similar problem occurs if the values you want to sample are very sparse, then the size of the subset needs to be very large (in the edge case the whole array) and you gain nothing.
If you are sampling often from the same array maybe it is also a good idea to store the indices for each value
indices_i = np.asarray(np.where(img_arr == i)).T
and use those for the your further computations.
i have an error at this line:neigh.fit(X, y) :
ValueError: setting an array element with a sequence.
I checked fit function and X is: {array-like, sparse matrix, BallTree, cKDTree}
My X is a list of list with first element solidity number and second elemnt humoment list (7 cells).
If i change and i take only first humoment number for having a pure list of list
give this error: query data dimension must match BallTree data dimension.
My code:
listafeaturevector = list()
path = 'imgknn/'
for infile in glob.glob( os.path.join(path, '*.jpg') ):
print("current file is: " + infile )
gray = cv2.imread(infile,0)
element = cv2.getStructuringElement(cv2.MORPH_CROSS,(6,6))
graydilate = cv2.erode(gray, element)
ret,thresh = cv2.threshold(graydilate,127,255,cv2.THRESH_BINARY_INV)
imgbnbin = thresh
#CONTOURS
contours, hierarchy = cv2.findContours(imgbnbin, cv2.RETR_TREE ,cv2.CHAIN_APPROX_SIMPLE)
print(len(contours))
for i in range (0, len(contours)):
fv = list() #1 feature vector
#HUMOMENTS
#print("humoments")
mom = cv2.moments(contours[i], 1)
Humoments = cv2.HuMoments(mom)
#print(Humoments)
fv.append(Humoments) #query data dimension must match BallTree data dimension
#SOLIDITY
area = cv2.contourArea(contours[i])
hull = cv2.convexHull(contours[i]) #ha tanti valori
hull_area = cv2.contourArea(hull)
solidity = float(area)/hull_area
fv.append(solidity)
#fv.append(elongation)
listafeaturevector.append(fv)
print("i have done")
print(len(listafeaturevector))
lenmatrice=len(listafeaturevector)
#KNN
X = listafeaturevector
y = [0,1,2,3]* (lenmatrice/4)
from sklearn.neighbors import KNeighborsClassifier
neigh = KNeighborsClassifier(n_neighbors=3)
neigh.fit(X, y) #ValueError: setting an array element with a sequence.
print(neigh.predict([[1.1]]))
print(neigh.predict_proba([[0.9]]))
If i try to covert it in a numpy array:
listafv = np.dstack(listafeaturevector)
listafv=np.rollaxis(listafv,-1)
print(listafv.shape)
data = listafv.reshape((lenmatrice, -1))
print(data.shape)
#KNN
X = data
i got: setting an array element with a sequence
A couple of suggestions/questions:
Humoments = cv2.HuMoments(mom)
What is the class of the return value Humoments? a float or a list? If float, that is fine.
for each image file
for i in range (0, len(contours)):
fv = list() #1 feature vector
...
fv.append(Humoments)
...
fv.append(solidity)
listafeaturevector.append(fv)
The above code does not seem correct. In your problem, I think you need to a construct a feature vector for each image. So anything that is related to image i should go to the same feature vector x_i. Then you combine all feature vectors to get a list of feature vectors X. However, your listafeaturevector (or X) presents in the inner-most loop, it's obviously not correct.
Second, you have a loop against the number of elements in the contours, are you sure the number of elements stays the same for each image? Otherwise, the number of features (|x_i|) is totally different across different images, that might cause the error of
setting an array element with a sequence.
Third, are you clear about how you want to classify the images? what are the target values/labels of different images? I see you just setting labels with [0,1,2,3]* (lenmatrice/4). Can you elaborate on what you are trying to do with those images? Are they containing different type of object? Are they showing different patterns? Are those images describe different topic/color? If yes, for each different type, you give a different label - either 0,1,2 or 'red','white','black' (assume you have only 3 types). The values of the label do not matter. What matters is how many values they have. I am trying to understand the difference of labels in your case.
On the other hand, if you only want to retrieve similar images, you don't need to use a classifier or specify a label for each image. Instead, try to use NearestNeighbors.
print(neigh.predict([[1.1]]))
print(neigh.predict_proba([[0.9]]))
Fourth, the above two lines of test are not correct. You need to set an X-like object in order to get a prediction from the classifier. That is to say, you need a feature vector x with the identical structure as you constructed in your training examples (with all h,e,s in the same order).