I'm using skimage.segmentation.felzenszwalb as shown here, and it's great.
I've tuned the parameters as best as I can, but it's still detecting some large segments I don't want it to. How can I remove segments above a certain size afterwards? I know you can use skimage.measure.regionprops to get the area of each segment, but i'm not sure how to then remove a segment from segments_fz, leaving the array in tact for mark_boundaries.
import numpy as np
import skimage.io
from skimage.util import img_as_float
from skimage.segmentation import felzenszwalb
from skimage.segmentation import mark_boundaries
img = img_as_float(astronaut()[::2, ::2])
segments_fz = felzenszwalb(img, scale=100, sigma=0.5, min_size=50)
print("Felzenszwalb number of segments: {}".format(len(np.unique(segments_fz))))
segmented_img = mark_boundaries(img, segments_fz)
skimage.io.imsave('img_labeled.png', segmented_img)
This is similar to this post, but they are trying to remove small objects, instead of large ones.
Have a look at the source code for skimage.morphology.remove_small_objects. There's a lot of pre-processing / input cleaning, but the business end is quite simple. Assuming segments is your input image with segment labels, here's what the function is doing:
out = np.copy(segments)
component_sizes = np.bincount(segments.ravel())
too_small = component_sizes < min_size
too_small_mask = too_small[segments]
out[too_small_mask] = 0
return out
You can change too_small to too_big, or really any condition you want! The key is to produce an array at that line that contains True at position i whenever you want to zero out label i, and False everywhere else.
Related
I'm trying to use Tensorflow to Machine Learning to analyze an image and return the probability if is positive or negative based on a model created (extension .h5). I couldn't found a documentation exactly for that, or repository, so even a link to read will be awesome.
Link for the application: https://share.streamlit.io/felipelx/hackathon/IDC_Detector.py
Libraries that I'm trying to use.
import numpy as np
import streamlit as st
import tensorflow as tf
from keras.models import load_model
The function to load the model.
#st.cache(allow_output_mutation=True)
def loadIDCModel():
model_idc = load_model('models/IDC_model.h5', compile=False)
model_idc.summary()
return model_idc
The function to work the image, and what I'm trying to see: model.predict - I can see but is not updating the %, independent of the image the value is always the same.
if uploaded_file is not None:
# transform image to numpy array
file_bytes = tf.keras.preprocessing.image.load_img(uploaded_file, target_size=(96,96), grayscale = False, interpolation = 'nearest', color_mode = 'rgb', keep_aspect_ratio = False)
c.image(file_bytes, channels="RGB")
Genrate_pred = st.button("Generate Prediction")
if Genrate_pred:
model = loadMetModel()
input_arr = tf.keras.preprocessing.image.img_to_array(file_bytes)
input_arr = np.array([input_arr])
probability_model = tf.keras.Sequential([model, tf.keras.layers.Softmax()])
prediction = probability_model.predict(input_arr)
dict_pred = {0: 'Benigno/Normal', 1: 'Maligno'}
result = dict_pred[np.argmax(prediction)]
value = 0
if result == 'Benigno/Normal':
value = str(((prediction[0][0])*100).round(2)) + '%'
else:
value = str(((prediction[0][1])*100).round(2)) + '%'
c.metric('Predição', result, delta=value, delta_color='normal')
Thank you in advance to any help.
The first thing I'm noticing is that your function for loading the model is named loadIDCModel, but then the function you call for loading the model is loadMetModel. When I check your source code, though, it looks like you've already addressed this issue. I'd recommend updating your question to reflect this.
Playing around with your application, I think the issue is your model itself. I tried various images — images containing carcinomas, and even a picture of a cat — and each gave me a probability around 73%. The lowest score I got was 72.74%, and the highest was 73.11% (this one was the cat). It seems that the output percentage is varying slightly, hinting that rather than something being wrong in the code, your model itself is likely at fault. You might need to retrain your model, as it seems to have learned to always return a value of approximately 0.73.
I'm looking at a global netcdf file. I want to set all land points that are within the 60-75 deg N band to zero but keep the ocean points in that band as nan. As a second step, I want to keep the values on the land points from 60-75 but set all other land points to zero. Ocean values are NaNs. I just don't get my xarray script to do that - here is what I tried
import numpy as np
import matplotlib.pyplot as plt
ds = xr.open_dataset('ifle.nc')
ds['Shrub_total'] = ds['Shrub']
shrub_total = ds.Shrub_total
tundra = shrub_total.where((shrub_total!=np.nan)&(shrub_total.Lat>60)&
(shrub_total.Lat<75), 0)
shrub = shrub_total.where((shrub_total!=np.nan)&(shrub_total.Lat<60)&
(shrub_total.Lat>75), 0)
ds['Tundra'] = tundra
ds['Shrub'] = shrub
fig, axes = plt.subplots(ncols=2,figsize=(12,3))
ds['Shrub_total'].isel(Time=0).plot(ax=axes[0])
ds['Tundra'].isel(Time=0).plot(ax=axes[1])
ds['Shrub'].isel(Time=0).plot(ax=axes[2])
plt.show()
This is what it looks like
The left panel is the original data, for the middle one at least I managed to keep the data I wanted - but instead of the two massive violet blocks I wanted to keep the map with all values outside the selected area set to zero. The right panel was intended to be the 'inverse' of the middle one but I completely failed there. It feels like this should be such an easy thing to do but I just can't figure it out!
This appeared to be mostly an issue with the logical side, as well as the method used to deal with the NaNs.
The below seems to work for me:
tundra = shrub_total.where((np.isnan(shrub_total)==True)|
((shrub_total.Lat>60)&(shrub_total.Lat<75)), 0)
shrub = shrub_total.where((np.isnan(shrub_total)==True)|
((shrub_total.Lat<60)|(shrub_total.Lat>75)), 0)
I changed the shrub logical to an OR statement (we want to mask either less than 60 or more than 75 - it's not possible for somewhere to be both!).
We use np.isnan()==True rather than ()!=np.nan. I am unsure about why we can't treat this the way you did... This necessitated further changes to the logic.
Note, I do not use python so this may be very hacky, and I'm sure someone else will have a much more elegant and knowledgeable answer but it intrigued me so I attempted it :)
I need to encode an image in 16UC1 format, but I receive the error:
cv_bridge.core.CvBridgeError:encoding specified as 16UC1, but image has incompatible type 32FC1
I tried to use skimage function img_as_uint but since my image values are not between -1 and 1 it doesn't work. i also tried to "normalize" my values by dividing all of them by the value obtained from np.amax, but using the skimage function only returns a blank image.
Is there a way of achieving this conversion?
This is the original 32FC1 image
With numpy you should be able to:
import numpy as np
img = np.random.normal(0, 1, (300, 300, 3)).astype(np.float32) # simulated image
uimg = img.astype(np.uint16)
You probably will first want to do some kind of normalization if it isn't already in an unsigned range. Probably something like:
img_normalized = (img-img.min())/(img.max()-img.min())*256**2
But your normalization strategy will depend on what you want to accomplish.
Thanks for sharing an image. I can visualize as follows:
import numpy as np
import matplotlib.pyplot as plt
arr = np.load('32FC1_image.npz')
img = arr['arr_0']
img = np.squeeze(img) # this gets rid of the extra dimensions that are causing matplotlib to not recognize it as an image, the extra dimensions also may be causing your problems
img_normalized = (img-img.min())/(img.max()-img.min())*256**2
img_normalized = img_normalized.astype(np.uint16)
plt.imshow(img_normalized)
Try using the normalized 16 bit image.
I encountered the following issue making some tests to demonstrate the usefulness of a pure pyarrow UDF in pyspark as compared to always going through pandas.
import awkward
import numpy
import pandas
import pyarrow
counts = numpy.random.randint(0,20,size=200000)
content = numpy.random.normal(size=counts.sum())
test_jagged = awkward.JaggedArray.fromcounts(counts, content)
test_arrow = awkward.toarrow(test_jagged)
def awk_arrow(col):
jagged = awkward.fromarrow(col)
jagged2 = jagged**2
return awkward.toarrow(jagged2)
def pds_arrow(col):
pds = col.to_pandas()
pds2 = pds**2
return pyarrow.Array.from_pandas(pds2)
out1 = awk_arrow(test_arrow)
out2 = pds_arrow(test_arrow)
out3 = awkward.fromarrow(out1)
out4 = awkward.fromarrow(out2)
type(out3)
type(out4)
yields
<class 'awkward.array.jagged.JaggedArray'>
<class 'awkward.array.masked.BitMaskedArray'>
and
out3 == out4
yields (at the end of the stack trace):
AttributeError: no column named 'reshape'
looking at the arrays:
print(out3);print();print(out4);
[[0.00736072240594475 0.055560612050914775 0.4094101942882973 ... 2.4428454924678533 0.07220045904440388 3.627270394986972] [0.16496227597707766 0.44899025266849046 1.314602433843517 ... 0.07384558862546337 0.5655043672418324 4.647396184088295] [0.04356259421421215 1.8983172440218923 0.10442121937532822 0.7222467989756899 0.03199694383894229 0.954281670741488] ... [0.23437909336737087 2.3050822727237272 0.10325064534860394 0.685018355096147] [0.8678765133108529 0.007214659054089928 0.3674379091794599 0.1891573101427716 2.1412651888713317 0.1461282900111415] [0.3315468986268042 2.7520115602119772 1.3905787720409803 ... 4.476255451581318 0.7237199572195625 0.8820112289563018]]
[[0.00736072240594475 0.055560612050914775 0.4094101942882973 ... 2.4428454924678533 0.07220045904440388 3.627270394986972] [0.16496227597707766 0.44899025266849046 1.314602433843517 ... 0.07384558862546337 0.5655043672418324 4.647396184088295] [0.04356259421421215 1.8983172440218923 0.10442121937532822 0.7222467989756899 0.03199694383894229 0.954281670741488] ... [0.23437909336737087 2.3050822727237272 0.10325064534860394 0.685018355096147] [0.8678765133108529 0.007214659054089928 0.3674379091794599 0.1891573101427716 2.1412651888713317 0.1461282900111415] [0.3315468986268042 2.7520115602119772 1.3905787720409803 ... 4.476255451581318 0.7237199572195625 0.8820112289563018]]
You can see the contents and shape of the arrays are the same, but they're not comparable to each other at face value, which is very counter intuitive. Is there a good reason for dense jagged structures with no Nulls to be represented as a BitMaskedArray?
All data in Arrow are nullable (at every level), and they use bit masks (as opposed to byte masks) to specify which elements are valid. The specification allows columns of entirely valid data to not write the bitmask, but not every writer takes advantage of that freedom. Quite often, you see unnecessary bitmasks.
When it encounters a bitmask, such as here, awkward inserts a BitMaskedArray.
It could be changed to check to see if the mask is unnecessary and skip that step, though that adds an operation that scales with the size of the dataset (though likely insignificant in most cases—bitmasks are 8 times faster to check than bytemasks). It's also a little complicated: the last byte may be incomplete if the length of the dataset is not a multiple of 8. One would need to check these bits individually, but the rest of the mask could be checked in bulk. (Maybe even cast as int64 to check 64 flags at a time.)
I have a ndarray representing an image with different channels like this:
image = (8,100,100) where 8=channels, 100x100 the actual image per channel
I am interested in extracting the RGB components of that image:
imageRGB = np.take(image, [4,2,1], axis = 0)
in this way I have an array of (3,100,100) with the RGB components.
However, I need to visualize it so I need an array of (100,100,3), I think it's quite straightforward to do it but I all the methods I try do not work.
numpy einsum is a good tool to be used.
Official document: https://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html
import numpy as np
imageRGB = np.random.randint(0,5,size=(3,100,101))
# set the last dim to 101 just to make stuff more clear
imageRGB.shape
# (3,100,101)
imageRGB_reshape = np.einsum('kij->ijk',imageRGB)
imageRGB_reshape.shape
# (100,101,3)
In my opinion it's the most clear way to write and read.
Wow thank you! I have never thought to use Einstein summation, actually it works very well.
Just for curiosity is it possible to build it manually?
For example:
R = image[4,:,:]
G = image[2,:,:]
B = image[1,:,:]
imageRGB = ???