Detecting the Image very very slow in device - objective-c

I am using this SURF code to detect the logo in my image. It is working fine but it is very slow. Any idea about how can I optimize it?
- (void)findObject
{
//NSLog(#"%# %#", self, NSStringFromSelector(_cmd));
width = 0;
CvMemStorage* storage = cvCreateMemStorage(0);
static CvScalar colors[] =
{
{{0,0,255}},
{{0,128,255}},
{{0,255,255}},
{{0,255,0}},
{{255,128,0}},
{{255,255,0}},
{{255,0,0}},
{{255,0,255}},
{{255,255,255}}
};
if( !objectToFind || !image )
{
NSLog(#"Missing object or image");
return;
}
CvSize objSize = cvGetSize(objectToFind);
IplImage* object_color = cvCreateImage(objSize, 8, 3);
cvCvtColor( objectToFind, object_color, CV_GRAY2BGR );
CvSeq *objectKeypoints = 0, *objectDescriptors = 0;
CvSeq *imageKeypoints = 0, *imageDescriptors = 0;
int i;
CvSURFParams params = cvSURFParams(500, 1);
double tt = (double)cvGetTickCount();
NSLog(#"Finding object descriptors");
cvExtractSURF( objectToFind, 0, &objectKeypoints, &objectDescriptors, storage, params );
NSLog(#"Object Descriptors: %d", objectDescriptors->total);
cvExtractSURF( image, 0, &imageKeypoints, &imageDescriptors, storage, params );
NSLog(#"Image Descriptors: %d", imageDescriptors->total);
tt = (double)cvGetTickCount() - tt;
NSLog(#"Extraction time = %gms", tt/(cvGetTickFrequency()*1000.));
CvPoint src_corners[4] = {{0,0}, {objectToFind->width,0}, {objectToFind->width, objectToFind->height}, {0, objectToFind->height}};
CvPoint dst_corners[4];
CvSize size = cvSize(image->width > objectToFind->width ? image->width : objectToFind->width,
objectToFind->height+image->height);
output = cvCreateImage(size, 8, 1 );
cvSetImageROI( output, cvRect( 0, 0, objectToFind->width, objectToFind->height ) );
//cvCopy( objectToFind, output );
cvResetImageROI( output );
cvSetImageROI( output, cvRect( 0, objectToFind->height, output->width, output->height ) );
cvCopy( image, output );
cvResetImageROI( output );
NSLog(#"Locating Planar Object");
#ifdef USE_FLANN
NSLog(#"Using approximate nearest neighbor search");
#endif
if( locatePlanarObject( objectKeypoints, objectDescriptors, imageKeypoints,
imageDescriptors, src_corners, dst_corners ))
{
for( i = 0; i < 4; i++ )
{
CvPoint r1 = dst_corners[i%4];
CvPoint r2 = dst_corners[(i+1)%4];
//cvLine( output, cvPoint(r1.x, r1.y+objectToFind->height ),
//cvPoint(r2.x, r2.y+objectToFind->height ), colors[6] );
cvLine( output, cvPoint(r1.x, r1.y+objectToFind->height ),
cvPoint(r2.x, r2.y+objectToFind->height ), colors[6],4 );
//if(i==0)
width = sqrt(((r1.x-r2.x)*(r1.x-r2.x))+((r1.y-r2.y)*(r1.y-r2.y)));
}
}
vector<int> ptpairs;
NSLog(#"finding Pairs");
#ifdef USE_FLANN
flannFindPairs( objectKeypoints, objectDescriptors, imageKeypoints, imageDescriptors, ptpairs );
#else
findPairs( objectKeypoints, objectDescriptors, imageKeypoints, imageDescriptors, ptpairs );
#endif
/* for( i = 0; i < (int)ptpairs.size(); i += 2 )
{
CvSURFPoint* r1 = (CvSURFPoint*)cvGetSeqElem( objectKeypoints, ptpairs[i] );
CvSURFPoint* r2 = (CvSURFPoint*)cvGetSeqElem( imageKeypoints, ptpairs[i+1] );
cvLine( output, cvPointFrom32f(r1->pt),
cvPoint(cvRound(r2->pt.x), cvRound(r2->pt.y+objectToFind->height)), colors[8] );
}*/
float dist = 629.0/width;
[distanceLabel setText:[NSString stringWithFormat:#"%.2f",dist]];
NSLog(#"Converting Output");
UIImage *convertedOutput = [OpenCVUtilities UIImageFromGRAYIplImage:output];
NSLog(#"Opening Stuff");
[imageView setImage:convertedOutput];
cvReleaseImage(&object_color);
[activityView stopAnimating];
}
In the above code image is my original image and objectToFind is the logo which I want to detect.
Please let me know if my question is not clear.

You need to use profiling to decide which part of your code is the slowest.
Since you are using XCode, you have a built-in profiler at hands reach:
in he top-left corner you press-and-hold "Run" button and choose "Profile".
click on Profile and and select Time Profiler.
after a while you press "stop" in the profiler, and select "Hide Missing Symbols", "Hide System Libraries" and "Top Functions", deselect "Separate by Thread".
Now look up function main and there is a hidden right arrow after it. Click on that arrow and you can see time in percent and call statistics by the calltree.
This is how you start.
In general I have the following suggestions without profiling:
Avoid creating new images and memory storages as much as you can. (You can pass images for temporary use to your function, and preserve those images outside so that you can reuse them later.)
scale down your image (and your logo) to role-out major parts of the image
use less descriptors
The two rules of thumb:
you need to decide what to improve after the profiling, as profiling often yields surprising results.
The quicker part you try to improve the less potential gain you have.

Related

Directx11 heightmap texture real-time modification problem

I'm making a terrain tool.
I made a 2D texture and am using it as a height map.
I want to change a specific part of the heightmap, but I'm having a problem.
I changed certain small parts, but the whole landscape of the texture is changed.
I would like to know the cause of this problem and how to solve it
thank you.
`HeightMap ShaderResourceView Create Code
void TerrainRenderer::BuildHeightmapSRV(ID3D11Device* device)
{
ReleaseCOM(mHeightMapSRV);
ReleaseCOM(m_hmapTex);
D3D11_TEXTURE2D_DESC texDesc;
texDesc.Width = m_terrainData.HeightmapWidth; //basic value 2049
texDesc.Height = m_terrainData.HeightmapHeight; //basic value 2049
texDesc.MipLevels = 1;
texDesc.ArraySize = 1;
texDesc.Format = DXGI_FORMAT_R16_FLOAT;
texDesc.SampleDesc.Count = 1;
texDesc.SampleDesc.Quality = 0;
texDesc.Usage = D3D11_USAGE_DYNAMIC;
texDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE;
texDesc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
texDesc.MiscFlags = 0;
// HALF is defined in xnamath.h, for storing 16-bit float.
std::vector<HALF> hmap(mHeightmap.size());
//current mHeightmap is all zero.
std::transform(mHeightmap.begin(), mHeightmap.end(), hmap.begin(), XMConvertFloatToHalf);
D3D11_SUBRESOURCE_DATA data;
data.pSysMem = &hmap[0];
data.SysMemPitch = m_terrainData.HeightmapWidth * sizeof(HALF);
data.SysMemSlicePitch = 0;
HR(device->CreateTexture2D(&texDesc, &data, &m_hmapTex));
D3D11_SHADER_RESOURCE_VIEW_DESC srvDesc;
srvDesc.Format = texDesc.Format;
srvDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
srvDesc.Texture2D.MostDetailedMip = 0;
srvDesc.Texture2D.MipLevels = -1;
HR(device->CreateShaderResourceView(m_hmapTex, &srvDesc, &mHeightMapSRV));
}
`HeightMap Texture modifying code
D3D11_MAPPED_SUBRESOURCE mappedData;
//m_hmapTex is ID3D11Texture2D*
HR(m_texMgr.m_context->Map(m_hmapTex, D3D11CalcSubresource(0, 0, 1), D3D11_MAP_WRITE_DISCARD, 0, &mappedData));
HALF* heightMapData = reinterpret_cast<HALF*>(mappedData.pData);
D3D11_TEXTURE2D_DESC heightmapDesc;
m_hmapTex->GetDesc(&heightmapDesc);
UINT width = heightmapDesc.Width;
for (int row = 0; row < width/4; ++row)
{
for (int col = 0; col < width/4; ++col)
{
idx = (row * width) + col;
heightMapData[idx] = static_cast<HALF>(XMConvertFloatToHalf(200));
}
}
m_texMgr.m_context->Unmap(m_hmapTex, D3D11CalcSubresource(0,0,1));
Please refer to the picture below
The lower right area renders the HeightMap texture.
I wanted to edit only 1/4 width and height, but that's all changed.
enter image description here
When the completed heightmap is applied, it works normally.
enter image description here
A texture does not always have the same width and height in memory as the definition suggests. Some textures strides (lines) are oversized. You have to use the Stride Size * Row to calculate the offset to write into.

Vulkan depth image binding error

Hi I am trying to bind depth memory buffer but I get an error saying as below. I have no idea why this error is popping up.
The depth format is VK_FORMAT_D16_UNORM and the usage is VK_IMAGE_USAGE_DEPTH_STENCIL_ATTACHMENT_BIT. I have read online that the TILING shouldnt be linear but then I get a different error. Thanks!!!
The code for creating and binding the image is as below.
VkImageCreateInfo imageInfo = {};
// If the depth format is undefined, use fallback as 16-byte value
if (Depth.format == VK_FORMAT_UNDEFINED) {
Depth.format = VK_FORMAT_D16_UNORM;
}
const VkFormat depthFormat = Depth.format;
VkFormatProperties props;
vkGetPhysicalDeviceFormatProperties(*deviceObj->gpu, depthFormat, &props);
if (props.linearTilingFeatures & VK_FORMAT_FEATURE_DEPTH_STENCIL_ATTACHMENT_BIT) {
imageInfo.tiling = VK_IMAGE_TILING_LINEAR;
}
else if (props.optimalTilingFeatures & VK_FORMAT_FEATURE_DEPTH_STENCIL_ATTACHMENT_BIT) {
imageInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
}
else {
std::cout << "Unsupported Depth Format, try other Depth formats.\n";
exit(-1);
}
imageInfo.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO;
imageInfo.pNext = NULL;
imageInfo.imageType = VK_IMAGE_TYPE_2D;
imageInfo.format = depthFormat;
imageInfo.extent.width = width;
imageInfo.extent.height = height;
imageInfo.extent.depth = 1;
imageInfo.mipLevels = 1;
imageInfo.arrayLayers = 1;
imageInfo.samples = NUM_SAMPLES;
imageInfo.queueFamilyIndexCount = 0;
imageInfo.pQueueFamilyIndices = NULL;
imageInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE;
imageInfo.usage = VK_IMAGE_USAGE_DEPTH_STENCIL_ATTACHMENT_BIT;
imageInfo.flags = 0;
// User create image info and create the image objects
result = vkCreateImage(deviceObj->device, &imageInfo, NULL, &Depth.image);
assert(result == VK_SUCCESS);
// Get the image memory requirements
VkMemoryRequirements memRqrmnt;
vkGetImageMemoryRequirements(deviceObj->device, Depth.image, &memRqrmnt);
VkMemoryAllocateInfo memAlloc = {};
memAlloc.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
memAlloc.pNext = NULL;
memAlloc.allocationSize = 0;
memAlloc.memoryTypeIndex = 0;
memAlloc.allocationSize = memRqrmnt.size;
// Determine the type of memory required with the help of memory properties
pass = deviceObj->memoryTypeFromProperties(memRqrmnt.memoryTypeBits, 0, /* No requirements */ &memAlloc.memoryTypeIndex);
assert(pass);
// Allocate the memory for image objects
result = vkAllocateMemory(deviceObj->device, &memAlloc, NULL, &Depth.mem);
assert(result == VK_SUCCESS);
// Bind the allocated memeory
result = vkBindImageMemory(deviceObj->device, Depth.image, Depth.mem, 0);
assert(result == VK_SUCCESS);
Yes, linear tiling may not be supported for depth usage Images.
Consult the specification and Valid Usage section of VkImageCreateInfo. The capability is queried by vkGetPhysicalDeviceFormatProperties and vkGetPhysicalDeviceImageFormatProperties commands. Though depth formats are "opaque", so there is not much reason to use linear tiling.
This you seem to be doing in your code.
But the error informs you that you are trying to use a memory type that is not allowed for the given Image. Use vkGetImageMemoryRequirements command to query which memory types are allowed.
Possibly you have some error there (you are using 0x1 which is obviously not part of 0x84 per the message). You may want to reuse the example code in the Device Memory chapter of the specification. Provide your memoryTypeFromProperties implementation for more specific answer.
I accidentally set the typeIndex to 1 instead of i and it works now. In my defense I have been vulkan coding the whole day and my eyes are bleeding :). Thanks for the help.
bool VulkanDevice::memoryTypeFromProperties(uint32_t typeBits, VkFlags
requirementsMask, uint32_t *typeIndex)
{
// Search memtypes to find first index with those properties
for (uint32_t i = 0; i < 32; i++) {
if ((typeBits & 1) == 1) {
// Type is available, does it match user properties?
if ((memoryProperties.memoryTypes[i].propertyFlags & requirementsMask) == requirementsMask) {
*typeIndex = i;// was set to 1 :(
return true;
}
}
typeBits >>= 1;
}
// No memory types matched, return failure
return false;
}

Script command to Align Slice Horizontally by Calibration

Is there a script command that I can specify a particular slice in a LinePlotImageDisplay and do the Align Slice Horizontally by Calibration (or Uncalibrated (channels)) action?
The following scipt is a complete implantation based on example codes provided by BmyGuest. It will align all slices in a LinePlotImageDisplay horizontally either by calibration or by channel (i.e. un-calibrated).
class SliceAlignment : object {
number true, false; // boolean
image imgLPID;
imageDisplay LPID; // line plot image display
number CalculateImageToGroupTransformFactors( object self, image slice_src, image slice_ref, number &relOff, number &relScale ) {
number origin_ref, scale_ref, origin_src, scale_src;
string unit_ref, unit_src;
number calFMT = 0; // origin is expressed in calibrated unit
//
slice_src.ImageGetDimensionCalibration( 0, origin_src, scale_src, unit_src, calFMT );
slice_ref.ImageGetDimensionCalibration( 0, origin_ref, scale_ref, unit_ref, calFMT );
//
relScale = scale_src / scale_ref;
relOff = (origin_src - origin_ref) / scale_ref ;
// check if both images are calibrated in same unit
if( unit_src != unit_ref ) return false
return true;
};
void AlignNthSliceHorizontallyByChannel( object self, number slice_idx ) {
// get current reference slice index
number refSlice_idx = LPID.LinePlotImageDisplayGetSlice();
// get slice ID's (as objects)
object slice_ref = LPID.ImageDisplayGetSliceIDByIndex(refSlice_idx);
object slice_src = LPID.ImageDisplayGetSliceIDByIndex(slice_idx);
number int_offset = 0, int_scale = 1.0; // vertical (intensity) offset and scaling factors
number pos_offset = 0, pos_scale = 1.0; // horizontal (position) offset and scaling factors
LPID.LinePlotImageDisplaySetImageToGroupTransform( slice_src, slice_ref, int_offset, int_scale, pos_offset, pos_scale );
};
void AlignNthSliceHorizontallyByCalibration( object self, number slice_idx ) {
// get current reference slice index
number refSlice_idx = LPID.LinePlotImageDisplayGetSlice();
// get slice ID's (as objects)
object slice_ref = LPID.ImageDisplayGetSliceIDByIndex(refSlice_idx);
object slice_src = LPID.ImageDisplayGetSliceIDByIndex(slice_idx);
number int_offset = 0, int_scale = 1.0; // vertical (intensity) offset and scaling factors
number pos_offset, pos_scale; // horizontal (position) offset and scaling factors
number unit_check = self.CalculateImageToGroupTransformFactors( imgLPID{slice_idx}, imgLPID{refSlice_idx}, pos_offset, pos_scale );
if( unit_check == false ) {
string prompt = "slice #" + slice_idx + " [" + LPID.ImageDisplayGetSliceLabelById( LPID.ImageDisplayGetSliceIDByIndex(slice_idx) ) + "] is calibrated in different unit!";
if( !ContinueCancelDialog( prompt ) ) return
};
LPID.LinePlotImageDisplaySetImageToGroupTransform( slice_src, slice_ref, int_offset, int_scale, pos_offset, pos_scale );
return;
};
void AlignAllSlicesHorizontallyByChannel( object self ) {
number nSlices = LPID.LinePlotImageDisplayCountSlices();
for( number idx = 0; idx < nSlices; idx++ ) self.AlignNthSliceHorizontallyByChannel( idx );
return;
};
void AlignAllSlicesHorizontallyByCalibration( object self ) {
number nSlices = LPID.LinePlotImageDisplayCountSlices();
for( number idx = 0; idx < nSlices; idx++ ) self.AlignNthSliceHorizontallyByCalibration( idx );
return;
};
object init( object self, image img ) {
// check if the image display is correct type
imgLPID := img;
LPID = imgLPID.ImageGetImageDisplay(0);
if( LPID.ImageDisplayGetDisplayType() != 3 ) throw( "Please choose a valid line plot display" );
return self;
};
SliceAlignment( object self ) {
true = 1; false = 0;
result( "SliceAlignment [obj ID:" + self.ScriptObjectGetID().hex() + "] constructured\n" );
};
~SliceAlignment( object self ) {
result( "SliceAlignment [obj ID:" + self.ScriptObjectGetID().hex() + "] destructured\n\n" );
}; };
{; object objAlign = alloc(SliceAlignment);
objAlign.init( GetFrontImage() );
if( OptionDown() ) objAlign.AlignAllSlicesHorizontallyByChannel();
else objAlign.AlignAllSlicesHorizontallyByCalibration(); };
No, there is no single 'convenience' command to achieve this alignment. You will have to create the according function yourself from reading a slices calibration and setting its display-coordinate system. You might find the following (old) tutorial PDF on the FELMI homepage might be useful:
SlicesInLinePlotDisplay.pdf
The following example script might also be useful. It shows how one slice is aligned relative to another slice. (Just on the X-axis)
// All Slices in a LinePlot are grouped into a single 'group'
// Slices can be moved relative to each other by specifying their image-to-group transform,
// and the whole image (i.e. the group) can be moved with respect to the display using the group-to-display transform.
// To set the image-to-group transform of the slice specified by 'slice_id', with respect to the slice specified by 'ref_id'
// use the command:
// LinePlotImageDisplaySetImageToGroupTransform( LinePlotImageDisplay lpid, ScriptObject slice_id, ScriptObject ref_id, double off_val, double scale_val, double off_dim_0, double scale_dim_0 )
/*********************************************************/
// Create 2 LinePlots and add them into one display
// (Initially they are aligned by their calibrations)
number sc1 = 1
number of1 = -50
number sc2 = 2
number of2 = -20
image sl1 := realImage("S1",4,300)
image sl2 := realImage("S2",4,300)
sl1 = (iwidth-icol)/iwidth
sl2 = (iwidth-icol)/iwidth
sl1[0,50,1,60] = 1
sl1[0,250,1,260] = 1
sl2[0,10,1,15] = 1
sl2[0,110,1,115] = 1
// Adding Calibrations
sl1.ImageSetDimensionCalibration(0,of1,sc1,"CH",0)
sl2.ImageSetDimensionCalibration(0,of2,sc2,"CH",0)
sl1.DisplayAt(20,30)
sl2.DisplayAt(750,30)
OKDialog( "Put into one Display" )
imageDisplay disp = sl1.ImageGetImageDisplay(0)
disp.ImageDisplayAddImage( sl2, "S2") // When added like this, the slices are automatically aligned by their respective calibration!
disp.LinePlotImageDisplaySetDoAutoSurvey( 0, 0 )
object ref_id = disp.ImageDisplayGetSliceIDByIndex(0) // Slice 0
object slice_id = disp.ImageDisplayGetSliceIDByIndex(1) // Slice 1
OKDialog("Now align by channels (i.e. undo any relative sclice alignment)")
// Simply set the relative "shifts" and "scales" to 0 and 1.
disp.LinePlotImageDisplaySetImageToGroupTransform( slice_id, ref_id, 0, 1, 0, 1 )
OKDialog("Now align by chalibration ")
number relScale = sc2/sc1
number relOff = of2-of1
disp.LinePlotImageDisplaySetImageToGroupTransform( slice_id, ref_id, 0, 1, relOff, relScale )

Check if image is dark-only bottom part

I am checking if UIImage is darker or more whiter . I would like to use this method ,but only to check the third bottom part of the image ,not all of it .
I wonder how exactly to change it to check that,i am not that familiar with the pixels stuff .
BOOL isDarkImage(UIImage* inputImage){
BOOL isDark = FALSE;
CFDataRef imageData = CGDataProviderCopyData(CGImageGetDataProvider(inputImage.CGImage));
const UInt8 *pixels = CFDataGetBytePtr(imageData);
int darkPixels = 0;
long length = CFDataGetLength(imageData);
int const darkPixelThreshold = (inputImage.size.width*inputImage.size.height)*.25;
//should i change here the length ?
for(int i=0; i<length; i+=4)
{
int r = pixels[i];
int g = pixels[i+1];
int b = pixels[i+2];
//luminance calculation gives more weight to r and b for human eyes
float luminance = (0.299*r + 0.587*g + 0.114*b);
if (luminance<150) darkPixels ++;
}
if (darkPixels >= darkPixelThreshold)
isDark = YES;
I can just crop that part of the image, but this will be not efficient way, and wast time .
The solution marked correct here is a more thoughtful approach for getting the pixel data (more tolerant of differing formats) and also demonstrates how to address pixels. With a small adjustment, you can get the bottom of the image as follows:
+ (NSArray*)getRGBAsFromImage:(UIImage*)image
atX:(int)xx
andY:(int)yy
toX:(int)toX
toY:(int)toY {
// ...
int byteIndex = (bytesPerRow * yy) + xx * bytesPerPixel;
int byteIndexEnd = (bytesPerRow * toY) + toX * bytesPerPixel;
while (byteIndex < byteIndexEnd) {
// contents of the loop remain the same
// ...
}
To get the bottom third of the image, call this with xx=0, yy=2.0*image.height/3.0 and toX and toY equal to the image width and height, respectively. Loop the colors in the returned array and compute luminance as your post suggests.

Adjust Width of Core Text Tab

I am fairly new to Core Text but have been getting on well, however I'm having trouble adjusting the width of a tab.
Currently I'm using this code which I have written by looking at the documentation and mailing list:
CFIndex theNumberOfSettings = 1;
CFIndex i = 0;
CTTextTabRef tabArray[1];
CTTextAlignment align = 0;
CGFloat location = 80;
for (;i < 1; i++ ) {
tabArray[i] = CTTextTabCreate( align, location, NULL );
}
CFArrayRef tabStops = CFArrayCreate( kCFAllocatorDefault, (const void**) tabArray, 1, &kCFTypeArrayCallBacks );
for (;i < 1; i++ ) { CFRelease( tabArray[i] ); }
CTParagraphStyleSetting theSettings[1] =
{
{ kCTParagraphStyleSpecifierTabStops, sizeof(CFArrayRef), &tabStops },
};
CTParagraphStyleRef paragraphStyle = CTParagraphStyleCreate(theSettings, theNumberOfSettings);
[self.attributedString addAttribute:(NSString *)kCTParagraphStyleAttributeName
value:(id)paragraphStyle
range:range];
By setting the value of location I am able to adjust the width of the tab however this only works for the first tab created after that it resets to a different width which is very small comparatively.
Why is it doing this?
I would imagine the issue is all your tabs are created with the exact same location. Each one should probably be a multiple of 80, rather than being exactly 80.