Reading in multiple STEM signals into multiple datacubes - dm-script

I've written a through focus STEM acquisition script that reads in an image using the DSAcquire function, where I specify the signal to be read in with DSAcquireData(img, signalindex, etc.).
The nice thing about the above is that I can read in the image without it appearing on screen, copy it into a datacube, and then acquire the next one in the series, etc.
If I want to use two signals instead of one (eg HAADF and BF), it looks like the only way to do this is to use DSStartAcquisition after setting the digiscan parameters?
How should I go about copying signals into two preallocated image stacks (stack1, stack2)? Preferably without tens of images cluttering the screen (but ideally with some measure of progress?)

One way of doing this - by iterating over x individual acquisitions is a straight forward expansion of the F1 help examples:
// Acquire 2 signals simultaneously, e.g. HAADF and BF detector
number paramID
number width = 512 // pixel
number height = 512 // pixel
number rotation = 0 // degree
number pixelTime= 2 // microseconds
number lSynch = 1 // activated
paramID = DSCreateParameters( width, height, rotation, pixelTime, lSynch )
number signalIndex, dataType, selected, imageID
signalIndex = 0
dataType = 2 // 2 byte data
selected = 1 // acquire this signal
image img1 := IntegerImage( "Signal 1", dataType, 0, width, height )
img1.ShowImage()
imageID = img1.ImageGetID() // use displayed image
DSSetParametersSignal( paramID, signalIndex, dataType, selected, imageID )
signalIndex = 1
dataType = 2 // 2 byte data
selected = 1 // acquire this signal
image img2 := IntegerImage( "Signal 1", dataType, 0, width, height )
img2.Showimage()
imageID = img2.ImageGetID() // use displayed image
DSSetParametersSignal( paramID, signalIndex, dataType, selected, imageID )
number continuous = 0 // 0 = single frame, 1 = continuous
number synchronous = 1 // 0 = return immediately, 1 = return when finished
// Create stack to copy data to
number nplanes = 10
image stack1 := img1.ImageClone()
stack1.ImageResize(3,width,height,nplanes)
stack1.SetName("Signal 1 (stack)")
stack1.ShowImage()
image stack2 := img2.ImageClone()
stack2.ImageResize(3,width,height,nplanes)
stack2.SetName("Signal 2 (stack)")
stack2.ShowImage()
//Quickly arrange image
EGUPerformActionWithAllShownImages("arrange")
// Iterated acquisition
for( number i=0; i<nPlanes; i++ )
{
DSStartAcquisition( paramID, continuous, synchronous )
// Copy data
stack1.slice2(0,0,i, 0,width,1, 1,height,1) = img1
stack2.slice2(0,0,i, 0,width,1, 1,height,1) = img2
}
DSDeleteParameters( paramID ) // remove parameters from memory
However, this will restart a new Digiscan acquisition at each frame. You might want to prefer doing this with a continuous acquisition and an image listener instead.
For this, you would most likely hook up an image-listener as described in the F1 help here:
The Digiscan acquisition - depending on the speed - will update the image several times per frame, so you will need some check if it's end-of-frame.
One way to do this would be to use a data_value_changed event and then check if the last pixel in the image has changed value. Another option would be to use the tags_changed event, as apparently the tags of the acquisition image are update once per frame. Both options have some potential issues, though.
See also this very relevant recent question on this topic: Fastest way to see which pixel has changed when using a listener

Related

How to rotate Annotations when converting HDF5 files to dm3?

I am working on converting Velox file (HDF5) to .dm3 file using Tore Niermann's plugin (gms_plugin_hdf5) to read string. Annotations on HDF5 file also need to transfer to .dm3 file. HDF5 file maybe rotate in any angle. But the position coordinate of annotation read from hdf5 file is corresponding to images without rotating.
I found that the annotations didn't move with rotating images. I had to re-calculate the position coordinate for every annotation. It isn't convenient for annotations such as box or oval. And I need to extract maximum area when rotating images. So the image size will change with rotation angle. So is there better solutions for rotating the annotations? Thanks.
Here is a sample function from my script. I didn't attach all because it's quite long.
image GetAnnotations(Taggroup names, string filename, string name, Taggroup Annotations, Image VeloxImg, number Angle)
{
number i, j, imagex, imagey, xscale, yscale
String Displaypath, AnnotationStr, DisplayStr, units
taggroup attr = NewTagList()
getsize(VeloxImg, imagex, imagey)
number centerx=imagex/2
number centery=imagey/2
getscale(veloximg, xscale, yscale)
units=getunitstring(veloximg)
component imgdisp=imagegetimagedisplay(VeloxImg, 0)
For (j=0; j<TagGroupCountTags(Annotations); ++j)
{
TagGroupGetIndexedTagAsString(Annotations, j, AnnotationStr)
string AnnotPath=h5_read_string_dataset(filename, AnnotationStr)
string AnnotDataPath=GetValueFromLongStr(AnnotPath, "dataPath\": \"", "\"")
AnnotDataPath=ReplaceStr(AnnotDataPath, "\\/", "\/")
string AnnotLabel=GetValueFromLongStr(AnnotPath, "label\": \"", "\"")
string AnnotDrawPath=h5_read_string_dataset(filename, AnnotDataPath)
image img := RealImage( "", 4, 1, 1 )
TagGroup AnnoTag=alloc(MetaStr2TagGroup).ParseText2ImageTag(AnnotDrawPath, img )
deleteimage(img)
string AnnotDrawType=TagGroupGetTagLabel(AnnoTag,0)
//AnnoTag.TagGroupOpenBrowserWindow( "AnnotationsTag", 0 )
if (AnnotDrawType=="arrow")
{
number p1_x,p1_y,p2_x,p2_y
TagGroupGetTagAsNumber(AnnoTag, "arrow:p1:x", p1_x)
TagGroupGetTagAsNumber(AnnoTag, "arrow:p1:y", p1_y)
TagGroupGetTagAsNumber(AnnoTag, "arrow:p2:x", p2_x)
TagGroupGetTagAsNumber(AnnoTag, "arrow:p2:y", p2_y)
//VeloxImg.CreateArrowAnnotation( p1y, p1x, p2y, p2x )
number p1_x_new=(p1_x-0.5)*cos(Angle)+(p1_y-0.5)*sin(Angle)+0.5
number p1_y_new=-(p1_x-0.5)*sin(Angle)+(p1_y-0.5)*cos(Angle)+0.5
number p2_x_new=(p2_x-0.5)*cos(Angle)+(p2_y-0.5)*sin(Angle)+0.5
number p2_y_new=-(p2_x-0.5)*sin(Angle)+(p2_y-0.5)*cos(Angle)+0.5
result(p1_x+" "+p1_y+" new "+p1_x_new+" "+p2_y_new+"\n")
component arrowAnno=newarrowannotation(p1_y_new*imagey, p1_x_new*imagex, p2_y_new*imagey, p2_x_new*imagex)
arrowAnno.ComponentSetForegroundColor( 1, 0 , 0 )
arrowAnno.ComponentSetDrawingMode( 2 )
imgdisp.ComponentAddChildAtEnd( arrowAnno )
}
Not directly answering your question, but maybe nevertheless of interest to you:
While GMS does not support the rotation of annotations (Rect, Oval, Text, ImageDisplay...) it does support a rotation property for ROIs. So maybe you can just use rect-ROIs and oval-ROIs instead of annotations in your application.
Example (never mind, that I did the shift-computation wrongly):
image test := realImage("Test",4,512,512)
test= abs(sin(6*PI()*icol/iwidth))*abs(cos(4*PI()*irow/iheight*iradius/150))
test.showimage()
imageDisplay disp = test.ImageGetImageDisplay(0)
ROI box = NewROI()
box.RoiSetRectangle(46,88,338,343)
box.RoiSetVolatile(0)
ROI oval = NewROI()
oval.ROISetOval(221,226,287,254)
oval.RoiSetVolatile(0)
disp.ImageDisplayAddROI(box)
disp.ImageDisplayAddROI(oval)
number rot_deg = 8
image rot := test.rotate( pi()/180*rot_deg)
rot.ShowImage()
imageDisplay disp_rot = rot.ImageGetImageDisplay(0)
ROI box_rot = box.ROIClone()
ROI oval_rot = oval.ROIClone()
disp_rot.ImageDisplayAddROI(box_rot)
disp_rot.ImageDisplayAddROI(oval_rot)
number shift_x = (rot.ImageGetDimensionSize(0)-test.ImageGetDimensionSize(0)) / 2
number shift_y = (rot.ImageGetDimensionSize(1)-test.ImageGetDimensionSize(1)) / 2
number t,l,b,r
box_rot.ROIGetRectangle(t,l,b,r)
box_rot.ROISetRectangle(t+shift_y,l+shift_x,b+shift_y,r+shift_x)
oval_rot.ROIGetOval(t,l,b,r)
oval_rot.ROISetOval(t+shift_y,l+shift_x,b+shift_y,r+shift_x)
box_rot.ROISetRotationAngle( rot_deg )
oval_rot.ROISetRotationAngle( rot_deg )
If I understood you correctly, then your source data (HDF5) stores the image (2D array?) plus a rotation angle, but the annotations in the coordinate system of the (not rotated) image? How is the source-data displayed in the original software then? (Is it showing a rotated rectangle-image?)
GMS does not support rotating imagesDisplays (as objects) and consequently also not rotations of annotations. The coordinates systems are always screen-axis aligned orthogonal. Hence the need for interpolation when "rotating" images. The data values are re-computed for the new grid.
If you don't need the annotations to be adjustable after your input, one potential thing you could do would be to create an "as displayed" image after import prior rotation, and then rotate the image with the annotations "burnt in".
This is obviously only good for creating "final display images" though.
image before := realImage("Test",4,512,512)
before = abs(sin(6*PI()*icol/iwidth))*abs(cos(4*PI()*irow/iheight*iradius/150))
before.showimage()
before.ImageGetImageDisplay(0).ComponentAddChildAtEnd(NewArrowAnnotation(30,80,60,430))
before.ImageGetImageDisplay(0).ComponentAddChildAtEnd(NewArrowAnnotation(30,80,206,240))
before.ImageGetImageDisplay(0).ComponentAddChildAtEnd(NewOvalAnnotation(206,220,300,256))
// Create as-displayed image
Number t,l,b,r,ofx,ofy,scx,scy
before.ImageGetOrCreateImageDocument().ImageDocumentGetViewExtent(t,l,b,r)
before.ImageGetOrCreateImageDocument().ImageDocumentGetViewToWindowTransform(ofx,ofy,scx,scy)
image asShown := before.ImageGetOrCreateImageDocument().ImageDocumentCreateRGBImageFromDocument(round((r-l)*scx),round((b-t)*scy),0,0)
asShown.ShowImage()
number angle_deg = 8
image rotated := asShown.Rotate( angle_deg/180*PI() )
rotated.ShowImage()

Adjusting image contrast in FIB/SEM image - without affecting the text bar at the bottom of the image

FIB/SEM images have a text bar at the bottom of the image.
When imported into GMS, any contrast, gamma, .. adjustment also affects the text bar.
Is it possible to break up the image and have the data processing affect only the actual image - not the text bar?
The best you can do here is to break the actual image array into 2 separate images and then have the text-bar section displayed as a separate imageDisplay which you can add onto the imageDisplay of the data. You can shift/scale them with respect to each other, and you can also lock the added display so that it can not be shifted by mouse anymore. The following example should do what you need:
void CropAndMerge(Image img, number h){
number sx = img.ImageGetDimensionSize(0)
number sy = img.ImageGetDimensionSize(1)
image data := img.slice2(0,0,0,0,sx,1,1,sy-h,1).ImageClone() // note ":=", we sub-slice keeping tags and all
image anno = img.slice2(0,sy-h,0,0,sx,1,1,h,1) // note "=", we just want the data copy
imageDisplay disp
// Simple way to get imageDisplay. First show, then grab
// data.ShowImage()
// disp = data.ImageGetImageDisplay(0)
// Better alternative: No need to show
imageDocument doc = NewImageDocument( img.ImageGetName() )
doc.ImageDocumentAddImage( data )
// doc.ImageDocumentAddImage( anno ) // Use this to add 'side ordered' in case of page-view type. However, I'd rather not use page-mode.
disp = data.ImageGetImageDisplay(0)
disp.ImageDisplaySetColorTableByName( "Black Body Extended" ) // Just to show you can act on the display before actually showing it that way
// Add Annotation area as annotation on imageDisplay (all are components)
imageDisplay annoDisp = NewImageDisplay( anno, "best" )
disp.ComponentAddChildAtEnd( annoDisp )
// move out of the way
// ComponentPositionAroundPoint: Moves the annotation so the 'rel_x' horizontal point in the bounding rect is at 'new_x' (if bool horz is true), and for y accordinglye
number doVert = 1
number rel_y = 1.0 // bottom (relative coordinate!)
number new_y = sy // becomes bottom (absolute position)
annoDisp.ComponentPositionAroundPoint( 0,new_y,0,rel_y,0,doVert)
// make sure nobody messes with the annotation area
annoDisp.ComponentSetSelectable(0)
doc.ImageDocumentShow()
}
number sx = 1024
number sy = 1024
number h = 300
image in := realimage("Imported",4,sx,sy)
in = (icol%100 + iradius*sin(irow/iheight*5*Pi() + itheta )**2)
in.slice2(0,sy-h,0,0,sx,1,1,h,1) = (icol+irow)%50>45?max(in)+100:0
//in.showimage()
CropAndMerge(in.imageClone(),h)

How to add annotation information in each frame of a stack image?

I know how to add the arrows in a single image displayed front, but now I need to add the arrow annotations on each frame of a image stack to indicate the contrast change position and show them using the GSM "slice player". How to do it?
There is no difference between a 2D image and a 3D stack in DigitalMicrograph. Both are just dimensional data. As such "slices" in a Stack do neither have their individual tags nor annotations - there is just a single imageTagGroup and a single imageDisplays.
So to achieve what you want you need a different approach. You need to move your annotation whenever the display updates to shows a different slice.
In order to do this, you need to add a display-listener to your image display and act on the slice_property_changed event.
A basic example script for this:
Class CStackAnno
{
ImageDisplay disp
Component arrow
Number ListenerID
// This method is called whenever the imagedisplay fires the slice update event
void OnSlicePropChanged( object self, Number disp_flags, ImageDisplay disp, Number flags1, Number flags2, object slice_id_beg, object slice_id_end )
{
image img := disp.ImageDisplayGetImage()
if ( 3 != img.ImageGetNumDimensions() ) return
if ( !arrow.ComponentIsValid() ) return
number sx = img.imageGetDimensionsize(0)
number sy = img.imageGetDimensionsize(1)
number sz = img.imageGetDimensionsize(2)
number start, end
disp.ImageDisplayGetDisplayedLayers( start, end )
number kLineEndPoint = 2
arrow.ComponentSetControlPoint( kLineEndPoint, sx/sz * start, sx/sz * start, 0 )
}
Object Launch( object self, image Img )
{
if ( !img.ImageIsValid() ) Throw( "Invalid input image." )
if ( 3 != img.ImageGetNumDimensions() ) Throw( "This script only supports 3D images." )
disp = img.ImageGetImageDisplay(0)
// Register DisplayListener to catch when it updates
ListenerID = disp.ImageDisplayAddEventListener( self, "slice_property_changed:OnSlicePropChanged" )
// Add the annotation
arrow = NewArrowAnnotation( img.ImageGetDimensionSize(1)/5, img.ImageGetDimensionSize(0)*4/5, 0, 0 )
arrow.ComponentSetForegroundColor( 0, 0.5 , 1 )
arrow.ComponentSetBackgroundColor( 0, 0.8 , 1 )
arrow.ComponentSetDrawingMode( 1 )
disp.ComponentAddChildAtEnd( arrow )
return self
}
}
//Main call
image fImg
GetFrontImage(fImg)
Alloc(CStackAnno).Launch(fImg)

How to apply virtual apperture with 4D-STEM dataset in EFFICIENT way?

I would like to apply arbitrarily defined bit mask as virtual aperture and apply it to 4D-STEM data set in an EFFICIENT way.
I did it using the SliceN function and apply the mask pixel-by-pixel, which is very slow for large datasets. How to optimize it to so to run faster?
Image 4DSTEM := GetFrontImage() // dimention [ScanX, ScanY, Dx, Dy]
Image mask: = iradius // just an arbitrary mask (aperture)
Image out // dimention [ScanX, ScanY]
for (number i=0; i<ScanX; i++)
{ for (number j=0; j<ScanY; j++)
{
Diff2D = 4DSTEM.SliceN(4,2,i,j,0,0,2,Dx,1,3,Dy,1)
out.setpixel(i,j, sum(diff2D*mask))
}
}
out.showimage()
for an [100,100,512,512] dataset, that took few minutes to finish. When I have to repeat the operation several times, that is way to slow compare to matrix operation. but I dont know how to make it in an efficient way.
Thanks!
you're hitting the limitations of scripting languages here. Using sliceN is already pretty much the optimum you can get to, unfortunately. Everything else in speed optimization requires parallelized, compiled code. (i.e. you could code C++ code and use the SDK to compile your own plugin.)
However, there is a bit of room for improvement over your example.
First of all, your example above doesn't run :c) But that is quickly fixed.
Point #1:
Try to avoid number type casting. DM script only knows number but internally there is a difference between the proper number types (integer, floating point, signed/unsigned, byte-size). The script languages uses real-4-byte as the default unless told differently explicitly. And some methods will return real-4-byte by default. For this reason, the processing will be fastest, if both data and mask use real-4-byte data as well.
In my testing, the time-difference between running with uint16 data plus uint8 mask and *real4 data plus real4 mask) was significant! Nearly 30% time difference.
Point #2:
Don't copy you sliced image! Use := not = for your Dif2D.
The SliceN command returns an expression directly addressing the required memory. You can use it directly in any other expression (like I do below) or you can assign an image variable to it using := to give it a name.
The speed increase is not huge, but it's one copy-operation less per loop iteration.
Point #3:
You additional knowledge: Now for arbitrary masks there is not much you can do, but most often masks are zero-valued over large stretches and it is possible to define a smaller ROI containing all non-zero points. If this is the case, you can limit your math operations to that region.
i.e. instead of multiplying the whole DP with the same sized mask, just use a smaller mask and use the according sub-section of the DP.
This can actually make a big difference, but it will depend on your mask.
Of course you need to "find" this ROI first. In my script below I'm having a helper method to do that, utilizing the comparatively fast max() command and image rotation as trick for speed-up.
Point #4:
...would be to get rid of the double-for loop and replace it with image-expressions. Unfortunately, DigitalMicrograph does currently (GMS 3.3) not support this for 4D or 5D data.
The script below executed on a [53 x 52 x 512 x 512] STEM DI (of real-4 byte data) gave me the following timings:
Original: 12.80910 sec
Test 1 : 10.77700 sec
Test 2 : 1.83017 sec
// Helper class for timing
class CTimer{
number s
string n
~CTimer(object self){result("\n"+n+": "+ (GetHighResTickCount()-s)/GetHighResTicksPerSecond()+" sec");}
object Start(object self, string n_) { n=n_; s=GetHighResTickCount(); return self;}
}
// Helper method to find best non-zero containing ROI
void GetNonZeroArea( image src, number &t, number &l, number &b, number &r )
{
image work = !!src // Make a binary image which is 0 only where src==0
number d
max(work,d,t) // get "first" non-zero pixel coordinate, this is y = dist from TOP
rotateRight(work) // rotate image right
max(work,d,l) // get "first" non-zero pixel coordinate, this is y = dist from LEFT
rotateRight(work) // rotate image right
max(work,d,b) // get "first" non-zero pixel coordinate, this is y = dist from BOTTOM
b = work.ImageGetDimensionSize(1) - b // Opposite side!
rotateRight(work) // rotate image right
max(work,d,r) // get "first" non-zero pixel coordinate
r = work.ImageGetDimensionSize(1) - r // Opposite side!
}
// The original proposed script (plus fixes to make it actually run)
image Original(image STEM4D, image mask)
{
Number ScanX = STEM4D.ImageGetDimensionSize(0)
Number ScanY = STEM4D.ImageGetDimensionSize(1)
Number Dx = STEM4D.ImageGetDimensionSize(2)
Number Dy = STEM4D.ImageGetDimensionSize(3)
Image out := RealImage("Test1",4,ScanX,ScanY)
for (number i=0; i<ScanX; i++)
{ for (number j=0; j<ScanY; j++)
{
image Diff2D = STEM4D.SliceN(4,2,i,j,0,0,2,Dx,1,3,Dy,1)
out.setpixel(i,j, sum(Diff2D*mask))
}
}
return out
}
// Remove copying the slice, just reference it
image Test1(image STEM4D, image mask)
{
Number ScanX = STEM4D.ImageGetDimensionSize(0)
Number ScanY = STEM4D.ImageGetDimensionSize(1)
Number Dx = STEM4D.ImageGetDimensionSize(2)
Number Dy = STEM4D.ImageGetDimensionSize(3)
Image out := RealImage("Test1",4,ScanX,ScanY)
for (number i=0; i<ScanX; i++)
{ for (number j=0; j<ScanY; j++)
{
image Diff2D := STEM4D.SliceN(4,2,i,j,0,0,2,Dx,1,3,Dy,1)
out.setpixel(i,j, sum(Diff2D*mask))
}
}
return out
}
// Limit mask size to what is needed!
image Test2(image STEM4D, image mask )
{
Number ScanX = STEM4D.ImageGetDimensionSize(0)
Number ScanY = STEM4D.ImageGetDimensionSize(1)
Number Dx = STEM4D.ImageGetDimensionSize(2)
Number Dy = STEM4D.ImageGetDimensionSize(3)
Image out := RealImage("Test1",4,ScanX,ScanY)
Number t,l,b,r
GetNonZeroArea(mask,t,l,b,r)
Number w = r - l
Number h = b - t
image subMask := mask.slice2(l,t,0, 0,w,1, 1,h,1 )
for (number i=0; i<ScanX; i++)
for (number j=0; j<ScanY; j++)
out.setpixel(i,j, sum(STEM4D.SliceN(4,2,i,j,l,t,2,w,1,3,h,1)*subMask))
return out
}
Image src := GetFrontImage() // dimention [ScanX, ScanY, Dx, Dy]
Number ScanX = src.ImageGetDimensionSize(0)
Number ScanY = src.ImageGetDimensionSize(1)
Number Dx = src.ImageGetDimensionSize(2)
Number Dy = src.ImageGetDimensionSize(3)
Number r = 50 // mask radius
Image maskImg := RealImage("Mask",4,Dx,Dy)
maskImg = iradius < r ? 1 : 0 // just an aperture mask
image resultImg
{
object timer = Alloc(CTimer).Start("Original")
resultImg := Original(src,maskImg)
}
resultImg.SetName("Oringal")
resultImg.ShowImage()
{
object timer = Alloc(CTimer).Start("Test 1")
Test1(src,maskImg).ShowImage()
}
resultImg.SetName("Test 1")
resultImg.ShowImage()
{
object timer = Alloc(CTimer).Start("Test 2")
Test2(src,maskImg).ShowImage()
}
resultImg.SetName("Test 2")
resultImg.ShowImage()
Compiled code comparison:
Now, it should be added that the above script still is rather slow. Because it is iterating and using script language. The fully compiled c++ code of DigitalMicrograph is much faster. So if you have the licensed packages giving you the SI menu, then you want to use the SI/Map/Signal command. This is near-instantaneous for the example STEM DI I've mentioned above. My other answer shows how one could utilize this functionality by script.
As mentioned in my other answer, a real speed-win comes when compiled, parallelized code is used. DigitalMicrograph does this, after all, in the available SI "signal" map functionality. This feature is not available in the free version, but if you have Spectrum-Imaging acquisition, you most likely have the appropriated license as well.
The answer below utilizes this functionality by accessing the UI with the command ChooseMenuItem() and applying a few more tricks. The script is a bit lengthy, but its parts also show some other nice tricks worthwhile knowing:
TestSignalIntegrationInSI is the main script demoing how things can work.
CreatePickerByScript shows how one can create picker-spectra on SIs. This is used to open a 'Picker Diffraction Pattern' image from the STEM DI.
AddTestMasksToDP_ROIs programmatically adds ROIs to the diffraction pattern to be used as mask
AddTestMasksToDP_Threshold programmatically adds an intensity-threshold mask to be used as mask.
AddTestMasksToDP_DPMasks programmatically adds the various types of diffraction-masks to be used as mask
GetIntegratedSignalViaSIMenu is the central step of the script. With a picker-DP and required 'masks' on it front-most, the menu command is called to perform the signal-extraction (as fast as possible.) Then the displayed result-image is returned.
GetNewestImage is just a utility method showing how on can access the latest memory-created image.
Here is the script:
image GetNewestImage()
{
// New images get the next higher imageID.
// This can be used to identify the "latest" created image.
if ( 0 == CountImages() ) Throw( "No image in memory!" )
// We create a temp. image to get the uppder limit
number lastID = RealImage("Dummy",4,1).ImageGetID()
// Then we search for the next lower existing one
image lastImg
for( number ID = lastID - 1; ID>0; ID-- )
{
lastImg := FindImageByID(ID)
if ( lastImg.ImageIsValid() ) break
}
return lastImg
}
image CreatePickerByScript( image SI, number t, number l, number b, number r )
{
if ( SI.ImageGetNumDimensions()<3 ) Throw( "Sorry, LineScans are not supprorted here." )
// Adding a non-volatile ROI of specific RoiNAME acts as if using
// the picker-tool. The ID string must be unique!
ROI pickerROI = NewROI()
pickerROI.RoiSetVolatile( 0 )
string uniqueID = GetDate(0)+"#"+GetTime(1)+";"+round(random()*1000)
pickerROI.RoiSetName( "SICursor(##"+uniqueID+"##)" )
SI.ImageGetImageDisplay(0).ImageDisplayAddROI( pickerROI )
// This creates the picker image.
// So the child is now the "newest" image in memory
image child := GetNewestImage()
return child
}
void AddTestMasksToDP_ROIs( image DP )
{
// Add ROIs to the DP which are your masks (any numebr and type of ROI works)
imageDisplay DPdisp = DP.ImageGetImageDisplay(0)
number dpX = DP.ImageGetDimensionSize(0)
number dpY = DP.ImageGetDimensionSize(1)
// Only simple RECT ROIs are supported
ROI maskRoi1 = NewROI()
maskRoi1.ROISetRectangle( dpY*0.1, dpX*0.1, dpY*0.8, dpX*0.3 )
DPdisp.ImageDisplayAddROI(maskRoi1)
// Arbitrary multi-vertex (use for ovals etc.)
ROI maskRoi2 = NewROI()
maskRoi2.ROISetRectangle( dpY*0.7, dpX*0.1, dpY*0.9, dpX*0.9 )
DPdisp.ImageDisplayAddROI(maskRoi2)
}
void AddTestMasksToDP_Threshold( image DP )
{
// Add intensity treshhold mask (highest 95% intensity range)
imageDisplay DPdisp = DP.ImageGetImageDisplay(0)
DPdisp.RasterImageDisplaySetThresholdOn( 1 )
number low = max(DP) * 0.05
number high = max(DP)
DPdisp.RasterImageDisplaySetThresholdLimits( low, high )
}
void AddTestMasksToDP_DPMasks( image DP )
{
// Add Diffraction masks to the DP
imageDisplay DPdisp = DP.ImageGetImageDisplay(0)
// Spot masks (always symmetric pair)
Component spotMask = NewComponent(8,0,0,0,0) // 8 = Spotmask
spotMask.ComponentSetControlPoint(4, 0, 0,0) // 4 = TopLeft of one spot [Size only]
spotMask.ComponentSetControlPoint(7,10,10,0) // 7 = BottomRight of one spot [Size only]
spotMask.ComponentSetControlPoint(8,150,0,0) // 8 = Spot position [center]
DPdisp.ComponentAddChildAtEnd(spotMask)
// Bandpass mask (Only circles are correctly supported)
Component bandpassMask = NewComponent(15,0,0,0,0) // 15 = Bandpass (ring)
number r1 = 100
number r2 = 120
bandpassMask.ComponentSetControlPoint(7,r1,r1,0) // 7 = BottomRight of one ring [Size only]
bandpassMask.ComponentSetControlPoint(14,r2,r2,0) // 14 = BottomRight of one ring [Size only]
DPdisp.ComponentAddChildAtEnd(bandpassMask)
// Wege mask (symmetric)
Component wedgeMask = NewComponent(19,0,0,0,0) // 19 = wedgemask (ringsegment)
wedgeMask.ComponentSetControlPoint(9,10,20,0) // 9 = One wedge vector
wedgeMask.ComponentSetControlPoint(10,-20,40,0) // 10 = Other wedge vector
DPdisp.ComponentAddChildAtEnd(wedgeMask)
// Array mask (symmetric)
Component arrayMask = NewComponent(9,0,0,0,0) // 9 = arrayMask (ringsegment)
arrayMask.ComponentSetControlPoint(9,-70,-60,0) // 9 = One array vector
arrayMask.ComponentSetControlPoint(10,99,-99,0) // 10 = Other array vector
arrayMask.ComponentSetControlPoint(4, 0, 0,0) // 4 = TopLeft of one spot [Size only]
arrayMask.ComponentSetControlPoint(7,20,20,0) // 7 = BottomRight of one spot [Size only]
DPdisp.ComponentAddChildAtEnd(arrayMask)
}
image GetIntegratedSignalViaSIMenu( image pickerChild )
{
// Call the Menu to do the work
// The picker-spectrum or DP needs to be front-most
pickerChild.SelectImage()
ChooseMenuItem("SI","Map","Signal")
// The created signal map is NOT the newest image
// (some internal iamges are created for the mask)
// but it is the front-most displayed one.
image signalMap := GetFrontImage()
return signalMap
}
image GetMaskFromSignalMap( image signalMap, number DPx, number DPy )
{
// The actual mask is stored in the tags
string tagPath = "Processing:[0]:Parameters:Mask"
tagGroup tg = signalMap.ImageGetTagGroup()
if ( !tg.TagGroupDoesTagExist(tagPath) )
Throw( "Sorry, no mask tag found." )
image mask := RealImage("Mask",4,DPx, DPy )
if ( !tg.TagGroupGetTagAsArray(tagPath,mask) )
Throw( "Sorry, could not retrieve mask. Maybe wrong size?" )
return mask
}
void TestSignalIntegrationInSI()
{
image STEMDI := GetFrontImage()
image DP := STEMDI.CreatePickerByScript(0,0,1,1)
if ( TwoButtonDialog( "Add ROIs as mask?", "Yes", "No" ) )
AddTestMasksToDP_ROIs( DP )
else if ( TwoButtonDialog( "Add intensity treshold as mask?", "Yes", "No" ) )
AddTestMasksToDP_Threshold( DP )
else if ( TwoButtonDialog( "Add diffraction masks as mask?", "Yes", "No" ) )
AddTestMasksToDP_DPMasks( DP )
image signalMap := GetIntegratedSignalViaSIMenu( DP )
number dpX = DP.ImageGetDimensionSize(0)
number dpY = DP.ImageGetDimensionSize(1)
// We may want to close the DP again. No longer needed
//DP.DeleteImage()
// Verification: Get Mask image form SignalMap
image usedMask := GetMaskFromSignalMap( signalMap, dpX, dpY )
usedMask.SetName( "This mask was used." )
usedMask.ShowImage()
}
TestSignalIntegrationInSI()
The solution below utilizes the intrinsic expression loops by performing in-place multiplication and then projection.
Disappointingly, it turns out the solution is actually a bit slower then the for-loop with the SliceN command.
For the same test-data of size [53 x 52 x 512 x 512] the resulting timing is:
Data copy: 1.28073 sec
Inplace multiply: 30.1978 sec
Project 1/2: 1.1208 sec
Project 2/2: 0.0019557 sec
InPlace multiplication with projections (total): 32.9045 sec
InPlace multiplication with projections (total): 34.9853 sec
// Helper class for timing
class CTimer{
number s
string n
~CTimer(object self){result("\n"+n+": "+ (GetHighResTickCount()-s)/GetHighResTicksPerSecond()+" sec");}
object Start(object self, string n_) { n=n_; s=GetHighResTickCount(); return self;}
}
image MaskMultipliedSum( image STEM4D, image MASK2D, number copyFirst )
{
// Boring feasability checks...
if ( 4 != STEM4D.ImageGetNumDimensions() )
Throw( "Input data is not 4D." )
if ( 2 != MASK2D.ImageGetNumDimensions() )
Throw( "Input mask is not 2D." )
Number ScanX = STEM4D.ImageGetDimensionSize(0)
Number ScanY = STEM4D.ImageGetDimensionSize(1)
Number Dx = STEM4D.ImageGetDimensionSize(2)
Number Dy = STEM4D.ImageGetDimensionSize(3)
if ( Dx != MASK2D.ImageGetDimensionSize(0) )
Throw ("X dimension of mask does not match input data." )
if ( Dy != MASK2D.ImageGetDimensionSize(1) )
Throw ("Y dimension of mask does not match input data." )
// Do the maths!
image workCopy4D
if ( copyFirst )
{
object timer = Alloc(CTimer).Start("Data copy")
workCopy4D = STEM4D
}
else
workCopy4D := STEM4D
{
object timer = Alloc(CTimer).Start("Inplace multiply")
workCopy4D *= MASK2D[idimindex(2),idimindex(3)]
}
// Now we want to "sum up" over Dx and Dy
image p1,p2
{
object timer = Alloc(CTimer).Start("Project 1/2")
p1 := project( workCopy4D, 3 )
}
{
object timer = Alloc(CTimer).Start("Project 2/2")
p2 := project( p1, 2 )
}
return p2
}
image stack4D, mask2D
If ( GetTwoLabeledImagesWithPrompt("Please select 4D data and 2D mask", "Select input", "4D data", stack4D, "2D mask", mask2D ) )
{
number doCopy = TwoButtonDialog("Create workcopy?","Yes (takes time)","No (overwrites input data!)")
object timer = Alloc(CTimer).Start("InPlace multiplication with projections (total)")
MaskMultipliedSum(stack4D,mask2D,doCopy).ShowImage()
}

Explain code in Kinect SDK

I am working with Kinect and reading example from DepthWithColor-D3D, has some code but i don't understand yet.
// loop over each row and column of the color
for (LONG y = 0; y < m_colorHeight; ++y)
{
LONG* pDest = (LONG*)((BYTE*)msT.pData + msT.RowPitch * y);
for (LONG x = 0; x < m_colorWidth; ++x)
{
// calculate index into depth array
int depthIndex = x/m_colorToDepthDivisor + y/m_colorToDepthDivisor * m_depthWidth;
// retrieve the depth to color mapping for the current depth pixel
LONG colorInDepthX = m_colorCoordinates[depthIndex * 2];
LONG colorInDepthY = m_colorCoordinates[depthIndex * 2 + 1];
How to calculate the value of colorInDepthX and colorInDepthY as above code?
colorInDepthX and colorInDepthY is a mapping between the depth and color images so that they will align. Because the Kinect's cameras are slightly offset from each other their field of views are not lined up perfectly.
m_colorCoordinates is defined at the top of the file as such:
m_colorCoordinates = new LONG[m_depthWidth*m_depthHeight*2];
This is a single dimension array representing a 2-dimensional image, it is populated just above the code block you post in your question:
// Get of x, y coordinates for color in depth space
// This will allow us to later compensate for the differences in location, angle, etc between the depth and color cameras
m_pNuiSensor->NuiImageGetColorPixelCoordinateFrameFromDepthPixelFrameAtResolution(
cColorResolution,
cDepthResolution,
m_depthWidth*m_depthHeight,
m_depthD16,
m_depthWidth*m_depthHeight*2,
m_colorCoordinates
);
As described in the comment, this is running an calculation provided by the SDK to map the color and depth coordinates onto each other. The result is placed inside of m_colorCoordinates.
colorInDepthX and colorInDepthY are simply values within the m_colorCoordinates array that are being acted upon in the current cycle of the loop. They are not "calculated", per se, but just point to what already exists in m_colorCoordinates.
The function that handles the mapping between color and depth images is explained in the Kinect SDK at MSDN. Here is a direct link:
http://msdn.microsoft.com/en-us/library/jj663856.aspx