Android 2D animation drawing: bad performance - android-animation

I have an app that draws a grid of dots (let's say 5x5). The user is asked to draw lines on that grid. If the user's finger touches one of the dots in the grid, this dot is being colored to show that this dot is part of a path drawn. In addition a line will be drawn between each two touched dots.
The issue - I get very bad performance, which causes few things:
The application gets really slow.
Motion events in event.getAction() get bad granularity. I meanenter code here that instead of registering a movement each 10 pixels for example, it registers movements each 100 pixels. This, in turn, will causes the app to NOT redraw some dots the user had touched.
Sometimes the motion coordinates are simple wrong: lets say the user is moving her finger from pixel 100 to pixel 500, the reading might show 100...200...150...140...300...400. For some reason the touch location gets messed up in some cases.
Look at the example on how the app "misses out" on dots the user have touched and doesn't draw the green dots:
I've tried few thing:
Adding Thread.sleep(100); to else if(event.getAction() == MotionEvent.ACTION_MOVE) inside onTouchEvent(MotionEvent event), I read that this might give the CPU time to catch up on all those touch events - didn't change a thing
Adding this.destroyDrawingCache() to the very end of doDraw() (I use it instead of onDraw, as was suggested by one tutorial I used). I thought this will clear all event/drawing caching which seems to be slowing down the system - didn't change a thing.
I am fairly new to Android animation so I am not sure how to proceed:
I understand I should do as little as possible in doDraw() (my onDraw()) and onTouchEvent().
I read some stuff about invalidate() but not sure how and when to use it. If I understand correctly, my View gets drawn anew each time doDraw() is called. My grid, for instance, is static - how can I avoid redrawing it?
++++++++++++++++++++++++ UPDATE 7th Oct +++++++++++++++++++++
I tried using canvas.drawCircle(xPos, yPos, 8, mNodePaint); instead of canvas.drawBitmap(mBitmap, xPos, yPos, null);. I thought that if I DIDN'T use actual bitmaps this might improve performance. As a matter of fact - it didn't! I am a bit confused how such a simple application can pose such a heavy load on the device. I must be doing something really the wrong way.
++++++++++++++++++++++++ UPDATE 12th Oct +++++++++++++++++++++
I took into account what #LadyWoodi suggested - I've eliminated all variable declarations out of the loops - anyway it is a bad practice and I also got rid of all the "System.Out" lines I use so I can log app behavior to better understand why I get such a lame performance. I am sad to say that if there was a change in performance (I didn't actually measure frame rate change) it is negligible.
Any other ideas?
++++++++++++++++++++++++ UPDATE 13th Oct +++++++++++++++++++++
As I have a static grid of dots (see hollow black/white dots in screenShot) that never changes during the game I did the following:
-Draw the grid once.
-Capture the drawing as bitmap using Bitmap.createBitmap().
-Use canvas.drawBitmap() to draw the bitmap of the static dots grid.
-When my thread runs I check to see it the grid of dots is drawn. If it is running I will NOT recreate the static dots grid. I will only render it from my previously rendered bitmap.
Surprisingly this changed nothing with my performance! Redrawing the dots grid each time didn't have a true visual effect on app performance.
I decided to use canvas = mHolder.lockCanvas(new Rect(50, 50, 150, 150)); inside my drawing thread. It was just for testing purposes to see if I limit the area rendered each time, I can get the performance better. This DID NOT help either.
Then I turned to the DDMS tool in Eclipse to try and profile the app. What it came up with, was that canvas.drawPath(path, mPathPaint); (Canvas.native_drawPath) consumed about 88.5% of CPU time!!!
But why??! My path drawing is rather simple, mGraphics contains a collection of Paths and all I do is figure out if each path is inside the boundaries of the game screen and then I draw a path:
//draw path user is creating with her finger on screen
for (Path path : mGraphics)
{
//get path values
mPm = new PathMeasure(path, true);
mPm.getPosTan(0f, mStartCoordinates, null);
//System.out.println("aStartCoordinates X:" + aStartCoordinates[0] + " aStartCoordinates Y:" + aStartCoordinates[1]);
mPm.getPosTan(mPm.getLength(), mEndCoordinates, null);
//System.out.println("aEndCoordinates X:" + aEndCoordinates[0] + " aEndCoordinates Y:" + aEndCoordinates[1]);
//coordinates are within game board boundaries
if((mStartCoordinates[0] >= 1 && mStartCoordinates[1] >= 1) && (mEndCoordinates[0] >= 1 && mEndCoordinates[1] >= 1))
{
canvas.drawPath(path, mPathPaint);
}
}
Can anyone see any ill programmed lines of code in my examples?
++++++++++++++++++++++++ UPDATE 14th Oct +++++++++++++++++++++
I've made changes to my doDraw()method. Basically what I do is draw the screen ONLY if something was changed. In all other cases I simply store a cached bitmap of the screen and render it. Please take a look:
public void doDraw(Canvas canvas)
{
synchronized (mViewThread.getSurefaceHolder())
{
if(mGraphics.size() > mPathsCount)
{
mPathsCount = mGraphics.size();
//draw path user is creating with her finger on screen
for (Path path : mGraphics)
{
//get path values
mPm = new PathMeasure(path, true);
mPm.getPosTan(0f, mStartCoordinates, null);
//System.out.println("aStartCoordinates X:" + aStartCoordinates[0] + " aStartCoordinates Y:" + aStartCoordinates[1]);
mPm.getPosTan(mPm.getLength(), mEndCoordinates, null);
//System.out.println("aEndCoordinates X:" + aEndCoordinates[0] + " aEndCoordinates Y:" + aEndCoordinates[1]);
//coordinates are within game board boundaries
if((mStartCoordinates[0] >= 1 && mStartCoordinates[1] >= 1) && (mEndCoordinates[0] >= 1 && mEndCoordinates[1] >= 1))
{
canvas.drawPath(path, mPathPaint);
}
}
//nodes that the path goes through, are repainted green
//these nodes are building the drawn pattern
for (ArrayList<PathPoint> nodePattern : mNodesHitPatterns)
{
for (PathPoint nodeHit : nodePattern)
{
canvas.drawBitmap(mDotOK, nodeHit.x - ((mDotOK.getWidth()/2) - (mNodeBitmap.getWidth()/2)), nodeHit.y - ((mDotOK.getHeight()/2) - (mNodeBitmap.getHeight()/2)), null);
}
}
mGameField = Bitmap.createBitmap(mGridNodesCount * mNodeGap, mGridNodesCount * mNodeGap, Bitmap.Config.ARGB_8888);
}
else
{
canvas.drawBitmap(mGameField, 0f, 0f, null);
}
Now for the results - as long as the device doesn't have to render no paths and simply draws from a bitmap, stuff goes very fast. But the moment I have to rerender the screen using canvas.drawPath() performance becomes as sluggish as a turtle on morphine... The more paths I have (up to 6 and more, which is NOTHING!) the slower the rendering. How odd is this?? - My paths are even not really curvy - the are all straight lines with an occasional turn. What I mean is that the line is not very "complex".
I've add more code below - if you have any improvements ideas.
Many thanks in advance,
D.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Class "Panel" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
public class Panel extends SurfaceView implements SurfaceHolder.Callback {
Bitmap mNodeBitmap;
int mNodeBitmapWidthCenter;
int mNodeBitmapHeightCenter;
Bitmap mDotOK;
ViewThread mViewThread;
ArrayList<PathPoint> mPathPoints;
private ArrayList<Path> mGraphics = new ArrayList<Path>(3);
private ArrayList<ArrayList<PathPoint>> mNodesHitPatterns = new ArrayList<ArrayList<PathPoint>>();
private Paint mPathPaint;
Path mPath = new Path();
//private ArrayList<Point> mNodeCoordinates = new ArrayList<Point>();
private int mGridNodesCount = 5;
private int mNodeGap = 100;
PathPoint mNodeCoordinates[][] = new PathPoint[mGridNodesCount][mGridNodesCount];
PathMeasure mPm;
float mStartCoordinates[] = {0f, 0f};
float mEndCoordinates[] = {0f, 0f};
PathPoint mPathPoint;
Boolean mNodesGridDrawn = false;
Bitmap mGameField = null;
public Boolean getNodesGridDrawn() {
return mNodesGridDrawn;
}
public Panel(Context context) {
super(context);
mNodeBitmap = BitmapFactory.decodeResource(getResources(), R.drawable.dot);
mNodeBitmapWidthCenter = mNodeBitmap.getWidth()/2;
mNodeBitmapHeightCenter = mNodeBitmap.getHeight()/2;
mDotOK = BitmapFactory.decodeResource(getResources(), R.drawable.dot_ok);
getHolder().addCallback(this);
mViewThread = new ViewThread(this);
mPathPaint = new Paint();
mPathPaint.setAntiAlias(true);
mPathPaint.setDither(true); //for better color
mPathPaint.setColor(0xFFFFFF00);
mPathPaint.setStyle(Paint.Style.STROKE);
mPathPaint.setStrokeJoin(Paint.Join.ROUND);
mPathPaint.setStrokeCap(Paint.Cap.ROUND);
mPathPaint.setStrokeWidth(5);
}
public ArrayList<ArrayList<PathPoint>> getNodesHitPatterns()
{
return this.mNodesHitPatterns;
}
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
}
public void surfaceCreated(SurfaceHolder holder) {
//setPadding(100, 100, 0, 0);
if (!mViewThread.isAlive()) {
mViewThread = new ViewThread(this);
mViewThread.setRunning(true);
mViewThread.start();
}
}
public void surfaceDestroyed(SurfaceHolder holder) {
if (mViewThread.isAlive()) {
mViewThread.setRunning(false);
}
}
//draw the basic nodes grid that the user will use to draw the lines on
//store as bitmap
public void drawNodesGrid(Canvas canvas)
{
canvas.drawColor(Color.WHITE);
for (int i = 0; i < mGridNodesCount; i++)
{
for (int j = 0; j < mGridNodesCount; j++)
{
int xPos = j * mNodeGap;
int yPos = i * mNodeGap;
try
{
//TODO - changed
mNodeCoordinates[i][j] = new PathPoint(xPos, yPos, null);
}
catch (Exception e)
{
e.printStackTrace();
}
canvas.drawBitmap(mNodeBitmap, xPos, yPos, null);
}
}
mNodesGridDrawn = true;
mGameField = Bitmap.createBitmap(mGridNodesCount * mNodeGap, mGridNodesCount * mNodeGap, Bitmap.Config.ARGB_8888);
}
public void doDraw(Canvas canvas)
{
canvas.drawBitmap(mGameField, 0f, 0f, null);
synchronized (mViewThread.getSurefaceHolder())
{
//draw path user is creating with her finger on screen
for (Path path : mGraphics)
{
//get path values
mPm = new PathMeasure(path, true);
mPm.getPosTan(0f, mStartCoordinates, null);
//System.out.println("aStartCoordinates X:" + aStartCoordinates[0] + " aStartCoordinates Y:" + aStartCoordinates[1]);
mPm.getPosTan(mPm.getLength(), mEndCoordinates, null);
//System.out.println("aEndCoordinates X:" + aEndCoordinates[0] + " aEndCoordinates Y:" + aEndCoordinates[1]);
//coordinates are within game board boundaries
if((mStartCoordinates[0] >= 1 && mStartCoordinates[1] >= 1) && (mEndCoordinates[0] >= 1 && mEndCoordinates[1] >= 1))
{
canvas.drawPath(path, mPathPaint);
}
}
//nodes that the path goes through, are repainted green
//these nodes are building the drawn pattern
for (ArrayList<PathPoint> nodePattern : mNodesHitPatterns)
{
for (PathPoint nodeHit : nodePattern)
{
canvas.drawBitmap(mDotOK, nodeHit.x - ((mDotOK.getWidth()/2) - (mNodeBitmap.getWidth()/2)), nodeHit.y - ((mDotOK.getHeight()/2) - (mNodeBitmap.getHeight()/2)), null);
}
}
this.destroyDrawingCache();
}
}
#Override
public boolean onTouchEvent(MotionEvent event) {
synchronized (mViewThread.getSurefaceHolder()) {
if(event.getAction() == MotionEvent.ACTION_DOWN)
{
//System.out.println("Action downE x: " + event.getX() + " y: " + event.getY());
for (int i = 0; i < mGridNodesCount; i++)
{
for (int j = 0; j < mGridNodesCount; j++)
{
//TODO - changed
//PathPoint pathPoint = mNodeCoordinates[i][j];
mPathPoint = mNodeCoordinates[i][j];
if((Math.abs((int)event.getX() - mPathPoint.x) <= 35) && (Math.abs((int)event.getY() - mPathPoint.y) <= 35))
{
//mPath.moveTo(pathPoint.x + mBitmap.getWidth() / 2, pathPoint.y + mBitmap.getHeight() / 2);
//System.out.println("Action down x: " + pathPoint.x + " y: " + pathPoint.y);
ArrayList<PathPoint> newNodesPattern = new ArrayList<PathPoint>();
mNodesHitPatterns.add(newNodesPattern);
//mNodesHitPatterns.add(nh);
//pathPoint.setAction("down");
break;
}
}
}
}
else if(event.getAction() == MotionEvent.ACTION_MOVE)
{
final int historySize = event.getHistorySize();
//System.out.println("historySize: " + historySize);
//System.out.println("Action moveE x: " + event.getX() + " y: " + event.getY());
coordinateFound:
for (int i = 0; i < mGridNodesCount; i++)
{
for (int j = 0; j < mGridNodesCount; j++)
{
//TODO - changed
//PathPoint pathPoint = mNodeCoordinates[i][j];
mPathPoint = mNodeCoordinates[i][j];
if((Math.abs((int)event.getX() - mPathPoint.x) <= 35) && (Math.abs((int)event.getY() - mPathPoint.y) <= 35))
{
int lastPatternIndex = mNodesHitPatterns.size()-1;
ArrayList<PathPoint> lastPattern = mNodesHitPatterns.get(lastPatternIndex);
int lastPatternLastNode = lastPattern.size()-1;
if(lastPatternLastNode != -1)
{
if(!mPathPoint.equals(lastPattern.get(lastPatternLastNode).x, lastPattern.get(lastPatternLastNode).y))
{
lastPattern.add(mPathPoint);
//System.out.println("Action moveC [add point] x: " + pathPoint.x + " y: " + pathPoint.y);
}
}
else
{
lastPattern.add(mPathPoint);
//System.out.println("Action moveC [add point] x: " + pathPoint.x + " y: " + pathPoint.y);
}
break coordinateFound;
}
else //no current match => try historical
{
if(historySize > 0)
{
for (int k = 0; k < historySize; k++)
{
//System.out.println("Action moveH x: " + event.getHistoricalX(k) + " y: " + event.getHistoricalY(k));
if((Math.abs((int)event.getHistoricalX(k) - mPathPoint.x) <= 35) && (Math.abs((int)event.getHistoricalY(k) - mPathPoint.y) <= 35))
{
int lastPatternIndex = mNodesHitPatterns.size()-1;
ArrayList<PathPoint> lastPattern = mNodesHitPatterns.get(lastPatternIndex);
int lastPatternLastNode = lastPattern.size()-1;
if(lastPatternLastNode != -1)
{
if(!mPathPoint.equals(lastPattern.get(lastPatternLastNode).x, lastPattern.get(lastPatternLastNode).y))
{
lastPattern.add(mPathPoint);
//System.out.println("Action moveH [add point] x: " + pathPoint.x + " y: " + pathPoint.y);
}
}
else
{
lastPattern.add(mPathPoint);
//System.out.println("Action moveH [add point] x: " + pathPoint.x + " y: " + pathPoint.y);
}
break coordinateFound;
}
}
}
}
}
}
}
else if(event.getAction() == MotionEvent.ACTION_UP)
{
// for (int i = 0; i < mGridSize; i++) {
//
// for (int j = 0; j < mGridSize; j++) {
//
// PathPoint pathPoint = mNodeCoordinates[i][j];
//
// if((Math.abs((int)event.getX() - pathPoint.x) <= 35) && (Math.abs((int)event.getY() - pathPoint.y) <= 35))
// {
// //the location of the node
// //mPath.lineTo(pathPoint.x + mBitmap.getWidth() / 2, pathPoint.y + mBitmap.getHeight() / 2);
//
// //System.out.println("Action up x: " + pathPoint.x + " y: " + pathPoint.y);
//
// //mGraphics.add(mPath);
// // mNodesHit.add(pathPoint);
// // pathPoint.setAction("up");
// break;
// }
// }
// }
}
//System.out.println(mNodesHitPatterns.toString());
//create mPath
for (ArrayList<PathPoint> nodePattern : mNodesHitPatterns)
{
for (int i = 0; i < nodePattern.size(); i++)
{
if(i == 0) //first node in pattern
{
mPath.moveTo(nodePattern.get(i).x + mNodeBitmapWidthCenter, nodePattern.get(i).y + mNodeBitmapHeightCenter);
}
else
{
mPath.lineTo(nodePattern.get(i).x + mNodeBitmapWidthCenter, nodePattern.get(i).y + mNodeBitmapWidthCenter);
}
//mGraphics.add(mPath);
}
}
mGraphics.add(mPath);
return true;
}
}
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Class "ViewThread" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
public class ViewThread extends Thread {
private Panel mPanel;
private SurfaceHolder mHolder;
private boolean mRun = false;
public ViewThread(Panel panel) {
mPanel = panel;
mHolder = mPanel.getHolder();
}
public void setRunning(boolean run) {
mRun = run;
}
public SurfaceHolder getSurefaceHolder()
{
return mHolder;
}
#Override
public void run()
{
Canvas canvas = null;
while (mRun)
{
canvas = mHolder.lockCanvas();
//canvas = mHolder.lockCanvas(new Rect(50, 50, 150, 150));
if (canvas != null)
{
if(!mPanel.getNodesGridDrawn())
{
mPanel.drawNodesGrid(canvas);
}
mPanel.doDraw(canvas);
mHolder.unlockCanvasAndPost(canvas);
}
}
}
}

It's just the idea, but I would try to take all the declarations out of the loops. I know that it can be useful to have them localized, however it's usually really time consuming so it could help a little. My second idea was already tested by you in your update so now I am also curious how it will go ;)

You are using a SurfaceView? First of all, I recommend you to use a graphic library for your game... AndEngine for example is pretty easy to use and you will achieve to develop a much more beautiful game than using the Java canvas. The performance is better too.
I canĀ“t find anything wrong with your code, but there is a lot of processing in the draw method, and more important, in the onTouch event. You should avoid to use divisions or heavy math operations in the loops and try to pre-calculate everything before.
But I insist; for something like what you are doing, take a look at this and you will have it up and running in no time!

Related

Adding two ints not doing anything

So the problem is, when I say ovrxp = ovrxp + xp, it never stacks and just resets every kill. A fix to this and an explanation to why this doesn't work would be much appreciated.
#EventHandler
public void onDeath(EntityDeathEvent e) {
Player player = (Player) e.getEntity().getKiller();
Skeleton s = (Skeleton) e.getEntity();
int ovrlvl = 1;
int ovrxp = 0;
Random random = new Random();
int xp = random.nextInt(30) + 21;
if (ovrlvl == 1 && ovrxp >= 200) {
player.sendMessage(ChatColor.GREEN + "You are now level two!");
player.playSound(player.getLocation(), Sound.LEVEL_UP, 1.0F, 0.0F);
ovrlvl = 2;
}
if (ovrlvl == 2 && ovrxp >= 400) {
player.sendMessage(ChatColor.GREEN + "You are now level three!");
player.playSound(player.getLocation(), Sound.LEVEL_UP, 1.0F, 0.0F);
ovrlvl = 3;
}
ovrxp = ovrxp + xp;
if (s.getCustomName() == "Undead Recruit") {
if (ovrlvl == 1) {
player.sendMessage(ChatColor.GREEN + "" + ovrxp + "/200");
}
if (ovrlvl == 2) {
player.sendMessage(ChatColor.GREEN + "" + ovrxp + "/400");
}
}
}
}
You've declared ovrxp as a local variable - it's initialized each time onDeath is called.
If you want the value to persist between multiple calls to the method, you'll need to make the variable a field (part of the object itself). Assuming the method is always called on the same object, and on the same thread, just making it an instance field should be fine.

Player should win when all objects are collected

I have a simple game where the player needs to collect 4 game objects within 30 sec. Now I already created the timer, so I need to let the game know that if all game objects are collected under the time limit the player wins.
This is my code so far:
using UnityEngine;
using System.Collections;
public class GameState : MonoBehaviour
{
public static int count = 0;
public float seconds = 30;
public float minutes = 0;
// Use this for initialization
void Start ()
{
}
// Update is called once per frame
void Update ()
{
if (seconds <= 0)
{
seconds = 30;
if (minutes >= 1)
{
minutes -- ;
}
else
{
minutes = 0;
seconds = 0;
GameObject.Find("TimerText").guiText.text = minutes.ToString("f0") + ":0" + seconds.ToString("f0");
}
}
else
{
seconds -= Time.deltaTime;
}
if (Mathf.Round(seconds) <=9)
{
GameObject.Find("TimerText").guiText.text = minutes.ToString("f0") + ":0" + seconds.ToString("f0");
}
else
{
GameObject.Find("TimerText").guiText.text = minutes.ToString("f0") + ":" + seconds.ToString("f0");
}
if(count >= 1)
{
print("You Won!");
}
}
void OnTriggerEnter(Collider collide)
{
if (collide.transform.tag == "Cube")
{
count = count + 1;
Destroy (collide.gameObject);
}
}
}
Note: cube is one of the game object that needs to be picked up.
you could interrupt the game or show a victory menu or something when you have all the cubes collected
void Update ()
{
bool cubescollected = false;
if(cubescollected == 4)
{
ShowVictoryOrSomething();
cubescollected = true
}
if(cubescollected == true)
return;
... your timer code
}
good luck and happy coding

WP8 implementing gestures without previous WP7 toolkit

I've recently rebuilt an app for Windows Phone 8, but with the new Silverlight Toolkit, GestureListener is no longer present, warning: "GestureListener is not supported in a Silverlight project". I really want to implement a gesture navigation system into my app whereby the page can be swiped left or right to navigate to one of two other pages, but only after a certain "drag threshold" - this was nicely shown in WP7 here (the behaviour for deleting items I would like to apply to my MainPage) - but without the old controls I can't see a clear way to do this, after trying relentlessly. Now there are apparently only three Manipulation Events we can use, which has complicated a process that was so much easier before. I'm trying to make the whole page (ie. the first ContentPanel) moveable along the horizontal, but can't even achieve this now. Please could somebody help, in any way?
using Microsoft.Phone.Controls;
namespace PhoneApp2
{
public partial class MainPage : PhoneApplicationPage
{
double _x = 0;
double _y = 0;
double _x2 = 0;
double _y2 = 0;
public MainPage()
{
InitializeComponent();
}
private void PhoneApplicationPage_ManipulationStarted_1(object sender, System.Windows.Input.ManipulationStartedEventArgs e)
{
_x = e.ManipulationOrigin.X;
_y = e.ManipulationOrigin.Y;
}
private void PhoneApplicationPage_ManipulationCompleted_1(object sender, System.Windows.Input.ManipulationCompletedEventArgs e)
{
_x2 = e.ManipulationOrigin.X;
_y2 = e.ManipulationOrigin.Y;
string _xx = string.Format(" x:{0} y:{1} x2:{2} y2:{3}", _x, _y, _x2, _y2);
if (_y > _y2 && _y - _y2 > 100)
{
lbl1.Text = "up" + _xx;
}
else if (_x > _x2 && _x - _x2 > 100)
{
lbl1.Text = "left" + _xx;
}
else if (_y < _y2 && _y2 - _y > 100)
{
lbl1.Text = "down" + _xx;
}
else if (_x < _x2 && _x2 - _x > 100)
{
lbl1.Text = "right" + _xx;
}
}
}
}

getUserPixels - alternative in official Kinect SDK

Is there an alternative for the getUserPixels method offered by OpenNI in the official Kinect SDK?
How would one implement this functionality with the official Kinect SDK?
The official Kinect for Windows SDK (v1.6) does not support a direct call, such as getUserPixels, to extract a player silhouette but does contain all the information necessary to do so.
You can see this in action, in different ways, by examining two of the examples available from the Kinect for Windows Developer Toolkit.
Basic Interactions-WPF: includes a function to create a simple silhouette of the user being tracked.
Green Screen (-WPF, or -D2D): shows how to perform background subtraction to produce a green screen effect. In this example the data from the RGB camera is superimposed over a image.
The two examples do this in different ways.
Basic Interactions will pull out a BitmapMask of from the depth data which corresponds to the requested player. This has the advantage of only showing tracked users; any object not thought to be a skeleton is ignored.
Green Screen does not look for a particular user, instead opting for motion. This gives the advantage silhouetting any moving object -- such as a ball being passed between two users.
I believe the "Basic Interactions" example will show you how you implement what you are looking for. You'll have to do the work yourself, but it is possible. For example, using the "Basic Interactions" example as a base I created a UserControl that generates a simple silhouette of the user being tracked...
When the skeleton frame is ready, I pull out the player index:
private void OnSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
{
if (skeletonFrame != null && skeletonFrame.SkeletonArrayLength > 0)
{
if (_skeletons == null || _skeletons.Length != skeletonFrame.SkeletonArrayLength)
{
_skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];
}
skeletonFrame.CopySkeletonDataTo(_skeletons);
// grab the tracked skeleton and set the playerIndex for use pulling
// the depth data out for the silhouette.
// NOTE: this assumes only a single tracked skeleton!
this.playerIndex = -1;
for (int i = 0; i < _skeletons.Length; i++)
{
if (_skeletons[i].TrackingState != SkeletonTrackingState.NotTracked)
{
this.playerIndex = i+1;
}
}
}
}
}
Then, when the next depth frame is ready, I pull out BitmapMask for the user that corresponds to playerIndex.
private void OnDepthFrameReady(object sender, DepthImageFrameReadyEventArgs e)
{
using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
{
if (depthFrame != null)
{
// check if the format has changed.
bool haveNewFormat = this.lastImageFormat != depthFrame.Format;
if (haveNewFormat)
{
this.pixelData = new short[depthFrame.PixelDataLength];
this.depthFrame32 = new byte[depthFrame.Width * depthFrame.Height * Bgra32BytesPerPixel];
this.convertedDepthBits = new byte[this.depthFrame32.Length];
}
depthFrame.CopyPixelDataTo(this.pixelData);
for (int i16 = 0, i32 = 0; i16 < pixelData.Length && i32 < depthFrame32.Length; i16++, i32 += 4)
{
int player = pixelData[i16] & DepthImageFrame.PlayerIndexBitmask;
if (player == this.playerIndex)
{
convertedDepthBits[i32 + RedIndex] = 0x44;
convertedDepthBits[i32 + GreenIndex] = 0x23;
convertedDepthBits[i32 + BlueIndex] = 0x59;
convertedDepthBits[i32 + 3] = 0x66;
}
else if (player > 0)
{
convertedDepthBits[i32 + RedIndex] = 0xBC;
convertedDepthBits[i32 + GreenIndex] = 0xBE;
convertedDepthBits[i32 + BlueIndex] = 0xC0;
convertedDepthBits[i32 + 3] = 0x66;
}
else
{
convertedDepthBits[i32 + RedIndex] = 0x0;
convertedDepthBits[i32 + GreenIndex] = 0x0;
convertedDepthBits[i32 + BlueIndex] = 0x0;
convertedDepthBits[i32 + 3] = 0x0;
}
}
if (silhouette == null || haveNewFormat)
{
silhouette = new WriteableBitmap(
depthFrame.Width,
depthFrame.Height,
96,
96,
PixelFormats.Bgra32,
null);
SilhouetteImage.Source = silhouette;
}
silhouette.WritePixels(
new Int32Rect(0, 0, depthFrame.Width, depthFrame.Height),
convertedDepthBits,
depthFrame.Width * Bgra32BytesPerPixel,
0);
Silhouette = silhouette;
this.lastImageFormat = depthFrame.Format;
}
}
}
What I end up with is a purple silhouette of the user in a WriteableBitmap, which can be copied to an Image on the control or pulled and used elsewhere. Once you have the BitmapMask you could also map the data the color stream if you wanted a to actually see the RGB data that corresponds to that area.
You can adapt the code to simulate more closely the getUserPixels function if you like. The big part you'd be interested in would be, given a depth frame and a playerIndex:
if (depthFrame != null)
{
// check if the format has changed.
bool haveNewFormat = this.lastImageFormat != depthFrame.Format;
if (haveNewFormat)
{
this.pixelData = new short[depthFrame.PixelDataLength];
this.depthFrame32 = new byte[depthFrame.Width * depthFrame.Height * Bgra32BytesPerPixel];
this.convertedDepthBits = new byte[this.depthFrame32.Length];
}
depthFrame.CopyPixelDataTo(this.pixelData);
for (int i16 = 0, i32 = 0; i16 < pixelData.Length && i32 < depthFrame32.Length; i16++, i32 += 4)
{
int player = pixelData[i16] & DepthImageFrame.PlayerIndexBitmask;
if (player == this.playerIndex)
{
// this pixel "belongs" to the user identified in "playerIndex"
}
else
{
// not the requested user
}
}
}

MonoDevelop.Components.Docking - Tabbed DockGroupType issue

Our application uses the MonoDevelop.Components.Docking framework in our
Windows application. We last updated to the latest version in November
2010. I have come across some interesting behavior that occurs in the
following situation:
Press the auto hide button of the first panel in a
DockGroupType.Tabbed ParentGroup
Hold mouse over collapsed panel until it expands
Drag panel into center of the tabbed group (back to original
spot) and drop
At this point the panel resizes to the size of the blue rectangle that
showed where the panel would be dropped, and then undocks from the main
window to float at that size. This only happens on the first item in a
tabbed group. I found a commented out section of code in
DockGroupItem.cs (line 112, GetDockTarget(..)) that seems as though it
might deal with this. However, it references a DockPosition type that is
not defined, CenterAfter. The method is below, with the commented out
portion in bold:
public bool GetDockTarget (DockItem item, int px, int py, Gdk.Rectangle rect, out DockDelegate dockDelegate, out Gdk.Rectangle outrect)
{
dockDelegate = null;
if (item != this.item && this.item.Visible && rect.Contains (px,py)) {
int xdockMargin = (int) ((double)rect.Width * (1.0 - DockFrame.ItemDockCenterArea)) / 2;
int ydockMargin = (int) ((double)rect.Height * (1.0 - DockFrame.ItemDockCenterArea)) / 2;
DockPosition pos;
/* if (ParentGroup.Type == DockGroupType.Tabbed) {
rect = new Gdk.Rectangle (rect.X + xdockMargin, rect.Y + ydockMargin,rect.Width - xdockMargin*2, rect.Height - ydockMargin*2);
pos = DockPosition.CenterAfter;
}
*/ if (px <= rect.X + xdockMargin && ParentGroup.Type != DockGroupType.Horizontal) {
outrect = new Gdk.Rectangle (rect.X, rect.Y, xdockMargin, rect.Height);
pos = DockPosition.Left;
}
else if (px >= rect.Right - xdockMargin && ParentGroup.Type != DockGroupType.Horizontal) {
outrect = new Gdk.Rectangle (rect.Right - xdockMargin, rect.Y, xdockMargin, rect.Height);
pos = DockPosition.Right;
}
else if (py <= rect.Y + ydockMargin && ParentGroup.Type != DockGroupType.Vertical) {
outrect = new Gdk.Rectangle (rect.X, rect.Y, rect.Width, ydockMargin);
pos = DockPosition.Top;
}
else if (py >= rect.Bottom - ydockMargin && ParentGroup.Type != DockGroupType.Vertical) {
outrect = new Gdk.Rectangle (rect.X, rect.Bottom - ydockMargin, rect.Width, ydockMargin);
pos = DockPosition.Bottom;
}
else {
outrect = new Gdk.Rectangle (rect.X + xdockMargin, rect.Y + ydockMargin, rect.Width - xdockMargin*2, rect.Height - ydockMargin*2);
pos = DockPosition.Center;
}
dockDelegate = delegate (DockItem dit) {
DockGroupItem it = ParentGroup.AddObject (dit, pos, Id);
it.SetVisible (true);
ParentGroup.FocusItem (it);
};
return true;
}
outrect = Gdk.Rectangle.Zero;
return false;
}
I have tried a few small things, but nothing as affected the behavior so
far. Any ideas on what I could edit to get this working properly?
Thanks!
To fix the problem above I added a check to see if the item being docked is the same as the first item in the tab group, if so, modifies the insertion index appropriately because trying to insert the item before itself in the group causes the float problem. Since its status was "AutoHide" it is still technically visible, so was kept in the tab group's list of visible objects. Changes are below.
DockGroup.cs (line 122) - commented out the index increase:
public DockGroupItem AddObject (DockItem obj, DockPosition pos, string relItemId)
{
...
else if (pos == DockPosition.CenterBefore || pos == DockPosition.Center) {
if (type != DockGroupType.Tabbed)
gitem = Split (DockGroupType.Tabbed, pos == DockPosition.CenterBefore, obj, npos);
else {
//if (pos == DockPosition.Center) // removed to fix issue with drag/docking the 1st tab item after autohiding
//npos++;
gitem = new DockGroupItem (Frame, obj);
dockObjects.Insert (npos, gitem);
gitem.ParentGroup = this;
}
}
ResetVisibleGroups ();
return gitem;
}
DockGroup.cs (line 912) - added check for same item
internal override bool GetDockTarget (DockItem item, int px, int py, out DockDelegate dockDelegate, out Gdk.Rectangle rect)
{
if (!Allocation.Contains (px, py) || VisibleObjects.Count == 0) {
dockDelegate = null;
rect = Gdk.Rectangle.Zero;
return false;
}
if (type == DockGroupType.Tabbed) {
// this is a fix for issue with drag/docking the 1st tab item after autohiding it
int pos = 0;
if (item.Id == ((DockGroupItem)VisibleObjects[0]).Id)
{
pos++;
}
// Tabs can only contain DockGroupItems
return ((DockGroupItem)VisibleObjects[pos]).GetDockTarget (item, px, py, Allocation, out dockDelegate, out rect);
}
...