Joint tracking image on Skeletal Tracking not following correctly - kinect

When I run the Channel9 MSDN Kinect quickstart series code for skeletal tracking, there are suppose to be images/ellipses that overlap the location of the joints selected. Instead I get images/ellipses that are slightly off and not exactly over the location of the joints. No matter where I move the joints, the images stay off to the side of the exact location of the joint. I have seen the video here (http://channel9.msdn.com/Series/KinectQuickstart/Skeletal-Tracking-Fundamentals) and theirs works. Is there something I did wrong that I am not noticing that I changed or forgot to update?
Update:
This is some line of code that might be causing it:
CoordinateMapper cm = new CoordinateMapper(kinectSensorChooser1.Kinect);
//Map a joint location to a point on the depth map
//head
DepthImagePoint headDepthPoint =
cm.MapSkeletonPointToDepthPoint(first.Joints[JointType.Head].Position, depth.Format);
This is a change over what is in the video as they updated with a new method to map skeleton to depth and depth to color point. It eliminates an error that occurred when the stream was terminated.
The method has parameters that ask for position and format. Could it be that the depth.Format is not the same format as the RGB640x480Resolution?
//Map a depth point to a point on the color image
//head
ColorImagePoint headColorPoint =
cm.MapDepthPointToColorPoint(depth.Format, headDepthPoint,
ColorImageFormat.RgbResolution640x480Fps30);
On the main window screen this is what I have for creating the images i am using:
<Canvas Name="MainCanvas">
<my:KinectColorViewer Canvas.Left="50" Canvas.Top="50" Height="480" Name="kinectColorViewer1" Width="640"
Kinect="{Binding ElementName=kinectSensorChooser1, Path=Kinect}" />
<Ellipse Canvas.Left="0" Canvas.Top="0" Height="50" Name="leftEllipse" Width="50" Fill="#FF4D298D" Opacity="1" Stroke="White"></Ellipse>
<Ellipse Canvas.Left="100" Canvas.Top="0" Height="50" Name="rightEllipse" Stroke="White" Width="50" Fill="#FF2CACE3" Opacity="1" />
<my:KinectSensorChooser Canvas.Left="250" Canvas.Top="380" Name="kinectSensorChooser1" Width="328" />
<Ellipse Canvas.Left="225" Canvas.Top="84" Height="50" Name="headImage" Stroke="White" Width="50" Fill="#FF2CACE3" Opacity="1" />
<my:KinectSkeletonViewer Canvas.Left="646" Canvas.Top="240" Name="kinectSkeletonViewer1" Width="640" Height="480" Kinect="{Binding ElementName=kinectSensorChooser1, Path=Kinect}" />
</Canvas>
And I have commented these lines of code about scaling position
//ScalePosition(headImage, first.Joints[JointType.Head]);
// ScalePosition(leftEllipse, first.Joints[JointType.HandLeft]);
//ScalePosition(rightEllipse, first.Joints[JointType.HandRight]);
and this function has been commented.
private void ScalePosition(FrameworkElement element, Joint joint)
{
//convert the value to X/Y
//Joint scaledJoint = joint.ScaleTo(1280, 720);
//convert & scale (.3 = means 1/3 of joint distance)
// Joint scaledJoint = joint.ScaleTo(1280, 720, .3f, .3f);
// Canvas.SetLeft(element, scaledJoint.Position.X);
// Canvas.SetTop(element, scaledJoint.Position.Y);
}
and this is commented out so no transform parameters are used:
//sensor.SkeletonStream.Enable(parameters);
and this is what stream and skeleton enable lines of code look like:
sensor.SkeletonStream.Enable();
sensor.AllFramesReady += new EventHandler<AllFramesReadyEventArgs>(sensor_AllFramesReady);
sensor.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30);
sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);

Try the following steps.
Comment the lines that scaleTo() for testing porposes.
You initialized kinect with 640x480? Try to put this sizes to your my:KinectSkeletonViewer element on the XAML. I see that actualy it is Width="320" Height="240", change it.
Also check to not use strech on your Image element or in the Canvas that hosts it, for security, delete the Height="Auto" and Width="Auto" from your MainCanvas.
Start your skeleton stream with no TransformSmoothParameter or, if you want to use this, try the following parameters
this.KinectSensorManager.TransformSmoothParameters = new TransformSmoothParameters
{
Smoothing = 0.5f,
Correction = 0.5f,
Prediction = 0.5f,
JitterRadius = 0.05f,
MaxDeviationRadius = 0.04f
};
I am do the same you are trying to do using kinect for xbox and it worked fine to me.
When the channel9 video was recorded the sdk did not have the CoordinateMapper class, so, you dont need to convert to depth then to color, the coordinate mapper has a method to directly map the skeleton point to the color point, the method is MapSkeletonPointToColorPoint(), use the method and dont worry about the Depth Data.

It looks the problem I had was in the xaml code. I replaced the xaml code I had and replaced it with the original xaml code for the images and objects on my canvas.
<my:KinectColorViewer Canvas.Left="0" Canvas.Top="0" Width="640" Height="480" Name="kinectColorViewer1"
Kinect="{Binding ElementName=kinectSensorChooser1, Path=Kinect}" />
<Ellipse Canvas.Left="0" Canvas.Top="0" Height="50" Name="leftEllipse" Width="50" Fill="#FF4D298D" Opacity="1" Stroke="White" />
<Ellipse Canvas.Left="100" Canvas.Top="0" Fill="#FF2CACE3" Height="50" Name="rightEllipse" Width="50" Opacity="1" Stroke="White" />
<my:KinectSensorChooser Canvas.Left="250" Canvas.Top="380" Name="kinectSensorChooser1" Width="328" />
<Image Canvas.Left="66" Canvas.Top="90" Height="87" Name="headImage" Stretch="Fill" Width="84" Source="/SkeletalTracking;component/c4f-color.png" />
It might been have been a property or event setting that I had set incorrectly or not at all that introduced a shift in the image objects.

Related

Xamarin Forms custom frame shape around Image

I have an image in my app with a frame around it to make the image a circle. I want a custom shape for the Frame, but can't figure out how to get this exact shape.
(Please ignore the house inside the drawing, it is simply there to represent an Image in Xamarin, I only need the outline to be the shape).
I also don't require the Frame to have a colored border, only the shape.
Any help will be appreciated!
Here is my code right now for the Frame + Image:
<Frame x:Name="PictureFrame" Padding="0" Margin="0" HorizontalOptions="Center" BorderColor="Transparent" WidthRequest="80" HeightRequest="80" HasShadow="False">
<Image x:Name="Picture" Source="picture" Margin="0,0,0,0" Aspect="AspectFill"/>
</Frame>
You probably have to create a custom renderer to achieve what you're looking for. For example, on Android the Frame mask is actually implemented with a bezier path that you could control to your liking if you dive a bit deeper into the code.
Take a look at the FreshEssentials AdvancedFrame view which almost has what you need. The modification they've made is that you can have a sharp 90 degree angle on either both corners on the left or on the right. You'd simply have to modify this code a little bit to have a rounded corner on all except the top left corner.
What you need to modify are the following:
AdvancedFrame (In case you want the corner radiuses to be configurable separately or something else. This is the base that inherits from the original Frame control in Xamarin.Forms)
AdvancedFrameRendererDroid
AdvancedFrameRendereriOS
AdvancedFrameRendererUWP (Only if you require UWP support)
Tips:
On Android, the background mask for the frame is defined with the Path class.
On iOS, UIBezierPath class from the UIKit framework is used.
Now with Xamarin.Forms you can use Shapes. The Path Data is your circle with point.
<AbsoluteLayout
BackgroundColor="AliceBlue"
>
<Path
AbsoluteLayout.LayoutBounds="0,0,1,1"
AbsoluteLayout.LayoutFlags="All"
Margin="8"
Data="M142.798828,0 C221.365114,0.164668482 285,63.9009723 285,142.5 C285,221.200577 221.200577,285 142.5,285 C63.7994232,285 0,221.200577 0,142.5 C0,139.784357 0.0759637392,137.086456 0.225882227,134.408306 L0,133.828125 L0,0 L142.798828,0 Z"
Stroke="LightBlue"
StrokeThickness="2"
HorizontalOptions="Center"
VerticalOptions="Center"
/>
<Image
AbsoluteLayout.LayoutBounds="0,0,1,1"
AbsoluteLayout.LayoutFlags="All"
Aspect="AspectFit"
Source="brigadeiro"
HorizontalOptions="Center"
VerticalOptions="Center"
WidthRequest="94"
HeightRequest="94"
/>
</AbsoluteLayout>

How to set a background to a textBlock that changes its content

Background
I'm trying to put a textBlock control at the bottom of the screen (with a small margin below it), and I also wish to set a background for it, so that no matter what is shown behind the textBlock, it will be easy to read.
On Android, you could simply set the background to it, and tell it to have the width and height to be WRAP_CONTENT, so that it will take only the space it needs, but I can't find a similar thing on WP8.
Current status
This is the xaml I've created:
...
<Grid >
<Image x:Name="fullScreenImage" Stretch="Fill"
Visibility="Collapsed" />
<TextBlock HorizontalAlignment="Center" VerticalAlignment="Bottom"
Margin="0,0,0,200" FontSize="40" x:Name="pictureLabel" TextWrapping="Wrap"
Foreground="#ff000000" />
</Grid>
The problem
Since the textBlock doesn't have a background property, I had to use something that wraps it. However, since its content changes dynamically, I can't simply set a size for it.
The question
For now, I would like to simply set its background color.
I would also very appreciate if it would be possible to use a rounded corners rectangle for the background, or a 9-patch image.
How can I achieve setting a background for the textBlock?
The solution is very simple. Just set the HirizontalAlignment to Left. Hope this will work in your case.
<StackPanel HorizontalAlignment="Left">
<Border Background="#66FFFFFF">
<TextBlock/>
</Border>
</StackPanel>
According to this question (which is for Silverlight - but is xaml nonetheless), there is no way to explicitly set a background color for a TextBlock. Your best bet is to wrap your TextBlock in a Grid or Border.
If a Grid does not work, this article suggests a Border will do the trick:
A simple border will do, and by not setting its width and height
properties, it will shrink/grow based on the size of the TextBlock.
I've come up with the next solution , which works quite fine , but what i really would like to also have is a way to set a min font and max font size , so that if there is a single word , the font might need to be of some size, and if the text is too long , the font will be of a smaller size , all in a dynamic way.
code:
label.Text = label;
label.Measure(new Size(RenderSize.Width, RenderSize.Height));
border.Width = label.DesiredSize.Width + border.Padding.Left + border.Padding.Right + border.BorderThickness.Left + border.BorderThickness.Right;
border.Height = label.DesiredSize.Height + border.Padding.Top + border.Padding.Bottom + border.BorderThickness.Bottom + border.BorderThickness.Top;
and the xaml:
<Border BorderBrush="#ff000000" BorderThickness="2" CornerRadius="8" Visibility="Collapsed" Padding="5" Background="#bfff0000" Margin="10,0,10,200" VerticalAlignment="Bottom" x:Name="border">
<StackPanel>
<TextBlock FontSize="40" x:Name="pictureLabel" TextWrapping="Wrap" Foreground="#ff000000" />
</StackPanel>
</Border>

How to centre an item over a point in a Canvas?

I'm using a ListBox with its DataTemplate containing a Canvas. I then bind the Left/Top of the Grid containing that Canvas to move it to a certain point.
I want to then have the child Grid centred at the X,Y coordinates I've specified, where the size of the child Grid is variable based on its content. I was planning on achieving this by using a TranslateTransform to move the Grid by half of its width.
I can't see how I can set that TranslateTransform however as ElementName binding doesn't work within a DataTemplate. Any ideas how I can achieve this?
<ItemsControl ItemsSource="{TemplateBinding SomeCollection}">
<ItemsControl.ItemsPanel>
<ItemsPanelTemplate>
<Grid HorizontalAlignment="Stretch" VerticalAlignment="Stretch" />
</ItemsPanelTemplate>
</ItemsControl.ItemsPanel>
<ItemsControl.ItemTemplate>
<DataTemplate>
<Canvas>
<Grid x:Name="Container"
Canvas.Left="{Binding X}"
Canvas.Top="{Binding Y}"
Background="#88000000">
<Grid.RenderTransform>
<TranslateTransform X="{Binding ActualWidth, ElementName=Container, Converter={StaticResource NegativeHalfConverter}}"
Y="{Binding ActualHeight, ElementName=Container, Converter={StaticResource NegativeHalfConverter}}" />
</Grid.RenderTransform>
<TextBlock Text="{Binding SomeValue}" FontSize="36" Foreground="White" />
</Grid>
</Canvas>
</DataTemplate>
</ItemsControl.ItemTemplate>
`
I think ElementName binding should work in the name scope of a DataTemplate, but I have seen people complaining about bindings not updating correctly when binding to ActualWidth/Height properties. Perhaps instead of doing the complicated setup you have you could just implement an attached behavior that takes a Point parameter and updates the Canvas.Left/Top properties whenever the parameter or size of the associated object (your grid) changes.
Looks like the fault wasn't with the binding itself, but with a feature that means the ActualWidth/ActualHeight properties aren't bindable. Thanks Filip.
To fix this I created a derived Grid with a couple of new dependency properties that I update in the SizeChanged events to have the ActualWidth/Height. I then use my DataTemplate as above, binding to these new DPs to translate and centre my Grid on a point. Seems to work a treat.
To move an object by half of its size you can use 2 rotations or scales: first is over 0.25,0.25 relative point, second is over the enter point 0.5,0.5. If you use rotations then angels are 180 degrees and -180 degrees. If you use scales then scale factors are -1,-1 and -1,-1.
Do not forget about RenderTransformOrigin property. And to apply two transforms you can apply them to two nested elements.

Windows 8 Image UniformFill centered

I have a little problem, I have a group item which I want to give a background image which it should scale keeping it's correct dimensions but by default it shows the image from the top left, I want the image to be centered.
Here is an illustration to further explain my problem. (the gray part is what is clipped)
And I have this XAML
<Image Source="/url/to/image.jpg" Stretch="UniformToFill"/>
I managed to solve my problem, I made the image larger than the container it was placed in and vertical aligned it in the center.
<Grid HorizontalAlignment="Left" Width="250" Height="125">
<Image Source="/url/to/image.jpg" Stretch="UniformToFill" Height="250" Margin="0" VerticalAlignment="Center"/>
</Grid>
The overflow of the image was invisible :)
In case the size of the source image is unknown in advance I had to use different trick:
<Border Width="250" Height="250">
<Border.Background>
<ImageBrush ImageSource="/url/to/image.jpg"
Stretch="UniformToFill"/>
</Border.Background>
</Border>
I wrote a behavior for Silverlight/Windows Phone that treats a similar situation. The picture I have to show may be larger or higher and I must display it in a square.
The behavior calculates the width/height ratio of both the container and the picture. Depending on which is the largest/highest, I resize the Image control to have this clipping effect with the parent control.
Here is a sample XAML to use with my behavior.
<Border Height="150" Width="150">
<Image Source="{Binding BitmapImage}" Stretch="UniformToFill"
HorizontalAlignment="Center" VerticalAlignment="Center">
<i:Interaction.Behaviors>
<b:FillParentBehavior />
</i:Interaction.Behaviors>
</Image>
</Border>
Here is an excerpt from the C# code. The full code can be found here: FillParentBehavior.cs
double width = this.AssociatedObject.Width;
double height = this.AssociatedObject.Height;
var parentSize = new Size(this.parent.ActualWidth, this.parent.ActualHeight);
var parentRatio = parentSize.Width / parentSize.Height;
// determine optimal size
if (this.AssociatedObject is Image)
{
var image = (Image)this.AssociatedObject;
if (image.Source is BitmapImage)
{
var bitmap = (BitmapImage)image.Source;
var imageSize = new Size(bitmap.PixelWidth, bitmap.PixelHeight);
var imageRatio = imageSize.Width / imageSize.Height;
if (parentRatio <= imageRatio)
{
// picture has a greater width than necessary
// use its height
width = double.NaN;
height = parentSize.Height;
}
else
{
// picture has a greater height than necessary
// use its width
width = parentSize.Width;
height = double.NaN;
}
}
}
this.AssociatedObject.Width = width;
this.AssociatedObject.Height = height;

Augmented reality with kinect

I am doing one project on kinect with windows SDK. I am trying to display the image on skeleton shoulder with the following code.
**
Xaml code
<Canvas Margin="250,0,0,0" Visibility="{Binding Path=Showcamera, Converter={StaticResource BVC}}" Background="Green" HorizontalAlignment="Stretch" VerticalAlignment="Top" Grid.Column="1" Grid.Row="1">
<Image MaxHeight="600" MaxWidth="950" x:Name="PART_KinectVideo" Visibility="{Binding Path=Showcamera, Converter={StaticResource BVC}}" />
<Canvas>
<Image RenderOptions.BitmapScalingMode="HighQuality" Visibility="{Binding Path=Showcamera, Converter={StaticResource BVC}}"
RenderOptions.EdgeMode="Aliased" Canvas.Top="60" Canvas.Left="350" Stretch="Uniform" x:Name="Jewel"/>
</Canvas>
Code Behind
var j1p = nui.MapSkeletonPointToDepth(closestSkeleton.Joints[JointType.ShoulderLeft].Position,DepthImageFormat.Resolution640x480Fps30);
Canvas.SetLeft(Jewel, j1p.X+90);
Canvas.SetTop(Jewel, j1p.Y-280);
by getting the j1p values i am trying to set the left and top position of the canvas for the Jewel image object.
Problem with the code is :
This is not correctly placing the image. When the user moves in the screen the image also moves out of body with left and right direction as user moves.
I need the image should be set to the position even when the user moves right or left.
How should I alter the code or is there any sample available to set the image in particular position of the kinect joints?
Maybe try this:
var left = (double)image.GetValue(Canvas.LeftProperty)
left += 90;
image.SetValue(Canvas.LeftProperty, left);
I wrote an app using that code and everything worked fine to me:)