I am creating windows 8 store application with MonoGame framework. I want to get each bitmap from camera in order to process some image recognition on that bitmap. The thing is I only get the whole video stream from camera (randomAccessStream) but not each frame from the video.
async private void Start_Click(object sender, RoutedEventArgs e)
{
//1. Initialize:
mediaCaptureMgr = new MediaCapture();
randomAccessStream = new InMemoryRandomAccessStream();
await mediaCaptureMgr.InitializeAsync();
//2. create profile
MediaEncodingProfile encordingProfile = MediaEncodingProfile.CreateWmv(VideoEncodingQuality.Auto);
//3. start recording
await mediaCaptureMgr.StartRecordToStreamAsync(encordingProfile, randomAccessStream);
}
How can I receive new up-coming frame/bitmap from camera?
Take a look here
It is a sample app from Microsoft. Among other things it shows:
How to capture an image using the new LowLagPhotoCapture and LowLagPhotoControl classes.
How to capture a sequence of photos using the new LowLagPhotoSequenceCapture and LowLagPhotoSequenceControl classes.
...
If you are going to process the images, you could just take pictures on an interval. I don't think you need to process every single frame of a video stream..
Related
I am developing an UWP application with a camera feature, I have studied this sets of codes,
https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CameraStarterKit
And successfully developed a camera feature on my application. However, in my application, I wish to not store the picture into my local machine as the application is actually like a kiosk system whereby everyone will be using the same machine to take the picture.
What I am planning to do is actually to allow users to send the picture that they have taken to their own email address via the kiosk system. When they have taken a photo, a preview will be shown, and only if the user want to send the picture, then will the picture be "save"
The codes for my take photo function is something like this:
rivate async Task TakePhotoAsync()
{
var stream = new InMemoryRandomAccessStream();
Debug.WriteLine("Taking photo...");
await _mediaCapture.CapturePhotoToStreamAsync(ImageEncodingProperties.CreateJpeg(), stream);
try
{
var file = await _captureFolder.CreateFileAsync("SimplePhoto.jpg", CreationCollisionOption.GenerateUniqueName);
Debug.WriteLine("Photo taken! Saving to " + file.Path);
var photoOrientation = CameraRotationHelper.ConvertSimpleOrientationToPhotoOrientation(_rotationHelper.GetCameraCaptureOrientation());
await ReencodeAndSavePhotoAsync(stream, file, photoOrientation);
Debug.WriteLine("Photo saved!");
}
catch (Exception ex)
{
// File I/O errors are reported as exceptions
Debug.WriteLine("Exception when taking a photo: " + ex.ToString());
}
}
And to get the preview of the picture will be:
private async Task GetPreviewFrameAsSoftwareBitmapAsync()
{
// Get information about the preview
var previewProperties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;
// Create the video frame to request a SoftwareBitmap preview frame
var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, (int)previewProperties.Width, (int)previewProperties.Height);
// Capture the preview frame
using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync(videoFrame))
{
// Collect the resulting frame
SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;
// Show the frame information
FrameInfoTextBlock.Text = String.Format("{0}x{1} {2}", previewFrame.PixelWidth, previewFrame.PixelHeight, previewFrame.BitmapPixelFormat);
// Create a SoftwareBitmapSource to display the SoftwareBitmap to the user
var sbSource = new SoftwareBitmapSource();
await sbSource.SetBitmapAsync(previewFrame);
// Display it in the Image control
PreviewFrameImage.Source = sbSource;
}
}
There are (at least) two ways:
1) "Better" one:
Create a backend that sends email using SmtpClient
Post your image from UWP to backend using HttpClient
2) "Easier" one: launch Mail app
The downside is that users will be able to edit the message, receiver and things like that.
Also you can try to use SmtpClient directly from UWP client (may be now with .Net Standard 2.0 it would be possible but I guess no one yet tried) or third-party open-source replacement (which might be a bit outdated as it was created for Windows 8) or even try to launch your backend as Windows Service on the client machine.
Instead of creating a Windows built-in camera UI (CameraCaptureUI) or using a custom MediaCapture control and capture picture I want to open any Camera App downloaded in the device to capture an image and get the result.
I have used
string uriToLaunch = "microsoft.windows.camera:";
var uri = new Uri(uriToLaunch);
var success = await Windows.System.Launcher.LaunchUriAsync(uri);
But this just opens the camera app, I need to get the result file back to the app and save it.
Is there a way to do this?
The method you are using:
var success = await Windows.System.Launcher.LaunchUriAsync(uri);
just opens the default camera, nothing more, the result is a boolean with information if the application has been opened successfully, nothing more.
With CameraCaptureUI you don't need to create camera - this seems to be designed for the task like you have described. With lines:
var captureUI = new CameraCaptureUI();
captureUI.PhotoSettings.Format = CameraCaptureUIPhotoFormat.Jpeg;
captureUI.PhotoSettings.CroppedSizeInPixels = new Size(200, 200);
var photo = await captureUI.CaptureFileAsync(CameraCaptureUIMode.Photo);
you just launch the camera app and your app waits for the photo, which you can process further/save.
If you don't want to use it or implement own camera capture, you can think of sharing a picture taken by other app. This is described well at app-to-app communication at MSDN. In this case user will have to click Share button and choose your app as a target. That will invoke OnShareTargetActivated event where you can process the received content.
I am using MediaCapture to capture photos and store them. It works in the emulator. But when running the app on a real phone (Nokia Lumia 530) the captured photos are just black. They have a correct size and the file has a certain byte length, but when displaying the photo it's black. Please note that I do not use Silverlight and am fixed on using MediaCapture. The camera on the phone works when using the default camera app. The App's manifest includes the capabilities "Pictures Library" and "Webcam".
Does someone know what could be wrong?
Here is the test code I use:
using (var mediaCapture = new MediaCapture())
{
await mediaCapture.InitializeAsync();
ImageEncodingProperties imageFormat = ImageEncodingProperties.CreateJpeg();
StorageFile photoFile = await KnownFolders.PicturesLibrary.CreateFileAsync("TestPhoto.jpg", CreationCollisionOption.GenerateUniqueName);
await mediaCapture.CapturePhotoToStorageFileAsync(imageFormat, photoFile);
BitmapImage bitmap = new BitmapImage();
using (var photoStream = await photoFile.OpenReadAsync())
{
bitmap.SetSource(photoStream);
}
}
Edit
I found a solution. The photo is captured correctly if we have a CaptureElement, set it's source to the MediaCapture object, invoke MediaCapture.StartPreviewAsync before taking the photo, take the photo (using CapturePhotoToStorageFileAsync) and finally invoke StopPreviewAsync. It seems that MediaCapture needs an existing (and displayed) preview to be able to capture photos. Strange that this is not documented and using CapturePhotoToStorageFileAsync without a preview does not throw an Exception.
The photo is captured correctly if we have a CaptureElement, set it's source to the MediaCapture object, invoke MediaCapture.StartPreviewAsync before taking the photo, take the photo (using CapturePhotoToStorageFileAsync) and finally invoke StopPreviewAsync. It seems that MediaCapture needs an existing (and displayed) preview to be able to capture photos. Strange that this is not documented and using CapturePhotoToStorageFileAsync without a preview does not throw an Exception.
I've been told that when I allow the user to create a video from within my app, a .jpg thumbnail image is supposed to be generated automatically. However, using windows phone power tools, I can see that only the video is generated, no image. Here is the code below:
EDIT: the QUESTION here, is HOW does one GET a thumbnail image from a given video that is saved in isolated storage?
' Set recording state: start recording.
Private Async Sub StartVideoRecording()
Try
App.ViewModel.IsDataLoaded = False
'isoStore = Await ApplicationData.Current.LocalFolder.GetFolderAsync("IsolatedStore")
strVideoName = GenerateVideoName()
isoFile = Await isoVideoFolder.CreateFileAsync(strVideoName, CreationCollisionOption.ReplaceExisting)
thisAccessStream = Await isoFile.OpenAsync(FileAccessMode.ReadWrite)
Await avDevice.StartRecordingToStreamAsync(thisAccessStream)
'save the name of the video file into the list of video files in isolated storage settings
Dim videoList As New List(Of InfoViewModel)
isoSettings = IsolatedStorageSettings.ApplicationSettings
If isoSettings.Contains("ListOfVideos") Then
videoList = isoSettings("ListOfVideos")
Else
isoSettings.Add("ListOfVideos", videoList)
End If
videoList.Add(New InfoViewModel With {.Name = strVideoName, .DateRecorded = Date.Now})
isoSettings("ListOfVideos") = videoList
isoSettings.Save()
isoSettings = Nothing
' Set the button states and the message.
UpdateUI(ButtonState.Recording, "Recording...")
Catch e As Exception
' If recording fails, display an error.
Me.Dispatcher.BeginInvoke(Sub() txtDebug.Text = "ERROR: " & e.Message.ToString())
End Try
End Sub
In Windows Phone 7.0 it wasn't possible using the "official" SDK to access the camera other than using CameraCaptureTask. But using Microsoft.Phone.InteropServices library it was possible to create a custom camera view in the application. If you used this VideoCamera class, it had an event called ThumbnailSavedToDisk, which might be what you have heard about. But apps using Microsoft.Phone.InteropServices was never allowed in the Windows Phone Marketplace. You can read more about it here.
To generate a thumbnail from a video file one solution would be to read up on how the MP4 file format works and extract one of the frames and use that as a thumbnail. Unfortunatly I haven't found any libraries that can do this.
Another, but far from optimal solution, would be to play the video in a MediaElement and then render that Control to a bitmap. This can be done with the following code (written in XAML/C#, I don't know VB.net, but should be easy to port):
<MediaElement
x:Name="VideoPlayer"
Width="640"
Height="480"
AutoPlay="True"
RenderTransformOrigin="0.5, 0.5"
VerticalAlignment="Center"
HorizontalAlignment="Center"
Stretch="Fill"
Canvas.Left="80"/>
And then to create the thumbnail:
// Set an event handler for when video has loaded
VideoPlayer.MediaOpened += VideoPlayer_MediaOpened;
// Read video from storage
IsolatedStorageFileStream videoFile = new IsolatedStorageFileStream("MyVideo.mp4", FileMode.Open, FileAccess.Read, IsolatedStorageFile.GetUserStoreForApplication());
VideoPlayer.SetSource(videoFile);
// Start playback
VideoPlayer.Play();
And then create the event handler which "captures" the thumbnail:
void VideoPlayer_MediaOpened(object sender, RoutedEventArgs e)
{
Thread.Sleep(100); // Wait for a short time to avoid black frame
WriteableBitmap wb = new WriteableBitmap(VideoPlayer, new TranslateTransform());
thumbnailImageHolder.Source = wb; // Setting it to a Image in the XAML code to see it
VideoPlayer.Stop();
}
Here is how it looks when using a modified version of the Video Recorder Sample. (The small image with the blue border in the upper right corner is the thumbnail of the recorded video. Behind is the camera Viewer.)
I'm trying to write a program that detects a circle when you hold it in front of the webcam. I know how the circle detection works for an image, but I can't figure out how to get it to work with a webcam stream, using the following code:
static class Program
{
/// <summary>
/// The main entry point for the application.
/// </summary>
[STAThread]
static void Main()
{
ImageViewer viewer = new ImageViewer(); //create an image viewer
Capture capture = new Capture(); //create a camera capture
Application.Idle += new EventHandler(delegate(object sender, EventArgs e)
{ //run this until application closed (close button click on image viewer)
Image<Bgr, Byte> image = capture.QueryFrame();
//MemStorage storage = new MemStorage();
//Contour<Point> contours = image.FindContours();
//Contour<Point> currentContour = contours.ApproxPoly(contours.Perimeter * 0.05, storage);
viewer.Image = image; //draw the image obtained from camera
});
viewer.ShowDialog(); //show the image viewer
}
As you can see I've tried using FindContours in the innermost loop but the program just freezes when I try running it, so I commented that particular part out. Can anyone tell me how to implement circle detection using a webcam?
You can use HoughCircle Method for circle detection
Image gray = img.Convert().PyrDown().PyrUp();
Gray cannyThreshold = new Gray(180);
Gray cannyThresholdLinking = new Gray(120);
Gray circleAccumulatorThreshold = new Gray(120);
CircleF[] circles = gray.HoughCircles(
cannyThreshold,
circleAccumulatorThreshold,
5.0, //Resolution of the accumulator used to detect centers of the circles
10.0, //min distance
5, //min radius
0 //max radius
)[0]; //Get the circles from the first channel
See Method HoughCircle