Is it possible to clear a preview window after preview from camera is done? I am using MFCaptureEngine, calling m_pPreview->SetRenderHandle(m_hwnd) to render the video. But when I stop the video I am not able to draw on the window. There remains a last frame from the camera. I need to fill the window by black brush and draw some text, but the image from the camera cannot be overdrawn.
It is not clear understand from you answer what is it MFCaptureManager, but by code SetRenderHandle(m_hwnd) I see that you use IMFCapturePreviewSink::SetRenderHandle. So, I can say that I had faced with similar problem some time ago, and it is related with difference between of the old window system which exist from WinXP and current window system from Vista. Code sets window context to the renderer by calling IMFCapturePreviewSink::SetRenderHandle - for IMFCapturePreviewSink it is DirectX11 - and DirectX11 has got FULL access to the window and it is switched to current window system. As a result, any calling fill the window by black brush and draw some text which is done by old Windows API from Win95-XP generation do nothing - because window handler context is LOCKED by DirectX11.
There are three ways for resolving of this problem:
Write the new UI by the new Microsoft DirectComposition GUI API which is based on DirectX11 and set it to IMFCapturePreviewSink::SetRenderSurface.
Create EVR Media Sink by MFCreateVideoRenderer - it creates DirectX9 video renderer which is compatible with old Windows API from Win95-XP generation, and set this IMFMediaSink in IMFCapturePreviewSink::SetCustomSink.
Create code of the video renderer on DirectX9 base - for example MFCaptureD3D/device.cpp, and draw raw IMFSample from callback IMFCapturePreviewSink::SetSampleCallback.
Regards.
I've implemented it this way:
// Sink
CComPtr<IMFCaptureSink> pSink;
m_pEngine->GetSink(MF_CAPTURE_ENGINE_SINK_TYPE_PREVIEW, &pSink);
CComPtr<IMFMediaSink> pCustomSink;
::MFCreateVideoRenderer(IID_IMFMediaSink, (void**)&pCustomSink);
CComPtr<IMFCapturePreviewSink> pPreviewSink;
pSink.QueryInterface(&pPreviewSink);
pPreviewSink->SetCustomSink(pCustomSink);
// preview
pSink.QueryInterface(&m_pPreview); // or pPreviewSink.QueryInterface(&m_pPreview)
m_pPreview->SetRenderHandle(m_hwndPreview);
But the behaviour is still the same (the screen cannot be redrawn after the preview is stopped).
Related
Presently I'm learning the basics of real-time raytracing with the DXR API in DirectX 12 Ultimate. I'm studying the D3D12 raytracing samples on the official GitHub and am using an i9/Intel Iris Xe/RTX3070 laptop and building the programs in VS2022.
Since the samples were written for Windows 10 and I'm using a hybrid graphics PC, a Debug build will run in Windows 11 after adding D3D12_MESSAGE_ID_RESOURCE_BARRIER_MISMATCHING_COMMAND_LIST_TYPE to D3D12_INFO_QUEUE_FILTER during device creation (see DirectX 12 application is crashing in Windows 11). The only trouble is that none of the sample programs change to fullscreen (i.e. borderless windowed) mode when pressing the Alt+Enter key combination. The programs always stay in windowed mode.
This hasn't worried me so far, because I've been copying the raytracing code over to a template (based on DirectX Tool Kit for Windows desktop) where fullscreen toggling works properly. In this way, I was able to run the HelloWorld and SimpleLighting samples successfully in both windowed mode and fullscreen (2560x1440 monitor resolution).
However, this hasn't been so successful for the ProceduralGeometry sample, which introduces intersection shaders. Once again, the original sample program renders the scene properly, but only in a bordered window. But when the code is reproduced in the template where I can toggle to fullscreen, the raytraced scene does not render properly.
In the scene, the triangle geometry used for the ground plane of the scene renders ok, but a translucent bounding box around the fractal pyramid is visible, and all other procedural geometry also appears translucent. Every couple of seconds, the bounding box for the metaballs also appears briefly, then vanishes.
I was able to determine that by freezing the scene, the reason for the translucency was that the following frames were being presented in sequence:
triangle ground plane quad only
floor geometry plus opaque fractal pyramid bounding box
all of the above plus opaque metaball bounding box
completed scene with opaque geometry and no bounding boxes
At the native framerate (165Hz on my machine), this results in both the procedural geometry and bounding boxes always being visible, but 'see-through' due to all the partially complete frames being presented to the display. This happens in both windowed and fullscreen modes, but it's worse in fullscreen, because the scene gets affected by random image corruption not seen in windowed mode.
I've been grappling with this issue for a few days and can't work out the problem. The only changes I've made to the sample program are the Windows 11 fix, and using a template for proper fullscreen rendering, which the original sample ignores or doesn't implement properly.
Hopefully someone can shed light on this perplexing issue!
I found the problem. Each sample has a header file called DXSampleHelper.h. For the ProceduralGeometry sample, this header file was updated with a helper class to manage structured buffers, which is very similar to the helper class for constant buffers.
The CopyStagingToGpu() method, which consists of a one line memcpy operation in both classes, is slightly different for the structured buffer class:
memcpy(m_mappedBuffers + instanceIndex * NumElementsPerInstance(), &m_staging[0], InstanceSize());
The same method in the constant buffer class is:
memcpy(m_mappedBuffers + instanceIndex, &m_staging[0], InstanceSize());
I.e. I was missing instanceIndex * NumElementsPerInstance() and thus the procedural geometry instances within the structured buffer were not correctly aligned in GPU memory.
When screen sharing a specific window on macOS with Zoom or Skype/Teams, they draw a red or green highlight border around that window (which belongs to a different application) to indicate it is being shared. The border is following the target window in real time, with resizing, z-order changes etc.
See example:
What macOS APIs and techniques might be used to achieve this effect?
You can find the location of windows using CGWindowListCopyWindowInfo and related API, which is available to Sandboxed apps.
This is a very fast and efficient API, fast enough to be polled. The SonOfGrab sample code is great platform to try out this stuff.
You can also install a global event tap using +[NSEvent addGlobalMonitorForEventsMatchingMask:handler:] (available in sandbox) to track mouse down, drag and mouse up events and then you can respond immediately whenever the user starts or releases a drag. This way your response will be snappy.
(Drawing a border would be done by creating your own transparent window, slightly larger than, and at the same window layer as, the window you are tracking. And then simply draw a pretty green box into it. I'm not exactly sure about setting the z-order. The details of this part would be best as a separate question.)
I am writing a MacOS or OSX application where I need to record only the View of my application (Not the Whole display) with the Audio it emits.
Think it as a game app and I need to record the complete GamePlay View Of the Application.How should I go about doing this?
I am aware of "AVCaptureScreenInput" and, the example. But how to capture only the view of my application?
From the website you posted:
Note: By default, AVCaptureScreenInput captures the entire screen. You may set its cropRect property to limit the capture rectangle to a subsection of the screen.
Just set this property to the windows/views rect and you're done
Of course you need to update and restart the recording when the windows/views rect changes.
Read the document carefully, there is a comment, about the displays:
// If you're on a multi-display system and you want to capture a secondary display,
// you can call CGGetActiveDisplayList() to get the list of all active displays.
// For this example, we just specify the main display.
// To capture both a main and secondary display at the same time, use two active
// capture sessions, one for each display. On Mac OS X, AVCaptureMovieFileOutput
// only supports writing to a single video track.
I like to make an overlay with the following properties:
should work at least on Windows 8.1
should be on top on everything, like a mouse cursor
should incorporate the pixels which are already on the background, like a blur filter
no flickering
Details to each of this points:
1) I assume that WDM is activated and DirectX 11.2 is used. Sure it would be nice to have it working on other Windows versions but this has no priority.
2) The problem is that with simply using the WS_EX_TOPMOST, menus from applications are over my overlay. In my case this really hurts as I like to display something with the same properties as a cursor. Imagine that a cursor suddenly is hidden if you open a menu -> unacceptable.
3) I like to read the pixels from the Windows desktop, including any effect Windows applies (like blur), and use this information for my filter. If I add my overlay, as described in 2, I should be able to get a fresh unobstructed copy of the background in the next frame and not read out my own overlay.
4) If I just write something into the Windows desktop directly, it gets overwritten immediately on the next frame by Windows itself. This is not acceptable.
One example of such an application is a magnifying glass, which exactly has all the properties I need. But for this case Windows 8.1 has an API. In contrast I like to write a program which displays a hand on the desktop (which is controlled by Leap Motion) which influences the Windows desktop, so you almost "feel" how you move your hand over the desktop.
If I write a tiny DirectX and/or OpenGL application for myself this is all very easy:
render all the regular stuff to a texture
use this texture for a post processing filter and add all my stuff on top of it
render just a quad to the back buffer
But I like to do that for the whole Windows desktop.
I found many different application, but they are to no use for me:
application which claim to be on top, are still behind menus. This normally doesn't really hurt, but is unacceptable for a cursor-alike thing
screen capturing programs which hook them self in all running programs are nice, but I want to hook myself into WDM
normally screen capturing programs do not draw anything into the back buffer, so they get every frame a new unobstructed back buffer
My questions can be boiled down to: How can I write my own magnifying glass for Windows 8.1.
I fear that my only serious option is to hook myself into WDM, what I try to avoid.
I'm happy to hear any idea how to achieve this, or hints to application which are doing what I describe.
I have a C++ DirectX-based third-party game engine compiled into a Windows Phone Runtime Component DLL. I'm working on integrating it into a project based off of a Windows Phone Direct3D with XAML App. The game engine DLL uses the the D3D device, context and render view texture provided by the application's Direct3DBackground::Draw() method.
The built-in renderer from the sample is gone and replaced by the game engine's.
I can render but there is constant black flickering. Every other frame is black. To prove to myself that it wasn't the renderer (which has been proven to work elsewhere), I cut out all the rendering code from the game engine DLL to a simply setting a clear color. The result is still the same.
At first I thought it was because the Direct3DXamlAppComponent generated by the sample was maybe running in a different thread from the game engine DLL, but that's not the case. They're on the same thread.
What rendering problem could this configuration be causing?
Does the game engine's renderer need a separate d3d device?
Does the game engine's renderer need a separate d3d device context?
Things I haven't tried yet:
creating a second d3d device on the DLL
converting the game engine to provide its own IDrawingSurfaceManipulationHandler. But I'm not sure if it'll just have the same problem as above.
The problem came from the render target view. I didn't realize that the pointer to it gets updated every frame. I had just set it to the game engine renderer once at start up. Now I update the render view target pointer every frame and now the black flickers are gone.