I am studying about Qt Quick 2 in Qt 5.1. It is cool. However, I do not get it about Scene Graph feature. If I just create Flickable, put a bunch of Images to it, and show this in QQuickView on embedded device with EGLFS backend - am I using the Scene Graph or not?
If yes, why do I see tearing - Scene Graph should be VSynced?
If not - what should I do to use it with Flickable? Or does it mean I should implement all from scratch?
My app is basically allowing to browse an endless generated image.
QtQuick2 has as its only backend the scene graph. So yes, you're using it. Whether that's going to be vsynced, it's another story -- mostly depends whether the scene graph is using a render thread or not, and the quality of the drivers (i.e. if swapping buffers will wait for the vsync). You can try
to run your app with the environment variable QSG_RENDER_TIMING set to a non-zero value; this will print frame statistics for your application. (See here for more info.) If your frame lengths are not an exact multiple of the vsync interval, then you're not having vsync
to run your app with the environment variable QT_QPA_EGLFS_FORCEVSYNC, which will try to use the FBIO_WAITFORVSYNC ioctl to attempt to sync with the vsync.
Related
Presently I'm learning the basics of real-time raytracing with the DXR API in DirectX 12 Ultimate. I'm studying the D3D12 raytracing samples on the official GitHub and am using an i9/Intel Iris Xe/RTX3070 laptop and building the programs in VS2022.
Since the samples were written for Windows 10 and I'm using a hybrid graphics PC, a Debug build will run in Windows 11 after adding D3D12_MESSAGE_ID_RESOURCE_BARRIER_MISMATCHING_COMMAND_LIST_TYPE to D3D12_INFO_QUEUE_FILTER during device creation (see DirectX 12 application is crashing in Windows 11). The only trouble is that none of the sample programs change to fullscreen (i.e. borderless windowed) mode when pressing the Alt+Enter key combination. The programs always stay in windowed mode.
This hasn't worried me so far, because I've been copying the raytracing code over to a template (based on DirectX Tool Kit for Windows desktop) where fullscreen toggling works properly. In this way, I was able to run the HelloWorld and SimpleLighting samples successfully in both windowed mode and fullscreen (2560x1440 monitor resolution).
However, this hasn't been so successful for the ProceduralGeometry sample, which introduces intersection shaders. Once again, the original sample program renders the scene properly, but only in a bordered window. But when the code is reproduced in the template where I can toggle to fullscreen, the raytraced scene does not render properly.
In the scene, the triangle geometry used for the ground plane of the scene renders ok, but a translucent bounding box around the fractal pyramid is visible, and all other procedural geometry also appears translucent. Every couple of seconds, the bounding box for the metaballs also appears briefly, then vanishes.
I was able to determine that by freezing the scene, the reason for the translucency was that the following frames were being presented in sequence:
triangle ground plane quad only
floor geometry plus opaque fractal pyramid bounding box
all of the above plus opaque metaball bounding box
completed scene with opaque geometry and no bounding boxes
At the native framerate (165Hz on my machine), this results in both the procedural geometry and bounding boxes always being visible, but 'see-through' due to all the partially complete frames being presented to the display. This happens in both windowed and fullscreen modes, but it's worse in fullscreen, because the scene gets affected by random image corruption not seen in windowed mode.
I've been grappling with this issue for a few days and can't work out the problem. The only changes I've made to the sample program are the Windows 11 fix, and using a template for proper fullscreen rendering, which the original sample ignores or doesn't implement properly.
Hopefully someone can shed light on this perplexing issue!
I found the problem. Each sample has a header file called DXSampleHelper.h. For the ProceduralGeometry sample, this header file was updated with a helper class to manage structured buffers, which is very similar to the helper class for constant buffers.
The CopyStagingToGpu() method, which consists of a one line memcpy operation in both classes, is slightly different for the structured buffer class:
memcpy(m_mappedBuffers + instanceIndex * NumElementsPerInstance(), &m_staging[0], InstanceSize());
The same method in the constant buffer class is:
memcpy(m_mappedBuffers + instanceIndex, &m_staging[0], InstanceSize());
I.e. I was missing instanceIndex * NumElementsPerInstance() and thus the procedural geometry instances within the structured buffer were not correctly aligned in GPU memory.
I have two issues in my Metal App.
My call to currentPassDescriptor is stalling. I have too many drawables, apparently.
I'm wholly confused on how to most performantly configure the multiple MTKViews I am using.
Issue (1)
I have a problem with currentPassDescriptor in my app. It is occasionally blocking (for 1.00s) which, according to the docs, is because there is no currentDrawable available.
Background: I have 4 HD 1920x1080 videos playing concurrently, tiled out onto a 3840x2160 second external display as a debugging configuration. The pixel buffers of these AVPlayer instances are captured by 4 independent CVDIsplayLink callbacks and, from within the callback, there is the draw call to its assigned MTKView. A total of 4 MTKViews are subviews tiled on a single NSWindow, and are configured for manual drawing.
I'm using CVDisplayLink callbacks manually. If I don't, then I get stutter when mousing up on the app’s menus, for example.
Within each draw call, I do a bit of kernel shader work then attempt to obtain the currentPassDescriptor. If successful, I do one pass of a fragment/vertex shader and then present the drawable. My code flow follows Apple’s sample code as well as published examples.
According to the Metal System Trace, most of draw calls take under 5ms. The GPU is about 20-25% utilized and there’s about 25% of the GPU memory free. I can also cause the main thread to usleep() for 1 second without any hiccups.
Without any user interaction, there’s about a 5% chance of the videos stalling out in the first minute. If there’s some UI work going then I see that as windowServer work in Instruments. I also note that AVFoundation seems to cache about 15 frames of video onto the GPU for each AVPlayer.
If the cadence of the draw calls is upset, there's about a 10% chance that things stall completely or some of the videos -- some will completely stall, some will stall with 1hz updates, some won't stall at all. There's also less chance of stalling when running Metal System Trace. The movies that have stalled seem to have done so on obtaining a currentPassDescriptor.
This is really a poor design to have this currentPassDescriptor block for ≈1s during a render loop. So much so that I’m thinking of eschewing the MTKView all together and just drawing to a CAMetalLayer myself. But the docs on CAMetalLayer seem to indicate the same blocking behaviour will occur.
I also grab these 4 pixel buffers on the fly and render sub-size regions-of-interest to 4 smaller MTKViews on the main monitor; but the stutters still occur if this code is removed.
Is the drawable buffer limit per MTKView or per the backing CALayer? The docs for maximumDrawableCount on CAMetalLayer say the number needs to be 2 or 3. This question ties into the configuration of the views.
Issue (2)
My current setup is a 3840x2160 NSWindow with a single content view. This subclass of NSView does some hiding/revealing of the mouse cursor by introducing an NSTrackingRectTag. The MTKViews are tiled subviews on this content view.
Is this the best configuration? Namely, one NSWindow with tiled MTKViews… or should I do one MTKView per window?
I'm also not sure how to best configure these windows/layers — ie. by setting (or clearing) wantsLayer, wantsUpdateLayer, and/or canDrawSubviewsIntoLayer. I'm currently just setting wantsLayer to YES on the single content view. Any hints on this would be great.
Does adjusting these properties collapse all the available drawables to the backing layer only; are there still 2 or 3 per MTKView?
NB: I've attached a sample run of my Metal app. The longest 'work' on the top graph is just under 5ms. The clumps of green/blue are rendering on the 4 MTKViews. The 'work' alternates a bit because one of the videos is a 60fps source; the others are all 30fps.
According to a Qt blog post from 2011, there is supposed to be an environment variable QML_FLASH_MODE that can be set to 1 to see which areas of a QML application are being repainted.
I have set this variable for a Qt Quick 2 application that I'm writing and there is no visual difference to running the application without this variable.
Is there any other way to see QML redrawing activity? I'm trying to troubleshoot performance issues.
It looks like QML_FLASH_MODE was removed in this commit as part of refactoring to support separate rendering loops per window. Reason why it was removed is not obvious, but anyway its gone now.
If you think QML_FLASH_MODE will help you to analyse your scene, you can always get back to Qt 5.1.1 where it still should be present.
On the other hand, to visually analyse scene is not always efficient. I suggest to use qtcreator's profiling. Take a look at property bindings that reevaluated just before painting happens. This should help you to figure which property changes have triggered redraw and could give you some hints on how to optimise your scene.
If you just want to count fps or log when frames being painted, check QQuickWindow::frameSwapped signal.
I have a C++ DirectX-based third-party game engine compiled into a Windows Phone Runtime Component DLL. I'm working on integrating it into a project based off of a Windows Phone Direct3D with XAML App. The game engine DLL uses the the D3D device, context and render view texture provided by the application's Direct3DBackground::Draw() method.
The built-in renderer from the sample is gone and replaced by the game engine's.
I can render but there is constant black flickering. Every other frame is black. To prove to myself that it wasn't the renderer (which has been proven to work elsewhere), I cut out all the rendering code from the game engine DLL to a simply setting a clear color. The result is still the same.
At first I thought it was because the Direct3DXamlAppComponent generated by the sample was maybe running in a different thread from the game engine DLL, but that's not the case. They're on the same thread.
What rendering problem could this configuration be causing?
Does the game engine's renderer need a separate d3d device?
Does the game engine's renderer need a separate d3d device context?
Things I haven't tried yet:
creating a second d3d device on the DLL
converting the game engine to provide its own IDrawingSurfaceManipulationHandler. But I'm not sure if it'll just have the same problem as above.
The problem came from the render target view. I didn't realize that the pointer to it gets updated every frame. I had just set it to the game engine renderer once at start up. Now I update the render view target pointer every frame and now the black flickers are gone.
In my app I am rendering a Video generated from images I retrieve from the users photos. I have set up an AVAssetwriter with a AVAssetwriterInput has an AVAssetWriterInputPixelBufferAdaptor. I'm able to transform the ALAsset objects I retrieve from the users library to CVPixelBuffers and add them to the Video, which then is saved as an mp4. Adding all the images to the video is done on a background thread which sends a Notification to the main thread every frame, so the interface can be updated. All this works well, and I get a usable Movie file out of the app.
My problem now is, that when the user enters another application, after becoming active again the status of the ALAssetWriter changes to "failed", an I am not able to add any more images to the movie file. First I thought I might have to end the current Session on the writer and reopen a new one, once the app has become active again, but that doesn't seem to help.
I was just wondering how the general approach would be when I'd like the user to enter other applications. The best solution would be, if the rendering could continue in the background. I suppose I'd need a background thread from the UIApplication. But for now I'd be happy, if rendering could just continue after resuming my app.
I won't post any code for now, because it's really a lot, and my question possibly is conceptual. If you need to see code, I'll post it.
Edit 1:
Tested on iOS 4.3 and iOS 5. I've seen background rendering on other apps such as iTimelapse, but I'm not sure which frameworks they use.
Edit2:
I now have the information of an apple devforum member, that AVAssetWriter does not work in the background. So is there any other framework out there capable of rendering quicktime videos?
Turns out that AVAssetWriter just won't survive the app being suspended. You can add an extra 10 Minutes of rendertime by requesting background time, but after that the AssetWriter fails. Same happens if you use certain services on the phone. For example making or answering a call will make the AVAssetWriter fail as well.
If there are any OpenGL calls made when your app is in the background then that would explain this behaviour, looks fairly likely. From the OpenGL ES Pragramming Guide
Background Applications May Not Execute Commands on the Graphics Hardware
An OpenGL ES application is terminated if it attempts to execute
OpenGL ES commands on the graphics hardware. This not only refers to
calls made to OpenGL ES while your application is in the background,
but also refers to previously submitted commands that have not yet
completed. The main reason for preventing background applications from
processing OpenGL ES commands is to make the graphics processor
completely available to the frontmost application. The frontmost
application should always present a great experience to the user.
Allowing background applications to hog the graphics processor might
prevent that. Your application must ensure that all previously
submitted commands have been finished prior to moving into the
background.
The docs go on to enumerate a set of guidelines for the enter background/foreground app delegate callbacks. I think finding a way to do the rendering without the graphics hardware would be tricky, also the frameworks that allow mp4 encoding (like ffMpeg) are mainly GPL/LGPL, so you need to be careful if dealing with a commercial product (LGPL means you can link to the library dynamically, not statically, which is useless on iOS), as the license would propagate to your code.