I've a question regarding IOSurface on Cocoa.
After an extensive research required to switch my OPENGL realtime application to 64 bit, I've taken the only path to support Quicktime playback spawning a background thread that pulls the frames installing a frame-ready callback and then with QTVisualContextCopyImageForTime , and pass the IOSurfaceRef through RPC to the parent process.
Everything works fine but there's one main issue. In my 32 bit application I was able to serialize any call to the GL subsystem by rendering a frame, pull the QT frames for the next pass and then wait for the next V-sync. This produced a very smooth and stable result.
Using the IOSurface technique gives me no way to synchronize when my app draws a frame and when the background process pulls the IOSurface from the quicktime movie. The result is that, on a random basis, I experience performances SPYKES. Indeed using the OPENGL driver monitor raises the CPU WAIT cycles up to 10% in my 64 bit app, while I have 0% CPU Wait graph under 32 bit.
Anyone here used IOSurface in a real world application and faced issues like this one ? I've though about an interprocess mutex/lock , but considering I need to lock/unlock about 120 times x second, I was not able to find a valid solution, it doesn't seems that darwin has something like the NAMED SIGNALS available in Win32...
Any suggestion, or I should take a totally different approach to the problem ?
Thanks !
Related
TLDR: How can I achieve low-latency, low-cpu impact webcam aquistition in Julia?
edit: I also posted this on the julia devs forum
I am new to Julia. I am interested in processing the video feed from a connected webcam, and see what kind of performance I can get out of Julia.
I am working on Linux Ubuntu, 16.04.
The only way I have found to get webcam input through video4linux, is through VideoIO, which is working on my system. The video has an unacceptable lag however, of up to 4 seconds. I assume this is given by the buffering of frames by the driver and/or libav (or is it ffmpeg, I dunno). With any camera api worth its name, I should be able to access the latest camera frame acquired... or at least set the size of the queue that Im popping frames from. Seems there is no such option in VideoIO, or maybe I am missing it.
It really is important for me to be able show-case Julia as a high performance language to non-techies... so this lag will ruin the demo I am hoping to put together.
edit: here is some of the code I have:
module myViewCam
export myView
import VideoIO, ImageView;
function myView()
camera = VideoIO.opencamera();
buf = VideoIO.read(camera);
guidict = ImageView.imshow(buf);
while !eof(camera)
VideoIO.read!(camera, buf);
ImageView.imshow(guidict["gui"]["canvas"], buf);
sleep(0.00001);
end
end
end
Assuming above is content of myViewCam.jl at the Julia prompt (the "REPL"), I type:
include("myViewCam.jl");
myViewCam.myView();
Note that this is a fix for the function "VideoIO.viewcam()" which does not work out of the box it seems.
On my system, this brings the Julia thread up to about 100% cpu usage, at the beginning of video-stream there is about 4 seconds lag, but this evens out over time, until it lands on about 0.5 seconds lag. There obviously is some queue where frames are popped from.
Also see Video4Linux wrapper in Julia which works well with Images.jl:
https://github.com/Affie/Video4Linux.jl
It's not registered yet, but has been around for a while. It is possible to make this process multithreaded in Julia using SharedArrays.jl, or likely the new Composible Threading model since Julia 1.3.
PS, this vendor specific camera interface package exists too: https://github.com/JuliaCameras/RealSense.jl
I'm trying to port an android/iOS game to windows phone 8(cocos2dx v 2.2). I'm using the exact same code base that I've used for android and iOS. The game functions just fine, but I facing some major FPS drop. The game runs flawlessly at 60FPS in android and iOS, but I'm getting roughly about 35FPS on wp8. Has this got to do anything with differences in OpenGL and directX?
I doubt its got to do with the game's logic and calculations because when the game starts in windows phone, it starts with 60FPS on the main menu, which has got like 5 sprites. But as I add more sprites on the screen, say about 30 of them(average number of sprites when I'm IN the game) the FPS rapidly drops to 35-40 range. Note that there are no schedulers or update functions running at this point. I did the same test on Android, but the FPS didn't drop. Does the win8 port of cocos2dx suck?
Any help,comments or redirection to useful articles would be appreciated.
Thank you.
In case anyone runs into similar issue, I reduced the number of children in the scene and deployed the build in release mode. Gave a major boost to the FPS. Also, I had a bunch of float to string and int to string conversions happening in every frame inside the update function. That was eating away on the processing speed too.
Actually, the Cocos2dx port for WP8 is ok, but outdated. Cocos2d-x is now at 3.0 beta, but the WP8 was left at 2.0 alpha.
Anyway... in Cocos there are some recursive drawing functions which are very heavy on the CPU, and also, keep in mind that even though WP8 is supposed tu support arrays, lists, maps etc. they are very slow on WP8.
And since you came to this subject, Please let me know if you managed to successfully put cocos2d-x on an XAML+D3D Interop project. I am getting tons of crashes.
EDIT: Indeed, the recursive calls which process (draw or update) child "CCNode"s are very heavy on the device. However, after putting Cocos2d-x ver. 2.0alpha for WP8 into a XAML+D3D interop project, I found a whole lot of memory related issues. Apparently, after doing this (or just because I don't know how to properly configure my VS project and allow loose addressing), a lot of uninitialized pointers and data cause some memory overlaps, leading to major crashes.
This proves only that it was truely an alpha release :) Too bad no newer version of Cocos2d-x for Wp8 is available.
I recently I came across an error that I cannot understand. The game I'm developing using Cocos2D just freezes at a certain random point -- it gets a SIGSTOP -- and I cannot find the reason. What tool can I use (and how do I use it) to find out where the error occurs and what's causing it?
Jeremy's suggestion to stop in the debugger is a good one.
There's a really quick way to investigate a freeze (or any performance issue), especially when it's not easy to reproduce. You have to have a terminal handy (so you'll need to be running in the iOS simulator or on Mac OS X, not on an iOS device).
When the hang occurs pop over to a terminal and run:
sample YourProgramName
(If there are spaces in your program name wrap that in quotes like sample "My Awesome Game".) The output of sample is a log showing where your program is spending time, and if your program is actually hung, it will be pretty obvious which functions are stuck.
I disagree with Aaron Golden's answer above as running on a device is extremely useful in order to have a real-case scenario of where the app freezes. The simulator has more memory and does not reproduce the hardware of the device in an accurate way (for example, the frame rate is in certain cases lower).
"Obviously", you need to connect your device (with a developer profile) on Xcode and look at the console terminal to look for traces that user #AaronGolden suggested.
If those are not enough you might want to enable a general exception breakpoint in Xcode to capture more of the stacktrace messages.
When I started learning Cocos2D my app often frooze. This is a list of common causes:
I wasn't using sprite sheets and hence the frame rate was dropping drammatically
I was using too much memory (too many high-definition sprites. Have a look at TexturePacker and use pvr.ccz or pvr.gz format; it cuts memory allocation in half)
Use instruments to profile your app for memory warnings (for example, look at allocation instruments and look for memory warnings).
Currently I am using the DTWGestureRecognizer open source tool for Kinect SDK v1.5. I have recorded a few gestures and use them to navigate through Windows 7. I also have implemented voice control for simple things such as opening PowerPoint, Chrome, etc.
My main issue is that the application uses quite a bit of my CPU power which causes it to become slow. During gestures and voice commands, the CPU usage sometimes spikes to 80-90%, which causes the application to be unresponsive for a few seconds. I am running it on a 64 bit Windows 7 machine with an i5 processor and 8 GB of RAM. I was wondering if anyone with any experience using this tool or Kinect in general has made it more efficient and less performance hogging.
Right now I removed sections which display the RGB video and the Depth video but even doing that did not make a big impact. Any help is appreciated, thanks!
Some of the factors I can think of are
Reduce the resolution.
Reduce the frames being recorded/processed by the application using polling model i.e. OpenNextFrame(int millisecondsWait) method of DepthStream, ColorStream & SkeletonStream
instead of event model.
Tracking mode is Default instead of Seated(sensor.SkeletonStream.TrackingMode =
SkeletonTrackingMode.Default) as seated consumes more resources.
Use sensor.MapDepthFrameToColorFrame instead of calling sensor.MapDepthToColorImagePoint method in a loop.
Last and most imp. is the algorithm used in the open source tool.
The Info
I recently launched an app on the AppStore. After testing on the simulator thousands of times, and actual devices hundreds of times we finally released our app.
The Problem
Reviews started popping up about app crashes when the user launches the app. We figured that the app crashes on launch on iOS devices with less than (or equal to) 256 Mb of RAM. The following devices are devices our app supports with less than 256:
iPod Touch 4G
iPhone 3GS
iPad 1
The app doesn't always crash. Sometimes it launches fine and runs smoothly. Other times it crashes. The time from launch (when the user taps the icon) to crash is usually two seconds, which would mean that the system isn't shutting it down.
Findings
When using Instruments to test on certain devices, I find the following:
There are no memory leaks (I'm using ARC), but there are memory warnings
Items are being allocated like crazy. There are so many allocated items, and even though I'm using ARC it's as if ARC isn't doing what it's supposed to be doing
Because of what I see as "over-allocation", the result is:
This app takes (on average) 60 MB of Real Memory and 166 MB of Virtual. When the app launches the memory being used quickly increases until it reaches about 60 MB at which point the view has been loaded.
Here is a snapshot of the Activity Monitor in Instruments:
I know that those figures are WAYY to high (although the CPU % never really gets up there). I am worried that ARC is not working properly, or the more likely case: I'm not allocating objects correctly. What could possibly be happening?
The Code and Warnings
In Xcode, there are only a few warnings, none of which pertain to the app launch or any files associated with the launching of the app. I placed breakpoints in both the App Delegate and my viewDidLoad method to check and see if the crash occurred there - it didn't.
More Background Info
Also, Xcode never generates any errors or messages in the debugger. There are also no crash reports in iTunes Connect, it just says, "Too few reports have been submitted for a report to be shown." I've added crash reporting to my app, but I haven't released that version.
A Few Questions
I started using Obj-C just as ARC arrived, so I'm new to dealing with memory, allocation, etc. (that is probably obvious) but I'd like to know a few things:
How can I use #autoreleasepool to reduce my memory impact? What do I do with memory warnings, what do I write in the didRecieveMemoryWarning since I'm using ARC?
Would removing NSLog statements help speed things up?
And the most important question:
Why does my app take up so much memory and how can I reduce my whopping 60 MB footprint?
I'd really appreciate any help! Thanks in advance!
EDIT: After testing on the iPhone 4 (A4), we noticed that the app doesn't crash when run whereas on devices with less than 256 MB of RAM it does.
I finally solved the issue. I spent a few hours pondering why my application could possibly take up more RAM than Angry Birds or Doodle Jump. That just didn't make sense, because my app does no CALayer Drawing, or complex Open GL Graphics Rendering, or heavy web connection.
I found this slideshow while searching for answers and slide 17 listed the ways to reduce memory footprint. One thing that stuck out was PNGCrush (Graphics Compression).
My app contains a lot of custom graphics (PNG files), but I hadn't thought of them affecting my app in any way, apparently images (when not optimized properly) severely increase an applications memory footprint.
After installing PNGCrush and using it on a particularly large image (3.2 MB) and then deleting a few unused images I ended up reducing my apps memory footprint from 60+ MB and severe lag to 35 MB and no lag. That took a whopping five minutes.
I haven't finished "crushing" all my images, but when I do I'll update everyone on the final memory footprint.
For all those interested, here is a link to a blog that explains how to install PNGCrush (it's rather complicated).
UPDATE: Instead of using the PNGCrush process (which is very helpful, although time consuming with lots of images) I now use a program called ImageOptim that provides a GUI for multiple scripts like PNGCrush. Heres a short description:
ImageOptim seamlessly integrates various optimisation tools: PNGOUT, AdvPNG, PNGCrush, extended OptiPNG, JpegOptim, jpegrescan, jpegtran, and Gifsicle.
Here's a link to the website with a free download for OS X 10.6 - 10.8. Note, I am not a developer, publisher or advertiser of this software.