I was exploring development on kinect, and wanted to be able to recognize fingers rather than the entire hand only. The skeletal API by kinect official SDK only has the hand joint - no provisions for finger tracking. I also read that very recently Microsoft has included the grip recognition API in the new sdk and might include finger tracking in future releases.
My question is given the current resources, how do i go about to do finger tracking ? Do we have external libraries for the same ? Will it be feasible to actually implement finger tracking using kinect, given the fact the UX guidelines discourage such gestures.
Thanks.
how do i go about to do finger tracking? Do we have external libraries for the same?
There are several projects out there that demonstrate the ability of the Kinect to perform finger tracking. Some 3rd party libraries, offering a finger tracking API of some sort, also exist.
Here a very interesting one, with code, I found with a simple web search:
http://www.kinecthacks.com/kinect-hand-tracking-gesture-experiment/
If you are wanting to use the official SDK or one of the other SDKs is another concern. There is nothing stopping the official SDK from performing hand tracking, but there is nothing built into it for performing such actions.
Will it be feasible to actually implement finger tracking using kinect, given the fact the UX guidelines discourage such gestures.
If by "feasible" you mean possible - Yes. There is nothing preventing you from implementing your own finger tracking mechanism using the official SDK.
If, on the other hand, you mean the UX practicality of tracking fingers vs. gross body movements, that is something left to your application design. Having finger tracking for the sake of finger tracking does not make a good control-free interactive experience. The "Distance-Dependent Interactions" sections of the Kinect for Windows 1.7 Human Interface Guidelines does a good job of illustrating how a user's distance from the screen effects how best to interact with it. Notice the user in the example I link to above is very close to the screen.
What your application is going to do; How the user approaches your application (i.e., on the street, in the lab, with or without learning, standing/seated, etc...); Distances; User age & capabilities (i.e., children and elderly are generally less dexterous, as are those with disabilities). All these (and yes... more) come into if your application should support finger tracking at all.
One more library for finger tracking:
https://fingertip.codeplex.com/
Related
I am trying to make an existing website accessible but I'm struggling with a banner that has a video background streamed from vimeo. I found an identical issue in this SO thread Making a video element with no sound accessible. The poster stated that axe Tools failed the video as it had no captions - the same situation I face.
The accepted answer does not solve that issue though. Setting a parent element to aria-hidden has no impact on axeTools. I need to make the site accessible but also to pass an audit. Does anyone know a way to do this? It seems to me that WCAG is not well thought out here as it seems to have no video equivalent to alt="" or aria-role="presentation" as it does for images when a video is purely decorative and conveys no meaningful information.
Ignore axe or any other automated testing tool for the moment because they often have errors in their reporting. Axe-like tools are not the standard for measuring accessibility. WCAG is. Using a human and the Web Content Accessibility Guidelines is your sure source to know if you are conformant or not.
In this case, you have two guidelines that apply.
1.2.1 Audio-only and Video-only (Prerecorded)
2.2.2 Pause, Stop, Hide
For the first, it says:
Either an alternative for time-based media or an audio track is provided that presents equivalent information for prerecorded video-only content.
So two key points here are:
You can either have an alternative for time-based media (which is often a transcript, and in this case you could have a transcript that describes what is going on in the video) or you can have an audio track that describes the same thing.
You only need this alternative if the video has meaning. The guideline says the alternative must provide "equivalent information". So what's the purpose or meaning of the video? Does it provide information or is it decorative? If the video were removed, would critical information be lost? If it's purely decorative and doesn't provide anything, then the "equivalent information" would essentially be "nothing". The video could be hidden from assistive technology. But you'd want to talk to your designers about the purpose or meaning of the video.
The second guideline, 2.2.2, says if there's moving content (usually an animation but a video counts as moving information), then the user needs a way to stop it. That's normally done with a pause button on the video but it could be done with lots of other ways.
In summary, you need to decide the purpose of the video and whether it conveys information, and if so, decide what an "equivalent" experience would be, and also provide a way to pause the video.
I want to do my final year project on augmented reality geo-localization,
Please tell me, from where to start ?
what technology to learn ?
what are recruitments to development this kind of application ?
If you want to perform Geo-Localisation and use GPS, I wouldn't recommend using Unity. It's arbitrary coordinate system can be a bit confusing and difficult to make an app using GPS that's reliable enough.
For Augmented Reality, you can't use anything like Oculus Rift or Google Cardboard, because those are Virtual Reality headsets and have no way of allowing the user to see the real world. Augmented Reality peripherals are things like Microsoft Hololens or Google Glass, neither of which are commercially available but there are cheap knock offs that are. AR can also of course be used on any mobile device, since they all have a camera built-in and chips powerful enough to process all the tracking data.
As for making an actual app, the best thing you can do is have a go. Analyse your market, see where the gaps are. If you want to make an app for a specific OS that isn't cross-platform, I would recommend learning some Objbective-C (for iOS) or Java (for Android), if you don't know any already.
For cross-platform, I would say something like Xamarin would be useful for making an app on both the major OSes, it was recently made free by Microsoft and you can essentially make one app in C# that works across all devices.
For the Augmented Reality itself, there are frameworks out there that can be used for your purposes. Things like Kudan, Vuforia, Wikitude, etc. Some of them offer free versions of their software. You can use these to deal with all the tracking and projection side of things so you don't have to go about creating your own AR engine.
The best thing to do is probably to sit down for a few minutes, or hours, and think about what you're going to do. Figure out what you want the end result to look like, then work backwards and think about the best way to achieve that goal. Eventually you will arrive at the language and engine you want to use to make your life as easy as possible and then you can get started learning from tutorials online and getting your app out into the world.
you can check my tutorial about geo-based augmented reality solution on Android: https://www.netguru.co/blog/augmented-reality-mobile-android
I have presented there the basics and how to start with simple implementation.
Well a good starting point would be to ask yourself few questions:
What type of devices, you plan to work on(oculus rift, google cardboard, Microsoft Hololens, web etc)?
Augmented Reality is achievable in both Web-Context and Application-context. Which route do you want to go for?
Depending on these questions, if you choose to do a normal application based on a device, then depending on the device(Oculus Rift, Google Cardboard, Microsoft Hololens), you would need to grab their specific developer kits and learn how to develop apps using the documentation. For Oculus rift and Microsoft Hololens, you would need the respective Headsets inorder to make an app in that, but If it comes to google cardboard, all you need is you mobile phone with a good processing power.
There is another way to work on augmented reality applications, that is by doing a Web Application using some amazing javascript libraries like Awe.js, Three.js and JSARToolkit.
You can google about them and find out more.
One of the more accessible ways to learn Augmented reality is Project Tango.
Devices are around $500 last I checked and you can use a free version of Unity + Project Tango's free plugin:
https://developers.google.com/project-tango/
Which ever hardware you pick I'd recommend checking out Unity3D as it seems to be the platform of choice for AR/VR at the moment. There are other options... this just provides the most flexibility based on all of the platforms it supports.
Side note: I have no affiliation with Project Tango and am in fact working on another platform... but it isn't as accessible at the moment.
I need a 3D engine for a very specific task in Artificial Intelligence, and I'd like some input.
The first part is the trivial one - basically, all I need is a FPS engine (3rd person would be good, too), such that it allows me to navigate a room and interact with objects (if you have Java and Windows, I'm looking for something similar to the Give Challenge, but a little more up-to-date). Physics would be nice, but is not a must.
Now, the non-trivial part would be: I need to impose a virtual grid over this room, such that at any moment I can say "the player is located at B5 - now he moved to B6", and so on. I need to redirect this information to another system (namely, one which will give the player instructions about what to do) and, at the same time, send messages to the player, so I must be able to have a single point through which the game logic passes through; also, I'd love not having to write my own collision detection and such.
So far, I've tried:
the Source SDK: it seems a little overkill (since I'm not really planning to shoot anyone, at least half the code base is useless to the task), and since I'm not really a Windows developer, I'm spending too much time with the "easy" stuff (such as getting VS up and running). Plus, cross-platform would be really nice.
Blender game engine: while this worked decently, the interaction model seems a little weird, and some easy stuff (such as making sure the camera stays inside the scene or showing the mouse on screen) gets too weird too soon.
Crystalspace 3D: I've tried their demos, but it looks a little old-fashioned, and since that was one of the problems of previous engines (it's easier to get volunteers when your game looks nice) I'd like to try something else.
Now, maybe I'm asking a little too much for a single software, but I'd love some input. Can anyone suggest me an alternative? Or should I give one of the previous ones a second chance?
Try the UDK. All of the things you request are present, and it's free for personal/noncommercial projects. Here are some highlights:
Modern looking. The UDK features an intuitive-ish visual material design system, post-processing effects, Scaleform Gfx UIs from Autodesk, and more.
A visual scripting interface called Kismet that can control gameplay elements, the camera, and more.
UnrealScript, a scripting language similar in syntax to C, C++, Java, that gives you the ability to extend existing functionality or create your own.
Comprehensive documentation available on UDN.
Lots of community support outside of Epic, in places such as Polycount, Eat3d, 3dbuzz, and more.
Basically, "and more".
If what you're looking for is a professional, free (as in beer) engine that will allow you to focus primarily or solely on your differentiating gameplay features, Epic has set the bar high.
I am a beginner in Obj C development, though quite experienced (over 10 years) with other ECMAscript based languages and OOP development.
I want to build a simple flipbook style animation, controlled through swiping motion. I'm sure extremely simple for any advanced ObjC coders.
Can anyone with extensive ObjC-CocoaTouch experience give me some higher level recommendations?
ie,
1 -general application design, should I start with a simple view based application, or navigation based or?
2 -should I use 3rd party animation frameworks such as Cocos2D, or stick with built in classes and methods?
3 -if using built in methods, classes, what is the recommended way of achieving a animation, that will be controlled via swipe and touch gestures?
4 -I want to eventually have multiple 'flipbooks' that I can 'instantly' swap with one another, ie to give the net effect of an object changing color, etc, but not sure how to approach this from a memory management point of view, related to #1 above
Except for point 3 above, I'm not expecting any actual code examples. Just general guidelines to follow and perhaps, what are some next steps I should take in my goal as an ObjC code samurai.
While I am no expert I can think of a few things you could pursue to get the effect you want. Sounds like you want have a somewhat immersive experience for users. Cocos2D would definitely fit that bill. Me and my team at Get Set Games have been using it for the past year and have not been dissapointed with it. We haven't done the flipbook effect you speak of but quite possibly someone in the Cocos2D forums has implemented are at least attempted this. Forums are accessible here.
Having said that I think starting with the iPhone SDK UIKit framework and basic examples that come with it are great. There is a great array of samples on scrolling, swiping, etc. If you want to hook in Cocos2D or other frameworks later that's definitely an option.
We're about to build a Blackberry application but would love some input on whether to implement using J2ME (MIDlet based) or Blackberry native (UIApplication).
I understand some of the tradeoffs. J2ME will be more flexible if we want to port the app to other devices. RIM has better support for Blackberry native.
The place I'm still lacking information, though, is on the UI side. We want to build an app that has a great user experience, and one that looks like other apps BB users are accustomed to. Can we do this if we go the J2ME route?
Apologies for the somewhat subjective and less technical nature of the question.
I've tried it both ways - building a pure MIDP app to run on BlackBerry and non-BLackBerry platforms and building a separate BlackBerry app (often using much of the same business-logic and networking code as the MIDP app). Definitely go the BlackBerry native route.
It's all about the BB UI classes. They'll give you the ability to (among other things) respond to the different type of menu events (trackball and menu key), respond to BB specific key codes, if you're interested in the Storm take advantage of the orientation sensor and touch support. Plus they're a much richer set of UI elements to work with. You can build up a lot (but not all) of what they do in pure MIDP, but end up customizing so much of it for each platform that you won't save anything in the end. Starting with the BB UI and customizing saves a lot of time and effort.
Even in gaming applications, or for applications where you're custom drawing all your components, you have better access to the BlackBerry graphics APIs and get better performance going the BlackBerry native app route. And you still have better detail about input events.
Finally there are some nice lifecycle things you can do with BlackBerry native apps, like pushing to the background, or auto-running on system startup that you can't do with MIDlets, which may be of interest depending on your application.
Also think about market - if you're planning on supporting BlackBerry right away, and then maybe other MIDP platforms down the road, it's usually a better plan to execute the best you can on your initial platform. There probably won't be much of a reason to port to MIDP later if you don't succeed on BlackBerry first.
BlackBerry is a very difficult platform to program for. I went the native route and found it to be very poorly documented and overall just a miserable design to work with. I did feel it to be necessary because you will lose the ability for certain features (scroll wheel?) if you go the J2ME route. It's a tradeoff and you'll have to choose the one that fits better for your specific need.
I have been working with J2ME since 2002 when the Nextel/Motorola phones were in the forefront of J2ME functionality. I deal primarily with non-gaming applications. Today with J2ME evolving and more phones conforming to the JSRs my application continues to port smoothly. This year I finally got around to moving it to a Blackberry device. Two days later my application is fully functional on a Blackberry. The application is purely generic J2ME and uses many of the JSRs (location, bluetooth, xml, etc ). A strong UI design helps. I extended some of my UI classes to support the QWERTY/SureType keypads. I have used a variety of native applications found on Blackberry devices (not games) and I have yet to justify any reason to venture off to the RIM API to see what I would benefit. The arguments expressed above by other fellow programmers further prove that there is very little to gain unless you are purely devoted to a Blackberry device platform.
Without knowing the scope of your application, meaning which devices you will target, another thing that you will need to take into consideration is the ever expanding differences in input for each BlackBerry model. Some devices, like the Curve and the Bold, are standard QWERTY pads where other devices, like the PEARL, have SureType pads. And then there's the Storm which supports both depending on the orientation of the screen. Also, the SureType pad needs to be used as a standard number pad if the user has it set that way. The native APIs have support for all of this where you won't have that luxury with standard MIDP.
Another tip when designing your user experience, I would get my hands on a few different BlackBerry models and try out some of the apps that come standard as there are a lot of shortcuts that users become accustomed to using that you might not even realize exist. For instance, the SPACE bar pages down. This is huge for reading docs, however I have a third party RSS feed reader that does not have this functionality and I always attempt to use it first before remembering that it doesn't exist in this app. That kind of small detail can make a huge difference to BlackBerry users.
There is a way in between: You could use J2ME-Polish and code natively in J2ME. Polish will add the look and feel of a native BlackBerry app to some extent. That has the advantage of porting your app to other J2ME phones and not be limited to BlackBerry.
I agree with Anthony, I'd go with the BlackBerry specific APIs for BlackBerry apps. J2ME is just not portable, and BlackBerry has a lot better support for their proprietary APIs.
From my experience: stick with the native BlackBerry UiApplication. It is the best for BlackBerry and provides the best user experience for BlackBerry users. You can't copy that with J2ME easily.
J2ME is simple to develop. And, BlackBerry API has many feature also you can use restricted class with simple signing. BlackBerry sells their sign certificate and it is cheap. 20$. You can use this certificate to sign and use all restricted class. In J2ME, sometimes you must use more than one certificate, even you can not use all phone capabilities. So, my suggestion is, use BlackBerry native API and some J2ME Wrapper Codes. So that, you can code easly as J2ME and can use all capabilities of BlackBerry native API.