What am trying to do is, count the revving("vroom" sound) of a physical car, through my app. Am coding in ReactNative. And I don't plan to create something complex, like communicating with the Car's inbuilt computer or anything to do this.
But instead, I was planning to create the app to listen to the nearby sounds. So if the nearby sound is that of a revving, then the app will simply count it.
I have done other features in my app, but listening to the sound and detect if it's a "vroom" sound is what am stuck with.
Based on my research, I can see that I have to make use of the Fast Fourier Transform algorithm. But am confused at how I can implement it in my ReactNative app. Am still searching for a package that has an implementation.
I have seen some apps that can be used to tune the sounds of Violin, Guitar, etc. What am trying to do is similar to this, but pretty simple. Once I get a basic idea, I will be able to get going. In my case, my app will be listening to the high decibel sound.
Any inputs would be highly appreciated.
This is known as Acoustic Event Detection. Possibly you can use an Audio Classification approach. The best way to solve it is using supervised machine learning. For example a CNN on mel-spectrograms. Here is an introduction. You can do the same in JavaScript using Tensorflow.JS. The official documentation contains a tutorial.
One of the first steps is to collect a small dataset of examples of "vroom" sounds versus other loud non-vroom sounds.
I have a task: to determine the sound source location.
I had some experience working with tensorflow, creating predictions on some simple features and datasets. I assume that for this task, there would be necessary to analyze the sound frequences and probably other related data on training and then prediction steps. The sound goes from the headset, so human ear is able to detect the direction.
1) Did somebody already perform that? (unfortunately couldn't find any similar project)
2) What kind of caveats could I meet while trying to achieve that?
3) Am I able to do that using this technology approach? Are there any other sound processing frameworks / technologies / open source projects that could help me ?
I am asking that here, since my research on google, github, stackoverflow didn't show me any relevant results on that specific topic, so any help is highly appreciated!
This is typically done with more traditional DSP with multiple sensors. You might want to look into time difference of arrival(TDOA) and direction of arrival(DOA). Algorithms such as GCC-PHAT and MUSIC will be helpful.
Issues that you might encounter are: DOA accuracy is function of the direct to reverberant ratio of the source, i.e. the more reverberant the environment the harder it is to determine the source location.
Also you might want to consider the number of location dimensions you want to resolve. A point in 3D space is much more difficult than a direction relative to the sensors
Using ML as an approach to this is not entirely without merit but you will have to consider what it is you would be learning, i.e. you probably don't want to learn the test rooms reverberant properties but instead the sensors spatial properties.
I am interested in getting into user interaction/shape detection with a simple usb webcam. I can use multiple webcams, but don't want to be restricted to using something like the kinect sensor. My detection cameras need to be set up on either side of a helmet (or if an individual one, on top). I have found some, but they don't really have the functionality I need and most are angled towards facial recognition. I need to be able to detect a basic human skeletal structure and determine if something is obstructing it. I would really rather be able to do it without using any sort of marker system on the target person. I would like for it to be able to target multiple structures. Obviously I am willing to do tweaking if necessary, but want to see how close I can get to what I need before I rebuild the wheel. I am trying to design an ai system that can determine how many people are in an area and where they are.
Doubt there will be anything like this since Microsoft spent a ton of money on the R&D for Kinect and it's probably all locked behind an NDA. I'm also guessing there's a lot of hardware within the Kinect that is not available in a standard webcam.
The closest thing that I could find to what you're looking for is the OpenKinect project, might be a good place to start your research.
I'm currently using a Processing Kinect library which supplies a depth map. I was wondering how I could take that and use it to create a 2D skeleton, if possible. Not looking for any code here, just a general process I could use to achieve those results.
Also, given that we've seen this in several of the Kinect games so far, would it be difficult to have multiple skeletons running at once?
Disclaimer: the reason why you still didn't get an answer for this question is probably because that's a current research problem. So I can't give you a direct answer but will try to help with some information and useful resources for this topic.
There are mainly 2 different approaches to create a skeleton from a depth map. The first one is to use machine learning, the second is purely algorithmic.
For the machine learning one, you'd need many samples of people doing a predetermined move, and use those samples to train your favorite learning algorithm. That's the approach that was taken and implemented by Microsoft in the XBox (source), it works really well BUT you need millions of samples to make it reliable... quite a drawback.
The "algorithmic" approach (understand without using a training set) can be done in many different ways and is a research problem. It's often based on modeling the possible body postures and trying to match that with the depth image received. That's the approach that was chosen by PrimeSense (the guys behind the kinect depth camera technology) for their skeleton tracking tool NITE.
The OpenKinect community maintains a wiki where they list some interesting research material about this topic. You might also be interested in this thread on the OpenNI mailing list.
If you're looking for an implementation of a skeleton tracking tool, PrimeSense released NITE (closed source), the one they made: it's part of the OpenNI framework. That's what's used in most of the videos you might have seen that involve skeleton tracking. I think it's able to handle up to 2 skeletons at the same time, but that requires confirmation.
The best solution is to use FAAST (http://projects.ict.usc.edu/mxr/faast/) which requires OpenNI. I have struggled to get OpenNI to work on my computer. I have not seen an approach yet using Code Laboratories' CL NUI.
An algorithmic approach is http://code.google.com/p/skeletonization/ but you may have a problem because your depthmap only represents surfaces and no closed objects.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I'm a beginner in game development and game programming. I have experience in computer graphics - mainly OpenGL
In those days Finally, I have some spare time to polish my game coding skills.
But when coming to program a simple 3d game, I couldn't find any good resource for free textures and models for 3d graphics (for 2d game for example, I found many resources for sprite sheets and so on).
Is there any good resource you're familiar with for 3d game textures/models?
This is not a programming queston.
As far as I know, good, free and high-quality modeling resources does not exist (from "good", "free" and "high-quality", select two).
There are multiple free model repositories, but quality of content is generally poor, and there are few places where you can buy models.
There are free textures in multiple places (like this one), and they are easier to find than good free models.
Also, most of free content frequently includes some kind of catch - "non-commercial use only", "creative commons share alike"(i.e. if you make derivative, it should use same license), or it is under GPL.
Anyway, if you're okay with "Creative Commons share alike" and GPL, then you can probably use content from some of opensource games (OpenArena ), and get quite a lot of textures from wikipedia or wikimedia commons, flickr, and you can google for "free textures". You should be careful about using content from opensource games - some opensource projects (like war$ow and sauerbraten) use closed-source/restricted licenses for game content (i.e. you're free to reuse modify engine, but you cannot modify game content and you cannot use it with modified engine. Reasons are pretty obvious).
Anyway, it depends on what kind of model you want. It is pretty easy to find "easy" stuff like boxes, barrels, etc, because everyone can do that. When it comes to guns and vehicles, there will be a trouble - quality will drop, and number of good models will decrease. And if you want a fully rigged animated character with multiple animation, normally you can forget about it - such content is almost impossible to find. But you can probably use mods for Q3 and Q2 if you want characters (you can forget about physics in this case, though)
I'd recommend to forget about "free stuff", and try to make content yourself or hire someone to do that.
If you decide to make content yourself, then you'll need digital photo camera and (optionally) graphic tablet. You can make mediocre textures from photos (digital camera is cheap) using gimp, gimp-resynthesizer plugin, gimp-texturisze plugin, high-pass filters, etc. You can also make normal maps using blender or gimp, and there are even tutorials about extracting them from photos (you still will need to process them by hand). Modeling and animation can be done in blender (after 1 or two weeks of training) using reference photos. Low poly modeling is pretty quick (20 minutes to make a low-poly low-quality gun, hour or two to make simple character), but texture and animation will take more (setting up animation for character can take a few hours for amateur, making one animation for character will take at least several hours as well, making texture unwrap - hour, painting texture - up to few days, depending on quality you want, available reference material, availability of graphic tablet, etc). It is possible to cut corners a bit - for example, for making animations, you can film motion using photo camera(or video camera), and then use it for rotoscoping. Also, you'll need to find some kind of model format blender can export to, or you'll have to write an export plugin in python.
The Blender foundation has a large model repository which may be of use.
There are some free models at Turbosquid that I use sometimes for my XNA games.
But of course, the best stuff is not free.
My experience is that there is very little in the way of quality 3d models with animation and full rigging freely available. There a few companies like this who sell suitable models cheaply and I guess most hobbyists could afford one or two models from them fairly easily which would probably be sufficient for learning. (I have no connection to them but I did buy one model pack from them which I quite liked)
It would be nice if there were a few more generally freely available 3d animated models around though. I even think it might be in the interests of some of the companies that make them to give a few away. If I'd been able to get further in my hobby projects I might have spent £100-200 in total on some nice model packs to make my project better, but due to the lack of any real 3d animated models I ended up losing interest in all my 3d projects before I got to the point of thinking maybe I'd spend a little money on this hobby. I wonder if the availability of a few more free quality models would actually significantly increase the size of the market for those companies as more people got their projects to the point where they were willing to spend a little money on it.
Some company should make a nice model pack with a few static models and a couple of fully rigged and animated humans and "monsters" and say that if the community donates £10000 they'll release them for free use. I suspect there are enough people out there who would like a few quality models they might reach this target in the same way that Blender was originally sold to the public.
I know that it's been a long time since this question was asked, but I ran into same problem when programming in XNA and I found a good solution. As long as you don't need rigged / animated models, Google Warehouse is the best place to search. As far as I know, each model submitted to Google Warehouse is available on Creative Commons license. You just need to:
Download and install Google Sketchup (Sketchup download)
Browse to find a model (Google Warehouse) - there's a 3D preview for each one!
Get a plugin to export Sketchup models to .X - I recommend the '3D RAD' plugin (3D RAD download)
If your model does not look good after the export, try to separate it into several less complex ones.
you are looking for open game art ...
http://thefree3dmodels.com/ has a multitude of free 3D models. I've used a few of these for animation purpose, maybe it'll help you too.