What I do wrong with algorithm to compare samples? - naudio

Question

I want compare and recognize two sound streams. I created my own algorithm, but its not working exactly like i would like. I trying for example, compare a few letters "A,B,C" with "D,E,F" or words "facebook" with "music" and algorithm give a true value for this comparing, but these are not the same words. My algorithm is so imprecise or it is cause of quality of sounds recorded using a microphone from laptop?
My concept of comparing algorithm:
Im taking for example 100 samples from one stream ( it can be in the middle of track) and im checking in loop every piece of second stream in specified way: first 0-99 samples , 1-100, 2-101 etc.
My program have about few tracks to compare with one input track, so my algorithm could get the best solution ( the most similar sample in track ) from every track Unfortunately it is getting wrong results.
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.ComponentModel;
using System.IO;
using System.Runtime.CompilerServices;
using System.Windows;
using Controller.Annotations;
using NAudio.Wave;
namespace Controller.Models
{
class DecompositionOfSound
{
private int _numberOfSimilarSamples;
private String _stream;
public string Stream
{
get { return _stream; }
set { _stream = value; }
}
public int IloscPodobnychProbek
{
get { return _numberOfSimilarSamples; }
set { _numberOfSimilarSamples = value; }
}
public DecompositionOfSound(string stream)
{
_stream = stream;
SaveSamples(stream);
}
private void SaveSamples(string stream)
{
var wave = new WaveChannel32(new WaveFileReader(stream));
Samples = new byte[wave.Length];
wave.Read(Samples, 0, (int) wave.Length);
}
private byte[] _samples;
public byte[] Samples
{
get { return _samples; }
set { _samples = value; }
}
}
class Sample: INotifyPropertyChanged
{
#region Cechy
private IList<DecompositionOfSound> _listoOfSoundSamples = new ObservableCollection<DecompositionOfSound>();
private string[] _filePaths;
#endregion
#region Property
public string[] FilePaths
{
get { return _filePaths; }
set { _filePaths = value; }
}
public IList<DecompositionOfSound> ListaSciezekDzwiekowych
{
get { return _listoOfSoundSamples; }
set { _listoOfSoundSamples = value; }
}
#endregion
#region Metody
public Sample()
{
LoadSamples(); // przy każdym nowym nagraniu należy zaktualizować !!!
}
public void DisplayMatchingOfSamples()
{
foreach (var decompositionOfSound in ListaSciezekDzwiekowych)
{
MessageBox.Show(decompositionOfSound.IloscPodobnychProbek.ToString());
}
}
public DecompositionOfSound BestMatchingOfSamples()
{
int max=0;
DecompositionOfSound referenceToObject = null;
foreach (var numberOfMatching in _listoOfSoundSamples)
{
if (numberOfMatching.IloscPodobnychProbek > max)
{
max = numberOfMatching.IloscPodobnychProbek;
referenceToObject = numberOfMatching;
}
}
return referenceToObject;
}
public void LoadSamples()
{
int i = 0;
_filePaths = Directory.GetFiles(#"Samples","*.wav");
while (i < _filePaths.Length)
{
ListaSciezekDzwiekowych.Add(new DecompositionOfSound(_filePaths[i]));
i++;
}
}
public void CheckMatchingOfWord(byte[] inputSound,double eps)
{
foreach (var probka in _listoOfSoundSamples)
{
CompareBufforsOfSamples(inputSound, probka, eps);
}
}
public void CheckMatchingOfWord(String inputSound,int iloscProbek, double eps)
{
var wave = new WaveChannel32(new WaveFileReader(inputSound));
var samples = new byte[wave.Length];
wave.Read(samples, 0, (int)wave.Length);
var licznik = 0;
var samplesTmp = new byte[iloscProbek];
while (licznik < iloscProbek)
{
samplesTmp[licznik] = samples[licznik + (wave.Length >> 1)];
licznik++;
}
foreach (var probka in _listoOfSoundSamples)
{
CompareBufforsOfSamples(samplesTmp, probka, eps);
}
}
private void CompareBufforsOfSamples(byte[] inputSound, DecompositionOfSound samples, double eps)
{
int max = 0;
for (int i = 0; i < (samples.Samples.Length - inputSound.Length); i++)
{
int counter = 0;
for (int j = 0; j < inputSound.Length; j++)
{
if (inputSound[j] * eps <= samples.Samples[i + j] &&
(inputSound[j] + inputSound[j] *(1 - eps)) >= samples.Samples[i + j])
{
counter++;
}
}
if (counter > max) max = counter;
}
samples.IloscPodobnychProbek = max;
}
#endregion
#region INotifyPropertyChange
public event PropertyChangedEventHandler PropertyChanged;
[NotifyPropertyChangedInvocator]
protected virtual void OnPropertyChanged([CallerMemberName] string propertyName = null)
{
PropertyChangedEventHandler handler = PropertyChanged;
if (handler != null) handler(this, new PropertyChangedEventArgs(propertyName));
}
#endregion
}
During comaparing all of sound samples, algorithm finds the soundtrack which have the highest number of matching samples, but it isnt correct record. Do my comparison of the two records make sense and how i can fix it to get expected result. Would you like to help me find solution of this problem ? Sorry for my English.
Kind Regards

Answer 1

You simply cannot do sample-level comparisons on recordings to determine your match. Even given two recordings on the same computer of the same word spoken by the same person - ie: every detail exactly the same - the recorded samples will differ. Digital audio is like that. It can sound the same, but the actual recorded samples will not match.
Speech To Text is not simple, and neither is voice recognition (ie: checking the identity of a person from their voice).
Instead of samples you need to examine the frequency profiles of the recordings. Various sounds in natural speech have distinctive frequency distributions. Sibilants - the s sound - have a wide distribution of higher frequencies for instance, so are easy to spot - which is why they used to use sibilant detection for the old yes/no response detection on phone systems.
You can get the frequency profile of a waveform by using a Fast Fourier Transform on a block of samples. Run through the audio stream and do a series of FFTs to get a 2D map of the frequencies of the waveform, then look for interesting things like sibilants (lots of high frequency, very little low frequency).
Of course you could just use one of the web-based Speech to Text APIs.