Speex encode/decode causing hissing noise (Objective-c) - objective-c

Question

When I bypass the speex encode/decode steps the raw audio output is correct. What I'd like is for the entire buffer captured from my recording callback to be encoded, decoded, and sent back to the playback loop. The few items I'm unsure of are:
What size to allocate for the enc_buffer and dec_buffer
What length to specify in speex_bits_read_from(SpeexBits* bits,char* bytes,int len)
What max size to specify in int speex_bits_write(SpeexBits* bits,char* bytes,int max_len)
Here is my speex codec initialization:
#define SAMPLE_RATE 8000
#define MAX_FRAMES 100
#define FRAME_SIZE 160
enc_state = speex_encoder_init(&speex_nb_mode);
dec_state = speex_decoder_init(&speex_nb_mode);
spx_int32_t tmp;
tmp=5;
speex_encoder_ctl(enc_state, SPEEX_SET_QUALITY, &tmp);
tmp=1;
speex_encoder_ctl(enc_state, SPEEX_SET_COMPLEXITY, &tmp);
speex_encoder_ctl(enc_state, SPEEX_GET_FRAME_SIZE, &enc_frame_size );
speex_decoder_ctl(dec_state, SPEEX_GET_FRAME_SIZE, &dec_frame_size );
tmp = SAMPLE_RATE;
speex_encoder_ctl(enc_state, SPEEX_SET_SAMPLING_RATE, &tmp);
speex_decoder_ctl(dec_state, SPEEX_SET_SAMPLING_RATE, &tmp);
speex_bits_init(&enc_bits);
speex_bits_init(&dec_bits);
//Unsure of this allocation size
enc_buffer = (char*)malloc(sizeof(char)*enc_frame_size*MAX_FRAMES);
dec_buffer = (spx_int16_t*)malloc(sizeof(spx_int16_t)*dec_frame_size*MAX_FRAMES);
My encode/decode methods:
-(char*)encodeAudioBuffer:(spx_int16_t*)audioBuffer withByteSize:(int)numberOfFrames andWriteSizeTo:(int*)inSize{
speex_bits_reset(&enc_bits);
speex_encode_int(enc_state, audioBuffer, &enc_bits);
//Unsure of this third argument. 'numberOfFrames' is the stored number of input frames from my recording callback.
*inSize = speex_bits_write(&enc_bits, enc_buffer, numberOfFrames*enc_frame_size);
return enc_buffer;
}
-(spx_int16_t*)decodeSpeexBits:(char*)encodedAudio withEncodedSize:(int)encodedSize andDecodedSize:(int)decodedSize{
//Unsure of this third argument. 'encodedSize' is the number written to *inSize in the encode method
speex_bits_read_from(&dec_bits, encodedAudio, encodedSize*dec_frame_size);
speex_decode_int(dec_state, &dec_bits, dec_buffer);
return dec_buffer;
}
And they are called like this:
- (void)encodeBufferList:(AudioBufferList*)bufferList withNumberOfFrames:(int)numberOfFrames{
AudioBuffer sourceBuffer = bufferList->mBuffers[0];
int speexSize = 0;
char* encodedAudio = [speexCodec encodeAudioBuffer:(spx_int16_t*)sourceBuffer.mData withByteSize:numberOfFrames andWriteSizeTo:&speexSize];
spx_int16_t* decodedAudio = [speexCodec decodeSpeexBits:encodedAudio withEncodedSize:speexSize andDecodedSize:sourceBuffer.mDataByteSize];
memcpy(audioBuffer.mData, sourceBuffer.mData, numberOfFrames * sizeof(SInt32));
}
where "bufferList" is that returned from my recording/playback callbacks. Can someone verify that I am filling my buffer properly? I saw a similar problem reported here, but couldn't see where in my code I could be doing it wrong:
static OSStatus recordingCallback(void *inRefCon, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList *ioData)
{
AudioBuffer buffer;
OSStatus status;
AudioStreamer *input = (__bridge AudioStreamer*) inRefCon;
buffer.mDataByteSize = inNumberFrames * sizeof(SInt16);
buffer.mNumberChannels = 1;
buffer.mData = malloc( inNumberFrames * sizeof(SInt16));
AudioBufferList bufferList;
bufferList.mNumberBuffers = 1;
bufferList.mBuffers[0] = buffer;
status = AudioUnitRender([input rioAUInstance], ioActionFlags, inTimeStamp, inBusNumber, inNumberFrames, &bufferList);
[input encodeBufferList:&bufferList withNumberOfFrames:inNumberFrames];
return noErr;
}
static OSStatus playbackCallback(void *inRefCon, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList *ioData)
{
AudioStreamer* input = (__bridge AudioStreamer*)inRefCon;
UInt32 size = MIN(ioData->mBuffers[0].mDataByteSize, [input audioBuffer].mDataByteSize);
memcpy(ioData->mBuffers[0].mData, input.audioBuffer.mData, size);
return noErr;
}
The noise produced by encode/decode as it stands is a grainy static hiss, but it's not completely random information - when I blow into the microphone I can hear it behind the noise.
Any help putting this issue to bed would be greatly appreciated. I'll probably end up blogging about it once I get everything sorted out, seems like lots of folks are running into various trivial issues setting up this codec.

Answer 1

So it was a problem in the encode/decode functions, I needed to call speex_encode_int across a number of frames, as it only seems to handle 1 frame at a time, then write them to the encode buffer like this:
-(char*)encodeAudioBuffer:(spx_int16_t*)audioBuffer withNumberOfFrames:(int)numberOfFrames andWriteSizeTo:(int*)inSize{
speex_bits_reset(&enc_bits);
for(int i = 0; i < numberOfFrames; ++i){
speex_encode_int(enc_state, audioBuffer+i, &enc_bits);
}
*inSize = speex_bits_write(&enc_bits, enc_buffer, numberOfFrames);
return enc_buffer;
}
And similarly for decoding, speex_bits_read_from the encoded buffer, and then iterate across the dec_bits for each frame, writing to the decoded buffer
-(spx_int16_t*)decodeSpeexBits:(char*)encodedAudio withEncodedSize:(int)encodedSize andNumberOfFrames:(int)numberOfFrames{
speex_bits_read_from(&dec_bits, encodedAudio, encodedSize);
for(int i = 0; i < numberOfFrames; ++i){
speex_decode_int(dec_state, &dec_bits, dec_buffer+i);
}
return dec_buffer;
}
This still runs quite slow for me. Even after configuring the speex library to use fixed point calculations instead of floating point calculations, it still runs slower than my audio loop (causing a new sort of choppiness). Any leads on how to get this running faster?

Answer 2

on both of your loops you're passing the audio buffer but not taking in consideration the frame size:
for(int i = 0; i < numberOfFrames; ++i){
speex_encode_int(enc_state, audioBuffer+i, &enc_bits);
}
and it should be:
for(int i = 0; i < numberOfFrames; ++i){
speex_encode_int(enc_state, audioBuffer + (i * enc_frame_size), &enc_bits);
}
hope that helps.