I'm trying to determine whether an audio file uses VBR or not in a Cocoa app (OS X 10.8+).
AVFoundation doesn't seem to be able to answer the question at all, and AudioToolbox is lying to me.
The code below adamantly claims that any mp3 file I throw at it is VBR; even for files where I know that isn't the case (cross-checked with MediaInfo)
OSStatus result = noErr;
UInt32 size;
AudioFileID audioFile;
AudioStreamBasicDescription audioFormat;
AudioFormatPropertyID vbrInfo;
// Open audio file.
result = AudioFileOpenURL( (__bridge CFURLRef)originalURL, kAudioFileReadPermission, 0, &audioFile );
if( result != noErr )
{
NSLog( #"Error in AudioFileOpenURL: %d", (int)result );
return;
}
// Get data format
size = sizeof( audioFormat );
result = AudioFileGetProperty( audioFile, kAudioFilePropertyDataFormat, &size, &audioFormat );
if( result != noErr )
{
NSLog( #"Error in AudioFileGetProperty: %d", (int)result );
return;
}
// Get vbr info
size = sizeof( vbrInfo );
result = AudioFormatGetProperty( kAudioFormatProperty_FormatIsVBR, sizeof(audioFormat), &audioFormat, &size, &vbrInfo);
if( result != noErr )
{
NSLog( #"Error getting vbr info: %d", (int)result );
return;
}
NSLog(#"%# is VBR: %d", originalURL.lastPathComponent, vbrInfo);
Why is it lying? What am I doing wrong?
How can I realiably determine the VBR-iness of an audio file in a Cocoa Application?
VBR and CBR sample files can be obtained here (not my page).
Your best bet is to read the MPEG frame headers from the file directly. If they all specify the same encoding parameters (version, layer, bitrate, samplerate, & channels), the file is CBR. Otherwise, the file is VBR / ABR.
There are several MP3 libraries out there that can either be used to do this directly or have the correct code for you to borrow to do it. If the file is on fast storage, performance is also well within your requirements.
If you need further assistance, I'm a major contributor to a C# MP3 decoder project and can point you to it for some examples.
In VBR mp3 first frame is different. So one doesn't need to read the whole file to find out if it is VBR or not.
Related
i'm recording Video/Audio using AVAssetwriter and want to be able to write silent Sample Buffers. I'm not that experienced in CoreAudio, so i'm having trouble coming up with a working solution.
The idea is to keep recording video when an audio device is disconnected until it's reconnected. The problem is, that AVFoundation somehow push the audio to the front, so the resulting movie file is massivley out of sync.
My current implementation tries to create an empty/silent CMSampleBuffer to place in between segments where there's no audio device connected.
if (audioOutput == captureOutput && audioWriterInput.readyForMoreMediaData) {
if (needToFillAudioGap) {
CMTime temp = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
CMItemCount numSamples = temp.value - lastAudioDisconnect.value;
OSStatus status;
CMBlockBufferRef bbuf = NULL;
CMSampleBufferRef sbuf = NULL;
int nchans = 2;
size_t buflen = numSamples * nchans * sizeof(float);
NSMutableData* data = [NSMutableData dataWithLength:buflen];
void* samples = [data mutableBytes];
status = CMBlockBufferCreateWithMemoryBlock(
kCFAllocatorDefault,
samples,
buflen,
kCFAllocatorNull,
NULL,
0,
buflen,
0,
&bbuf);
if (status != noErr) {
NSLog(#"CMBlockBufferCreateWithMemoryBlock error: %d", (int)status);
return;
}
CMBlockBufferRef blockBufferContiguous;
status = CMBlockBufferCreateContiguous(kCFAllocatorDefault,
bbuf,
kCFAllocatorNull,
NULL,
0,
buflen,
0,
&blockBufferContiguous);
CFRelease(bbuf);
if(status != noErr)
{
printf("CMBlockBufferCreateContiguous failed with error %d\n", (int)status);
return;
}
status = CMAudioSampleBufferCreateReadyWithPacketDescriptions(kCFAllocatorDefault, blockBufferContiguous, CMSampleBufferGetFormatDescription(sampleBuffer), numSamples, lastAudioDisconnect, NULL, &sbuf);
CFRelease(blockBufferContiguous);
if (status != noErr) {
NSLog(#"CMSampleBufferCreate error: %d", (int)status);
return;
}
BOOL r = [audioWriterInput appendSampleBuffer:sbuf];
if (!r) {
NSLog(#"appendSampleBuffer error: %d", (int)status);
return;
}
CFRelease(sbuf);
NSLog(#"Filling Audio Gap");
needToFillAudioGap = false;
} else {
if ([audioWriterInput appendSampleBuffer:sampleBuffer])
lastAudioDisconnect = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
}
}
The sampleBuffer at the top is the first samplebuffer after the audio device is reconnected. That gives me information on how long the gap I have to fill should be. LastAudioDisconnect always holds the Presentation Timestamp from the last audio samplebuffer that was written.
With Guard Malloc enabled the program crashes with: CrashIfClientProvidedBogusAudioBufferList
EDIT:
With Guard Malloc disabled i am able to reconnect the audio device multiple times while recording and when i stop to record, the gap is there without problems.
The problem then is that i only have a couple a minutes to stop the recording after reconnecting a device, because the AVAssetWriter fails randomly with the error code 11800 (AVErrorUnknown).
The error is because your created CMSampleBuffer is too long.
The created CMSampleBuffer should be the same length (contain the same number of samples) as the proceeding sample buffers that are filled with live audio. You can create multiple silent buffers, if necessary, and deliver them at the same rate as the live audio buffers.
I need to find whether a movie is Drop Frame or Non-Drop Frame.
I'm trying to find it in the attributes for a video file in one of the xcode video frameworks (either QTMovie or something from AVFoundation). Not having much luck.
I'm doing this to fill in necessary information in an FCP-X XML file.
Does anyone have any experience with this?
Important note, I am working in a 64 bit environment, and must stay there.
You can use CMTimeCodeFormatDescriptionGetTimeCodeFlags() to get the time code flags for a given timecode format description ref. You can get the format description ref by asking an AVAssetTrack for its formatDescriptions.
I think it would look something like this:
BOOL isDropFrame (AVAssetTrack* track)
{
BOOL result = NO;
NSArray* descriptions = [track formatDescriptions];
NSEnumerator* descriptionEnum = [descriptions objectEnumerator];
CMFormatDescriptionRef nextDescription;
while ((!result) && ((nextDescription = (CMFormatDescriptionRef)[descriptionEnum nextObject]) != nil))
{
if (CMFormatDescriptionGetMediaType(nextDescription) == kCMMediaType_TimeCode)
{
uint32_t timeCodeFlags = CMTimeCodeFormatDescriptionGetTimeCodeFlags ((CMTimeCodeFormatDescriptionRef)nextDescription);
result = ((timeCodeFlags & kCMTimeCodeFlag_DropFrame) != 0);
}
}
return result;
}
I've been pulling my hair out over this.
I've found a few things here, but nothing actually seems to work. And the documentation is really limited.
What I'm trying to figure out here is how to get the start time code of a Quicktime movie in Objective-C from the timecode track, and getting a human-readable output from that.
I've found this:
SMPTE TimeCode from Quick Time
It works perfectly in 32-bit mode. But it doesn't work in 64-bit mode because of the Quicktime API. The software I need to incorporate it into already has been and must continue to run 64-bit.
I'm losing my mind here. Anyone out there know about these APIs?
Ultimately, the goal here is to figure out the start timecode of the Quicktime because its needed to set the OFFSET in FCP-X XML files. Without it, the video files are brought in without audio (or, really, its just slipped a lot).
Use AVFoundation framework instead of QuickTime. The player initialisation is well explained in the documentation: https://developer.apple.com/library/mac/#documentation/AudioVideo/Conceptual/AVFoundationPG/Articles/02_Playback.html#//apple_ref/doc/uid/TP40010188-CH3-SW2
Once your AVAsset is loaded in memory, you can extract the first sample frame number (timeStampFrame) by reading the content of the timecode track if present:
long timeStampFrame = 0;
for (AVAssetTrack * track in [_asset tracks]) {
if ([[track mediaType] isEqualToString:AVMediaTypeTimecode]) {
AVAssetReader *assetReader = [AVAssetReader assetReaderWithAsset:_asset error:nil];
AVAssetReaderTrackOutput *assetReaderOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:track outputSettings:nil];
if ([assetReader canAddOutput:assetReaderOutput]) {
[assetReader addOutput:assetReaderOutput];
if ([assetReader startReading] == YES) {
int count = 0;
while ( [assetReader status]==AVAssetReaderStatusReading ) {
CMSampleBufferRef sampleBuffer = [assetReaderOutput copyNextSampleBuffer];
if (sampleBuffer == NULL) {
if ([assetReader status] == AVAssetReaderStatusFailed)
break;
else
continue;
}
count++;
CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer);
size_t length = CMBlockBufferGetDataLength(blockBuffer);
if (length>0) {
unsigned char *buffer = malloc(length);
memset(buffer, 0, length);
CMBlockBufferCopyDataBytes(blockBuffer, 0, length, buffer);
for (int i=0; i<length; i++) {
timeStampFrame = (timeStampFrame << 8) + buffer[i];
}
free(buffer);
}
CFRelease(sampleBuffer);
}
if (count == 0) {
NSLog(#"No sample in the timecode track: %#", [assetReader error]);
}
NSLog(#"Processed %d sample", count);
}
}
if ([assetReader status] != AVAssetReaderStatusCompleted)
[assetReader cancelReading];
}
}
This is a little more tricky than the QuickTime API and there must be some improvement to the code above but it works for me.
How do you convert CMSampleBufferRef to NSData?
I've managed to get the data for an MPMediaItem by following Erik Aigner's answer on this thread, however the data is of type CMSampleBufferRef.
I know CMSampleBufferRef is a struct and is defined in the CMSampleBuffer Reference in the iOS Dev Library, but I don't think I fully understand what it is. None of the CMSampleBuffer functions seem to be an obvious solution.
Here you go this works for audio sample buffer which is what you are looking at, and if you want to look at the whole process (getting all audio data from MPMediaItem into a file check out this question
CMSampleBufferRef ref=[output copyNextSampleBuffer];
// NSLog(#"%#",ref);
if(ref==NULL)
break;
//copy data to file
//read next one
AudioBufferList audioBufferList;
NSMutableData *data=[[NSMutableData alloc] init];
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(ref, NULL, &audioBufferList, sizeof(audioBufferList), NULL, NULL, 0, &blockBuffer);
// NSLog(#"%#",blockBuffer);
for( int y=0; y<audioBufferList.mNumberBuffers; y++ )
{
AudioBuffer audioBuffer = audioBufferList.mBuffers[y];
Float32 *frame = (Float32*)audioBuffer.mData;
[data appendBytes:frame length:audioBuffer.mDataByteSize];
}
CFRelease(blockBuffer);
CFRelease(ref);
ref=NULL;
blockBuffer=NULL;
[data release];
I have been working on reading in an audio asset using AVAssetReader so that I can later play back the audio with an AUGraph with an AudioUnit callback. I have the AUGraph and AudioUnit callback working but it reads files from disk and if the file is too big it would take up too much memory and crash the app. So I am instead reading the asset directly and only a limited size. I will then manage it as a double buffer and get the AUGraph what it needs when it needs it.
(Note: I would love know if I can use Audio Queue Services and still use an AUGraph with AudioUnit callback so memory is managed for me by the iOS frameworks.)
My problem is that I do not have a good understanding of arrays, structs and pointers in C. The part where I need help is taking the individual AudioBufferList which holds onto a single AudioBuffer and add that data to another AudioBufferList which holds onto all of the data to be used later. I believe I need to use memcpy but it is not clear how to use it or even initialize an AudioBufferList for my purposes. I am using MixerHost for reference which is the sample project from Apple which reads in the file from disk.
I have uploaded my work in progress if you would like to load it up in Xcode. I've figured out most of what I need to get this done and once I have the data being collected all in one place I should be good to go.
Sample Project: MyAssetReader.zip
In the header you can see I declare the bufferList as a pointer to the struct.
#interface MyAssetReader : NSObject {
BOOL reading;
signed long sampleTotal;
Float64 totalDuration;
AudioBufferList *bufferList; // How should this be handled?
}
Then I allocate bufferList this way, largely borrowing from MixerHost...
UInt32 channelCount = [asset.tracks count];
if (channelCount > 1) {
NSLog(#"We have more than 1 channel!");
}
bufferList = (AudioBufferList *) malloc (
sizeof (AudioBufferList) + sizeof (AudioBuffer) * (channelCount - 1)
);
if (NULL == bufferList) {NSLog (#"*** malloc failure for allocating bufferList memory"); return;}
// initialize the mNumberBuffers member
bufferList->mNumberBuffers = channelCount;
// initialize the mBuffers member to 0
AudioBuffer emptyBuffer = {0};
size_t arrayIndex;
for (arrayIndex = 0; arrayIndex < channelCount; arrayIndex++) {
// set up the AudioBuffer structs in the buffer list
bufferList->mBuffers[arrayIndex] = emptyBuffer;
bufferList->mBuffers[arrayIndex].mNumberChannels = 1;
// How should mData be initialized???
bufferList->mBuffers[arrayIndex].mData = malloc(sizeof(AudioUnitSampleType));
}
Finally I loop through the reads.
int frameCount = 0;
CMSampleBufferRef nextBuffer;
while (assetReader.status == AVAssetReaderStatusReading) {
nextBuffer = [assetReaderOutput copyNextSampleBuffer];
AudioBufferList localBufferList;
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(nextBuffer, NULL, &localBufferList, sizeof(localBufferList), NULL, NULL,
kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment, &blockBuffer);
// increase the number of total bites
bufferList->mBuffers[0].mDataByteSize += localBufferList.mBuffers[0].mDataByteSize;
// carefully copy the data into the buffer list
memcpy(bufferList->mBuffers[0].mData + frameCount, localBufferList.mBuffers[0].mData, sizeof(AudioUnitSampleType));
// get information about duration and position
//CMSampleBufferGet
CMItemCount sampleCount = CMSampleBufferGetNumSamples(nextBuffer);
Float64 duration = CMTimeGetSeconds(CMSampleBufferGetDuration(nextBuffer));
Float64 presTime = CMTimeGetSeconds(CMSampleBufferGetPresentationTimeStamp(nextBuffer));
if (isnan(duration)) duration = 0.0;
if (isnan(presTime)) presTime = 0.0;
//NSLog(#"sampleCount: %ld", sampleCount);
//NSLog(#"duration: %f", duration);
//NSLog(#"presTime: %f", presTime);
self.sampleTotal += sampleCount;
self.totalDuration += duration;
frameCount++;
free(nextBuffer);
}
I am unsure about the what that I handle mDataByteSize and mData, especially with memcpy. Since mData is a void pointer this is an extra tricky area.
memcpy(bufferList->mBuffers[0].mData + frameCount, localBufferList.mBuffers[0].mData, sizeof(AudioUnitSampleType));
In this line I think it should be copying the value from the data in localBufferList to the position in the bufferList plus the number of frames to position the pointer where it should write the data. I have a couple of ideas on what I need to change to get this to work.
Since a void pointer is just 1 and not the size of the pointer for an AudioUnitSampleType I may need to multiply it also by sizeof(AudioUnitSampleType) to get the memcpy into the right position
I may not be using malloc properly to prepare mData but since I am not sure how many frames there will be I am not sure what to do to initialize it
Currently when I run this app it ends this function with an invalid pointer for bufferList.
I appreciate your help with making me better understand how to manage an AudioBufferList.
I've come up with my own answer. I decided to use an NSMutableData object which allows me to appendBytes from the CMSampleBufferRef after calling CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer to get an AudioBufferList.
[data appendBytes:localBufferList.mBuffers[0].mData length:localBufferList.mBuffers[0].mDataByteSize];
Once the read loop is done I have all of the data in my NSMutableData object. I then create and populate the AudioBufferList this way.
audioBufferList = (AudioBufferList *)malloc(sizeof(AudioBufferList));
if (NULL == audioBufferList) {
NSLog (#"*** malloc failure for allocating audioBufferList memory");
[data release];
return;
}
audioBufferList->mNumberBuffers = 1;
audioBufferList->mBuffers[0].mNumberChannels = channelCount;
audioBufferList->mBuffers[0].mDataByteSize = [data length];
audioBufferList->mBuffers[0].mData = (AudioUnitSampleType *)malloc([data length]);
if (NULL == audioBufferList->mBuffers[0].mData) {
NSLog (#"*** malloc failure for allocating mData memory");
[data release];
return;
}
memcpy(audioBufferList->mBuffers[0].mData, [data mutableBytes], [data length]);
[data release];
I'd appreciate a little code review on how I use malloc to create the struct and populate it. I am getting a EXC_BAD_ACCESS error sporadically but I cannot pinpoint where the error is just yet. Since I am using malloc on the struct I should not have to retain it anywhere. I do call "free" to release child elements within the struct and finally the struct itself everywhere that I use malloc.