FFmpeg jump to most recent frame - objective-c

I am looking for some help with dropping/skipping FFmpeg frames. The project I am working on streams live video which when the app goes into the background, upon returning to an active state the video stream spends a long time catching up by fast forwarding itself to the current frame. This isn't ideal and what I am aiming to achieve is have the app immediately jump to the most recent frame.
What I need to do is drop the amount of frames that are being fast-forwarded in order to catch up to the most recent frame. Is this possible? Here is my current code which decodes the frames:
- (NSArray *) decodeFrames: (CGFloat) minDuration
{
NSMutableArray *result = [NSMutableArray array];
#synchronized (lock) {
if([_reading integerValue] != 1){
_reading = [NSNumber numberWithInt:1];
#synchronized (_seekPosition) {
if([_seekPosition integerValue] != -1 && _seekPosition){
[self seekDecoder:[_seekPosition longLongValue]];
_seekPosition = [NSNumber numberWithInt:-1];
}
}
if (_videoStream == -1 &&
_audioStream == -1)
return nil;
AVPacket packet;
CGFloat decodedDuration = 0;
CGFloat totalDuration = [TimeHelper calculateTimeDifference];
do {
BOOL finished = NO;
int count = 0;
while (!finished) {
if (av_read_frame(_formatCtx, &packet) < 0) {
_isEOF = YES;
[self endOfFileReached];
break;
}
[self frameRead];
if (packet.stream_index ==_videoStream) {
int pktSize = packet.size;
while (pktSize > 0) {
int gotframe = 0;
int len = avcodec_decode_video2(_videoCodecCtx,
_videoFrame,
&gotframe,
&packet);
if (len < 0) {
LoggerVideo(0, #"decode video error, skip packet");
break;
}
if (gotframe) {
if (!_disableDeinterlacing &&
_videoFrame->interlaced_frame) {
avpicture_deinterlace((AVPicture*)_videoFrame,
(AVPicture*)_videoFrame,
_videoCodecCtx->pix_fmt,
_videoCodecCtx->width,
_videoCodecCtx->height);
}
KxVideoFrame *frame = [self handleVideoFrame];
if (frame) {
[result addObject:frame];
_position = frame.position;
decodedDuration += frame.duration;
if (decodedDuration > minDuration)
finished = YES;
}
} else {
count++;
}
if (0 == len)
break;
pktSize -= len;
}
}
av_free_packet(&packet);
}
} while (totalDuration > 0);
_reading = [NSNumber numberWithInt:0];
return result;
}
}
return result;

Related

How to play pcm audio buffer from a socket server using audio unit circular buffer

I hope someone can help me. I am new to Objective-c and OSX and I am trying to play audio data I am receiving via socket into my audio queue. I found out this link https://stackoverflow.com/a/30318859/4274654 which in away address my issue with circular buffer.
However when I try to run my project it returns
It returns an error (OSStatus) -10865. That is why the code logs " Error enabling AudioUnit output bus".
status = AudioUnitSetProperty(_audioUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Output, kOutputBus, &one, sizeof(one));
Here is my code:
Test.h
#import <Foundation/Foundation.h>
#import <AudioToolbox/AudioToolbox.h>
#import "TPCircularBuffer.h"
#interface Test : Communicator
#property (nonatomic) AudioComponentInstance audioUnit;
#property (nonatomic) TPCircularBuffer circularBuffer;
-(TPCircularBuffer *) outputShouldUseCircularBuffer;
-(void) start;
#end
Test.m
#import "Test.h"
#define kOutputBus 0
#define kInputBus 1
#implementation Test{
BOOL stopped;
}
static OSStatus OutputRenderCallback(void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData){
Test *output = (__bridge Test*)inRefCon;
TPCircularBuffer *circularBuffer = [output outputShouldUseCircularBuffer];
if( !circularBuffer ){
SInt32 *left = (SInt32*)ioData->mBuffers[0].mData;
for(int i = 0; i < inNumberFrames; i++ ){
left[ i ] = 0.0f;
}
return noErr;
};
int32_t bytesToCopy = ioData->mBuffers[0].mDataByteSize;
SInt16* outputBuffer = ioData->mBuffers[0].mData;
uint32_t availableBytes;
SInt16 *sourceBuffer = TPCircularBufferTail(circularBuffer, &availableBytes);
int32_t amount = MIN(bytesToCopy,availableBytes);
memcpy(outputBuffer, sourceBuffer, amount);
TPCircularBufferConsume(circularBuffer,amount);
return noErr;
}
-(void) start
{
[self circularBuffer:&_circularBuffer withSize:24576*5];
stopped = NO;
[self setupAudioUnit];
// [super setup:#"http://localhost" port:5321];
}
-(void) setupAudioUnit
{
AudioComponentDescription desc;
desc.componentType = kAudioUnitType_Output;
desc.componentSubType = kAudioUnitSubType_VoiceProcessingIO;
desc.componentManufacturer = kAudioUnitManufacturer_Apple;
desc.componentFlags = 0;
desc.componentFlagsMask = 0;
AudioComponent comp = AudioComponentFindNext(NULL, &desc);
OSStatus status;
status = AudioComponentInstanceNew(comp, &_audioUnit);
if(status != noErr)
{
NSLog(#"Error creating AudioUnit instance");
}
// Enable input and output on AURemoteIO
// Input is enabled on the input scope of the input element
// Output is enabled on the output scope of the output element
UInt32 one = 1;
status = AudioUnitSetProperty(_audioUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Output, kOutputBus, &one, sizeof(one));
if(status != noErr)
{
NSLog(#"Error enableling AudioUnit output bus");
}
// Explicitly set the input and output client formats
// sample rate = 44100, num channels = 1, format = 16 bit int point
AudioStreamBasicDescription audioFormat = [self getAudioDescription];
status = AudioUnitSetProperty(_audioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, kOutputBus, &audioFormat, sizeof(audioFormat));
if(status != noErr)
{
NSLog(#"Error setting audio format");
}
AURenderCallbackStruct renderCallback;
renderCallback.inputProc = OutputRenderCallback;
renderCallback.inputProcRefCon = (__bridge void *)(self);
status = AudioUnitSetProperty(_audioUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Global, kOutputBus, &renderCallback, sizeof(renderCallback));
if(status != noErr)
{
NSLog(#"Error setting rendering callback");
}
// Initialize the AURemoteIO instance
status = AudioUnitInitialize(_audioUnit);
if(status != noErr)
{
NSLog(#"Error initializing audio unit");
}
}
- (AudioStreamBasicDescription)getAudioDescription {
AudioStreamBasicDescription audioDescription = {0};
audioDescription.mFormatID = kAudioFormatLinearPCM;
audioDescription.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked | kAudioFormatFlagsNativeEndian;
audioDescription.mChannelsPerFrame = 1;
audioDescription.mBytesPerPacket = sizeof(SInt16)*audioDescription.mChannelsPerFrame;
audioDescription.mFramesPerPacket = 1;
audioDescription.mBytesPerFrame = sizeof(SInt16)*audioDescription.mChannelsPerFrame;
audioDescription.mBitsPerChannel = 8 * sizeof(SInt16);
audioDescription.mSampleRate = 44100.0;
return audioDescription;
}
-(void)circularBuffer:(TPCircularBuffer *)circularBuffer withSize:(int)size {
TPCircularBufferInit(circularBuffer,size);
}
-(void)appendDataToCircularBuffer:(TPCircularBuffer*)circularBuffer
fromAudioBufferList:(AudioBufferList*)audioBufferList {
TPCircularBufferProduceBytes(circularBuffer,
audioBufferList->mBuffers[0].mData,
audioBufferList->mBuffers[0].mDataByteSize);
}
-(void)freeCircularBuffer:(TPCircularBuffer *)circularBuffer {
TPCircularBufferClear(circularBuffer);
TPCircularBufferCleanup(circularBuffer);
}
-(TPCircularBuffer *) outputShouldUseCircularBuffer
{
return &_circularBuffer;
}
-(void) stop
{
OSStatus status = AudioOutputUnitStop(_audioUnit);
if(status != noErr)
{
NSLog(#"Error stopping audio unit");
}
TPCircularBufferClear(&_circularBuffer);
_audioUnit = nil;
stopped = YES;
}
-(void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)event{
switch (event) {
case NSStreamEventOpenCompleted:
NSLog(#"Stream opened");
break;
case NSStreamEventHasBytesAvailable:
if (stream == [super inputStream]) {
NSLog(#"NSStreamEventHasBytesAvailable");
uint8_t buffer[1024];
NSUInteger len;
while ([[super inputStream] hasBytesAvailable]) {
len = [[super inputStream] read:buffer maxLength:sizeof(buffer)];
if (len > 0) {
//converting buffer to byte data
NSString *output = [[NSString alloc] initWithBytes:buffer length:len encoding:NSASCIIStringEncoding];
if (nil != output) {
//NSLog(#"server overideddddd said: %#", output);
}
NSData *data0 = [[NSData alloc] initWithBytes:buffer length:len];
if (nil != data0) {
SInt16* byteData = (SInt16*)malloc(len);
memcpy(byteData, [data0 bytes], len);
double sum = 0.0;
for(int i = 0; i < len/2; i++) {
sum += byteData[i] * byteData[i];
}
Byte* soundData = (Byte*)malloc(len);
memcpy(soundData, [data0 bytes], len);
if(soundData)
{
AudioBufferList *theDataBuffer = (AudioBufferList*) malloc(sizeof(AudioBufferList) *1);
theDataBuffer->mNumberBuffers = 1;
theDataBuffer->mBuffers[0].mDataByteSize = (UInt32)len;
theDataBuffer->mBuffers[0].mNumberChannels = 1;
theDataBuffer->mBuffers[0].mData = (SInt16*)soundData;
NSLog(#"soundData here");
[self appendDataToCircularBuffer:&_circularBuffer fromAudioBufferList:theDataBuffer];
}
}
}
}
}
break;
case NSStreamEventErrorOccurred:
NSLog(#"Can't connect to server");
break;
case NSStreamEventEndEncountered:
[stream close];
[stream removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
break;
default:
NSLog(#"Unknown event");
}
[super stream:stream handleEvent:event];
}
#end
I would highly appreciate if there is any one with an example of playing buffers returned from a socket server into audio queue so that I can be able to listen to sound as it comes from the socket server.
Thanks
Your code seems to be asking for a kAudioUnitSubType_VoiceProcessingIO audio unit. But kAudioUnitSubType_RemoteIO would be a more suitable iOS audio unit for just playing buffers of audio samples.
Also, your code does not seem to first select an appropriate audio session category and activate it before playing audio. See Apple's documentation for doing this: https://developer.apple.com/library/content/documentation/Audio/Conceptual/AudioSessionProgrammingGuide/Introduction/Introduction.html

How to find the recently viewed records using objective C

Im working on recently viewed functionality, I have to create one model class of History. In history i have FileName,Title,ActivityName. Im using 2 methods to store the filename and title.
History.h
-(BOOL)history:(NSString *)sFile activityName:(NSString *)activityName title:(NSString *)title GID_ID:(NSString *)dataGID_ID;
-(id)getHistoryInstance;
-(BOOL)addNewHistory:(Histrory *)history;
-(NSMutableArray *)recentlyViewedDict;
History.m
+(Histrory *)getSharedInstance{
if (!sharedInstance) {
sharedInstance = [[super allocWithZone:NULL]init];
[sharedInstance recentlyViewedDict];
}
return sharedInstance;
}
-(NSMutableArray *)recentlyViewedDict {
return recentlyViewedDict;
}
-(BOOL)history:(NSString *)sFile activityName:(NSString *)activityName title:(NSString *)title GID_ID:(NSString *)dataGID_ID
{
_sFile = sFile;
_titleName = title ;
_dataGID_ID = dataGID_ID;
_activityName = activityName;
[self performSelector:#selector(addNewHistory:) withObject:self afterDelay:0.1];
return YES;
}
-(id)getHistoryInstance {
NSLog(#"RECENTLY VIEWED DICY %#", recentlyViewedDict);
if ([recentlyViewedDict count] == 0) {
recentlyViewedDict = [[NSMutableArray alloc]init];
}
else {
return recentlyViewedDict;
}
return recentlyViewedDict;
}
-(BOOL)addNewHistory:(Histrory *)his {
BOOL val = false;
recentlyViewedDict = [self getHistoryInstance];
if ([recentlyViewedDict count] != 0) {
for (int i = 0 ; i < [recentlyViewedDict count]; i++) {
Histrory *one = (Histrory *) recentlyViewedDict[i];
if ([his.getData_GIDID isEqualToString:one.getData_GIDID]) {
val = false;
[recentlyViewedDict removeObjectAtIndex:i];
}
}
}
else {
NSString *addData = [NSString stringWithFormat:#"%#,%#,%#,%#",his.getsFile,his.getTitle,his.getAcivityName,his.getData_GIDID];
[recentlyViewedDict addObject:addData];
}
if ([recentlyViewedDict count] > 11) {
[recentlyViewedDict removeObjectAtIndex:0];
}
NSLog(#"Recently Viewed %#", recentlyViewedDict);
return YES;
}
In this every time i have to store only 1 file.I want to store at least 10 records, if 11 record inserting then 1 record will be deleted using below method.
if ([recentlyViewedDict count] > 11) {
[recentlyViewedDict removeObjectAtIndex:0];
}
How can i store each time recent record.
ContactDetails.m
historyObj = [[Histrory alloc]init];
[historyObj history:sFile activityName:activityName title:contactName GID_ID:contactGID_ID];
Im calling this method every time to visit details classes. this method works only one time like, it will store only first record, you are visited second time it is showing nil value. How can store the multiple detail classes to store each record.thanks in advance.

EXC_BAD_ACCESS Error for type NSString

I'm new to this mac application development.
The app is working fine for some data and the app crashes for few entries.
-(void)presentClientsss
{
[productVendorTextField setStringValue:[NSString stringWithFormat:#"%#", [[popUpVendor selectedItem] title]]];
NSMenuItem *tempMenuItem = [popUpVendor selectedItem];
NSString *selectedItemTitle = [tempMenuItem title];
for (int k = 0; k < [appDelegate.vendorInfoArr count]; k++)
{
VendorInfo *tempCustomerInfoModel = [appDelegate.vendorInfoArr objectAtIndex:k];
if ([tempCustomerInfoModel.vendorName isEqualToString:selectedItemTitle])
{
oldVendorIde = [NSString stringWithFormat:#"%ld", tempCustomerInfoModel.rowId];
NSLog(#"Selected RowID = %#",oldVendorIde);
break;
}
}
}
I'm sending the oldVendorIdestring to next method.
- (ItemModel *)itemNodelWithAttributes {
isProductIdExist = NO;
if ([senderInfo isEqualToString:#"nP"]) {
for (int i = 0; i < [appDelegate.itemsArr count]; i++) {
ItemModel *tempIM = [appDelegate.itemsArr objectAtIndex:i];
if ([tempIM.productId isEqualToString:[[productIdTextField stringValue] uppercaseString]]) {
isProductIdExist = YES;
break;
}
}
}
if ([senderInfo isEqualToString:#"eP"]) {
for (int i = 0; i < [appDelegate.itemsArr count]; i++) {
ItemModel *tempIM = [appDelegate.itemsArr objectAtIndex:i];
if (tempIM.itemId == itemIdentity) {
if ([tempIM.productId isEqualToString:[[productIdTextField stringValue] uppercaseString]]) {
isProductIdExist = NO;
}
}
else if ([tempIM.productId isEqualToString:[[productIdTextField stringValue] uppercaseString]]) {
isProductIdExist = YES;
}
}
}
int tempItemExists = [self saveProductImage:[[productIdTextField stringValue] uppercaseString]];
NSLog(#"oldVendorIde =%#",oldVendorIde);
ItemModel *iM = [[ItemModel alloc] initWithItemId:itemIdentity defaultItemMinimumValue:[productMinValueTextField floatValue] staticItemPrice:[productPriceTextField doubleValue] dynamicItemQuantity:[productCurrentStockTextField doubleValue] staticItemDescription:[productDescriptionTextField stringValue] prodId:[[productIdTextField stringValue] uppercaseString] itemVendor:oldVendorIde itemImgExists:tempItemExists stockAvailable:0 itemNotes:[notesTextField string] BarcodeDesc:[BarcodeDescTextView stringValue]];
return iM;
}
In this method the same oldVendorIde is working fine for some data and some time it gets crashed at this point.
The oldVendorIde sometime doesnot get any value in itemNodelWithAttributes method and the app crashes at that point.
Can Sone help me to solve the issue.. Thanks in advance..
The text from a UITextField is accessed through the text property ([productIdTextField text]), not through stringValue.

How do I parse through an array of objects in Objective-C?

Coming from C++, here's my question :
I have created objects of this type :
Size *one = [[Size alloc] initWithX: 3 andY: 1];
Size *two = [[Size alloc] initWithX: 4 andY: 7];
// etc...
Size *thirtythree = [[Size alloc] initWithX: 5 andY: 9];
( with a #property int x; & #property int y; for each object.. )
that I have stored in an array as follows :
NSArray *arrayOfSizes;
arrayOfSizes = [NSArray arrayWithObjects:one,two,three,four,five,six,
seven,eight,nine,ten,eleven,twelve,thirteen,
fourteen,fifteen,sixteen,seventeen,eighteen,
nineteen,twenty,twentyone,twentytwo,
twentythree,twentyfour,twentyfive,twentysix,
twentyseven,twentyeight,twentynine,thirty,
thirtyone,thirtytwo,thirtythree nil];
now I have a single object of type :
Myobject *myObject = [[Myobject alloc] initWithX: 5 andY: 3];
that also has a #property int x; & #property int y; ...
and I want to compare its values to the values of the objects found in the array, until I find an array object of similar values.. But I don't know how to do that in Obj-C. (in c++ I would simply use a vector v; with v.size(); and v[x]; ..etc... I suppose..)
here's what I'm looking for.. :)
while( !wholeOfArrayOfSizesChecked && !found)
{
if ( // x & y of object in array is equal to x & y of myObject )
{
found = YES;
}
else if( // whole of array checked)
{
wholeOfArrayOfSizesChecked = YES;
}
else
{
//move on to the next object of the array..
}
}
Thanks in advance for any help!
Well, you could just use fast enumeration on the array. Something like this:
Myobject *myObject = [[Myobject alloc] initWithX: 5 andY: 3];
for (Size *s in arrayOfSizes)
{
if (s.x == myObject.x && s.y == myObject.y)
{
// Found one
// Do something useful...
break;
}
}
Another one:
NSUInteger index = [arrayOfSizes indexOfObjectPassingTest:
^BOOL(Size *s, NSUInteger idx, BOOL *stop)
{
return (s.x == myObject.x) && (s.y == myObject.y);
}
];
if (index != NSNotFound) {
id object = [arrayOfSizes objectAtIndex:index];
}
Just to use your given structure. There are smarter ways of doing it though :)
wholeOfArrayOfSizesChecked = NO;
int currObj = 0
while( !wholeOfArrayOfSizesChecked && !found)
{
Size *current = (Size *)[arrayOfSizes objectAtIndex:i];
if (myObject.x == current.x && myObject.y == current.y)
{
found = YES;
}
else if(currObj == [arrayOfSizes count] -1 )
{
wholeOfArrayOfSizesChecked = YES;
}
else
{
currObj++;
}
}
Try something like this:
for (int i = 0; i < [arrayOfSizes size]; i++)
{
Size *current = (Size *)[arrayOfSizes objectAtIndex:i];
if (myObject.x == current.x && myObject.y == current.y)
{
// found
break;
}
}
How'bout a for-in loop?
for (Size *item in array) {
// compare 'item' to myObject
if (/* equal condition here */) break;
}
-(BOOL) isSize:(Size*)size equalToMyObject:(MyObject*)object{
return (size.x == object.x) && (size.y == object.y);
}
//In some method where you are checking it:
for (Size* size in arrayOfSizes){
if ([self isSize:size equalToMyObject:myObject]){
//You found it! They're equal!
break;
}
}

How to correctly measure NSInputStream data rate

I'm trying to measure actual transfer speed during ftp download, download itself is working, streams are hooked up in run loop. Measurment is done in NSStreamEventHasBytesAvailable using CFTimeGetCurrent on event start and at the end, after data is written to file elapsed time is computed with (double)previousTimestamp-CFAbsoluteTimeGetCurrent, but the time I get is absolutely unreasonable. Tested on simulator and device, can anyone enlighten me?
code:
switch (eventCode) {
case NSStreamEventOpenCompleted: {
} break;
case NSStreamEventHasBytesAvailable: {
if ([[self.fileStream propertyForKey:NSStreamFileCurrentOffsetKey] intValue]==0) {
previousTimestamp = CFAbsoluteTimeGetCurrent();
}
NSInteger bytesRead;
uint8_t buffer[32768];
bytesRead = [self.networkStream read:buffer maxLength:sizeof(buffer)];
if (bytesRead == -1)
{
[self _stopReceiveWithStatus:#"Err"];
}
else if (bytesRead == 0)
{
[self _stopReceiveWithStatus:nil];
}
else
{
[self completition:bytesRead];
NSInteger bytesWritten;
NSInteger bytesWrittenSoFar;
// Write to the file.
bytesWrittenSoFar = 0;
do {
bytesWritten = [self.fileStream write:&buffer[bytesWrittenSoFar] maxLength:bytesRead - bytesWrittenSoFar];
assert(bytesWritten != 0);
if (bytesWritten == -1) {
[self _stopReceiveWithStatus:#"File err"];
break;
} else {
bytesWrittenSoFar += bytesWritten;
}
} while (bytesWrittenSoFar != bytesRead);
[self downloadSpeedSave:bytesRead :previousTimestamp-CFAbsoluteTimeGetCurrent()];
previousTimestamp = CFAbsoluteTimeGetCurrent();
An alternative that I have used is to use the time.h and c routines to capture time.
http://www.cplusplus.com/reference/clibrary/ctime/time/
Another good link on SO
iPhone: How to get current milliseconds?