NAudio asynchronous audio - naudio

I want to be able to dump sound files to the audio output asynchronously throughout the life of the application. I need to be able to request a file to be played and have it mixed in with any files already playing. The same file may be called multiple times. Is WaveMixerStream the way to go?
Is it safe/recommended to keep a player and mixer stream open for the life of the application or will that cause performance problems?
//globals
IWavePlayer _Context;
WaveMixerStream32 _Mixer;
//constructor
_Context = new DirectSoundOut();
_Mixer = new WaveMixerStream32();
_Context.Init( Instance._Mixer );
_Context.Play();
//asynchronous sound output, on demand from user interface
_Mixer.AddInputStream( sound.FileWaveStream );

There are several ways you could do this, but here's one option. Create a player and mixer open all the time. I would use the MixingSampleProvider as the mixer, passing it through a SampleToWaveProvider or SampleToWaveProvider16 as the input to the player.
To ensure that playback never stops you would either need to customise MixingSampleProvider so that Read always returns the number of samples asked for even if there are no active inputs, or add a dummy input which is a never-ending stream of silence.
Now to play a sound, just pass an ISampleProvider (e.g. AudioFileReader) into your mixer's AddMixerInput and it will play it. It auto-removes inputs when they reach the end.
An alternative is to create a new player object for each sound to play. The disadvantage is working out a way to keep the player object alive until playback stops (probably by storing in a dictionary and then Disposing and taking it out when the PlaybackStopped event fires). The advantage is that you aren't keeping the sound device open when you don't need to, and you can easily play sounds at different sample rates / number of channels.

Related

How do I know the duration of a sound in Web Audio API

It's basicly all in the title, how do I know or how can I access the duration of a soundNode in Web Audio API. I was expecting something like source.buffer.size or source.buffer.duration to be available.
The only alternative I can think of in case this is not possible to acomplish is to read the file metadata.
Assuming that you are loading audio, when you decode it with context.decodeAudioData the resulting arrayBuffer has a .duration property. This would be the same arraybuffer you use to create the source node.
You can look at the SoundJS implementation, although there are easier to follow tutorials out there too.
Hope that helps.
Good and bad news; nodes do not have length - but I bet you can achieve what you want another way.
Audio nodes are sound processors or generators. The duration of a sound processed or made by a node can change - to the computer it is all the same, lack of sound is a buffer full of zeroes instead of other values.
So, if you needed to dynamically determine the duration of a sound, you could write a node which timed 'silences' in the input audio it received.
However I suspect from your mentioning of meta data that you are loading existing sounds - in which case you should indeed use their meta data, or determine the length by loading into an audio element and requesting the length from that.

WebRTC Changing Media Streams on the Go

Now since device enumeration is present in chrome, i know i can select a device during "getUserMedia" negotiation. I was also wondering whether i could switch devices during the middle of a call (queue up a local track and switch tracks or do i have to renegotiate the stream)? I am not sure if this is something that is still blocked or now is "allowable"
I have tried to make a new track, but i can't figure out how to switch the track on the go. I know this was previously impossible, but was wondering now if it is possible?
Even i have the same requirement. I have to record the video using MediaRecorder. For this I am using navigator.getUserMedia with constraints of audio and video. You can pass the video or audio tracks dynamically by getting the available devices from navigator.mediaDevices.enumerateDevices() and attaching the respective device to constraints and calling navigator.getUserMedia with new constraints again. The point to be noted when doing this is, you have to kill the existing tracks using track.stop() method.
You can see my example here.
StreamTrack's readyState is getting changed to ended, just before playing the stream (MediaStream - MediaStreamTrack - WebRTC)
In Firefox, you can use the RTPSender object to call replaceTrack() to replace a track on the fly (with no renegotiation). This should eventually be supported by other browsers as part of the spec.
Without replaceTrack(), you can remove the old stream, add a new one, deal with onnegotiationnedded, and let the client process the change in streams.
See the replaceTrack() test in the Mozilla source: https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/replaceTrack
Have you tried calling getUserMedia() when you want to change to a different device?
There's an applyConstraints() method in the Media Capture and Streams spec that makes it possible to change constraints on the fly, but it hasn't been implemented yet:
dev.w3.org/2011/webrtc/editor/getusermedia.html#the-model-sources-sinks-constraints-and-states
dev.w3.org/2011/webrtc/editor/getusermedia.html#methods-1

libspotify: sometimes created playlists get no names

I am using the latest version of the libspotify API (12.1.51). And I am coding using C# and libspotify.NET (a simple interop wrapper library for libspotify). I have built a program that creates playlists using the libspotify API. I am using the latest native Spotify client for Windows for checking my created playlists.
I have built a program that waits for a playlist name and a list of track URIs or one album URI and creates the corresponding playlist. If an album URI is supplied then all the tracks from the album will be added to the new playlist otherwise the supplied list of track URIs will be used to create the new playlist. The program waits until the callback
playlist_update_in_progress
is called and the done parameter equals true before accepting new playlists to create.
It works well except that sometimes the playlists are created with no names but do have all the tracks. I can just tell the program to create the exact same playlist that got no name and it might work the second time. It happens randomly but I have noticed that if I create about 200 playlists then maybe 5-10 playlists get no names. I can see the playlists being added in your Spotify client as the program is running and creating the playlists and randomly seeing playlists with no names. The playlists appear almost at the same time the callback is called and done equals true (not always of course, there might be a delay). I tried manually renaming the playlists in the Spotify client for Windows with no problems (the playlist name length was 0 or an empty string if you will). The native Spotify client does not even allow empty playlist names.
I use:
sp_playlistcontainer_add_new_playlist
to add a new playlist and I have tried using IntPtr and a C# string to pass to it. Tried both:
[DllImport("libspotify")]
public static extern IntPtr sp_playlistcontainer_add_new_playlist(IntPtr playlistContainerPtr, string name);
[DllImport("libspotify")]
public static extern IntPtr sp_playlistcontainer_add_new_playlist(IntPtr playlistContainerPtr, IntPtr namePtr);
And I have also tried
sp_playlist_rename
to rename the playlist several times (as some sort of fix) with no success. I do not know exactly what happens at the server when the playlists are created. Is this a bug or what is going on?
I have really tried a lot of hacks to make this work, but to no avail.
More info about the program:
First it connects to Spotify and then waits for some console input.
It checks the input for the following strings:
To create a playlist using a name and a list of tracks:
createplaylistfromtracks "Tracks playlist" spotify:track:36MuLw248uzLPtrJ6073ZR spotify:track:5WPkvx0MARhlWhXp1sJg4k spotify:track:1VrdbSFVU9wJkuDM2sWYVe spotify:track:66RG0BBwpQqHxZs06UUyeo spotify:track:0zp3uPuhnARR1XYsgg5JLV
and to create a playlist using a name and a list of tracks from an album URI:
createplaylistfromalbum "Album playlist" spotify:album:5rVwDKRKa1FjDlLofDZyRb
And then the program parses the input and creates the playlist and settings a flag Busy = true so that the console will not read any input until Busy = false, which occurs when the playlist_update_in_progress callback is called and done is true. Then it reads input again...
Any help would be highly appreciated.
We have many applications running libspotify both internally and externally and we've had no problems like this with the library, which would suggest this isn't a bug in libspotify.
A few pointers that might cause this to happen:
Make sure you're managing threads properly. When you get a notify_main_thread() call (which arrives on some background thread), you must asynchronously call sp_sesion_process_events() from the main thread, so you need to make sure you're marshalling this over to the main thread properly. If this process isn't working properly, stuff will start acting weird.
Since you're doing many operations, make sure libspotify has time to synchronise all of your changes with the service. The playlist_update_in_progress is designed to notify you when multiple mutations are happening to the playlist, and is typically useful when changes are coming in from elsewhere. Instead, playlist_state_changed is what you need for listening to changes like that (typically, after a rename, I believe the playlists goes into a "loading" state while the change is applied, then back out (that is, sp_playlist_is_loading returns false again) once it's done.
Also, when you're exiting your application, make sure you perform a sp_session_logout() call and wait until the logged_out callback is fired before exiting. If you don't, some changes might not be synchronised with the service.

Best multithreading approach in Objective C?

I'm developing an iPad-app and I'm currently struggling with finding the best approach to multithreading. Let me illustrate this with a simplified example:
I have a view with 2 subviews, a directory picker and a gallery with thumbnails of all the images in the selected directory. Since 'downloading' and generating these thumbnails can take quite a while I need multithreading so the interaction and updating of the view doesn't get blocked.
This is what I already tried:
[self performSelectorInBackground:#selector(displayThumbnails:) withObject:currentFolder];
This worked fine because the users interactions didn't get blocked, however it miserably fails when the user taps on another folder while the first folder is still loading. Two threads are trying to access the same view and variables which results in messing up each others proper execution. When the users taps another folder, the displayThumbnails of the currently loading folder should get aborted. I didn't find any way to do this..
NSThreads
I tried this but struggled with almost the same problems as with the first method, I didn't find a (easy) way to cancel the ongoing method. (Yes, I know about [aThread cancel] but didn't find a way to 'resume' the thread). Maybe I should subclass NSThread and implement my own isRunning etc methods? But isn't there any better way or a third (or even fourth and fifth) option I'm overlooking?
I think this is a fairly simple example and I think there is perhaps a better solution without subclassing NSThread. So, what would you do? Your opinions please!
NSOperationQueue should work well for this task.
Another option would be plain GCD, however, if you've never worked with it, NSOperationQueue is probably the better choice since it pretty much automatically guides you to implementing things "the right way", has obvious ways for cancellation, etc.
You want to use Concurrent NSOperations to download and process images in the background. These would be managed by an NSOperationsQueue. Essentially these operations would be configured to fetch one image per operation, process it, save it in the file system, then message back to the main app in the main thread that the image is available.
There are several projects on github that you can look at that show how to do this - just search github using "Concurrent" or "NSOperation".
iOS has a really nice facility for doing background work. Grand Central Dispatch (GCD) and Blocks, but those don't let you have an object using delegate callbacks - thus NSOperation.
So you need to read up on blocks, GCD, and then look at some open source Concurrent NSOperations code. Using Concurrent NSOperations is not as simple as using blocks.
If I had this problem, I would probably go for an approach like this:
a single thread that will load the images, and causes the main thread to display results (I'm not a big fan of having thread mess around with GUI objects)
when a new directory is requested... well, it depends on how you want to manage things. Basically, a standard queue construct (condition variable and array) could be used for the main thread to tell the thread that "this directory will be needed" by passing it the path name; the thread will check the queue even when it's loading images (like after every image or so), and switch to the new directory whenever one shows up
you could make a directory-reader object that keeps all the state, and store this indexed by the path into a dictionary. When a new directory is requested, check that dictionary first, and only create a new object if there's none for this directory. That way, partially loaded directories would stick around until they are needed again, and can continue to load instead of having to start from scratch.
Pseudocode for the thread:
while (forever)
new element = nil
if we have an active directory loader
tell directory loader to load one image
if false then make directory loader inactive
lock queue condition
if queue has elements
new element = retrieve LAST element (we aren't interested in the others)
empty queue
unlock with status "empty"
else
unlock queue
else
lock queue on condition "has elements"
new element = retrieve last element
empty queue
unlock with status "empty"
if new element != nil
if directory loader for new path does not exist
setup new directory loader for new path
store in dictionary
make it the "active" one
else
make the current one the "active"
As for the directory loader, it might look something like this:
read one image:
if there are still images to read:
read, process and store one
return true
else
performSelectorOnMainThread with an "update GUI" method and the image list as parameter
return false;
This is just a quick sketch; there's some code duplication in the thread, and the way I wrote it will only update the GUI after all images have been read, instead of making them appear as we read them. You'll have to copy the current image list, or add synchronization if you want to do that.

Designing asynchronous procedure for downloading & displaying images

I have a JSON feed which contains URLs for images. I am using NSURLConnection to download the JSON feed extract the URLs. I want to download all the images asynchronously. I subclassed UIImage and sent that class a URL which it downloads, one image at at a time, in an asynchronous manner.
First, is that a good way to do it? Second, I'd like to show four images at a time. Shouldn't I download every set of four together instead of downloading one by one?
My second concern is that I have twoNSURLConnections. That's probably bad. Should I use the very same NSURLConnection to download the JSON feed and at the same time get the image?
I am trying to display four images at a time, with a next button that displays the next four on the next line.
I am not sure UIImage is expected to be subclassed. If you need good design practice, you should have a look at TopPaid
sample code from Apple, as it is showing how to properly download a feed and then asynchronously download images. Take a close look at the IconDownloader class from this project, which is a class handling image downloading, and notifying its delegate when it's finished.
As far as I know there is no problem with having multiple NSURLConnections at a time. You might run into trouble if the number of connections becomes very large, because this could saturate the number of open file descriptors allowed on iPhone, or more likely create a memory warning. In your case if you only have 2 connections, you don't have any problem.