What is the right way to "synchronize" function which mutates shared state using Mutex? Using Mutex within a coroutine started with launch() or async() works as expected, but if I start a coroutine with runBlocking() the thread looks like blocked (locked) for a very long time. The thing is that the function might be called from multiple threads and I cannot solve this with thread confinement. What's the right way to use Mutex in this scenario?
The right way is to design your software in so that avoids usage of Mutex and other forms of shared mutable state altogether. If you have some resource or data structure that needs to be shared you can always encapsulate this data structure inside a separate coroutine and communicate with this coroutine when you need to do anything with this data structure. This design pattern is known as an actor. An actor is a pair of a coroutine and a channel it reads incoming messages from.
The advantage of this approach is that you can communicate asynchronously with an actor. If you send a message to an actor and don't wait for a response, then you can continue the work without having to wait for an actor to finish processing your message.
You can read a bit more about actors in the guide to kotlinx.coroutines.
Related
Reading here:
https://petabridge.com/blog/akkadotnet-async-actors-using-pipeto/
The actor’s mailbox pushes a new message into the actor’s OnReceive method once the previous call to OnReceive exits.
Followed by
On a ReceiveActor, use ReceiveAsync where T is the type of message this receive handler expects. From there you can use async and await inside the actor to your hearts’ desire.
However, there is a cost associated with this. While your actor awaits any given Task, the actor will not be able to process any other messages sent to it until it finishes processing the message in its entirety. (emphasis mine)
It seems to me that I can use this blocking quality to force an Actor to be a kind of serial operation queue. Yes, if the process crashes and the messages enqueued were not persisted, that will cause those messages to be lost. Assuming that is ok however, and in my case that is desirable. Are there any other reasons not to an Actor like this?
Are there any other reasons not to an Actor like this?
Your overall question has a flaw in its premise, but the short answer is that you should absolutely use Actors in this manner.
The flaw in your question is that you are referencing a blog post that is talking about using async and PipeTo. What you seem to be missing is that all Actors work this way, whether synchronous or asynchronous, and whether using PipeTo or not!
The whole idea of an Actor (at least in Akka.Net) is built around processing messages from a mailbox one at a time (a "Serial Operation Queue" as you called it).
Imagine we are reading messages from a message queue and on receive pushing them for processing into a threadpool. There are limited number of threads, so if all the threads are busy, we'll have natural backpressure.
How this can be solved in Kotlin coroutine world? If we'll create a coroutine for each incoming message, we can very fast end up with Out of memory errors (e.g if each task requires to load some data from DB) and other issues.
Are there any mechanisms or patterns to solve this problem?
One way to solve the issue, is to create a Channel and send your data onto it. Other threads can put a consumeEach on the channel to receive data from it. The channel's capacity can be tweaked to your threading needs.
Fan-out and Fan-in examples in the coroutines docs can be helpful too.
I am working on a bug that related to an unmanaged MTA COM object. The object has Lock and Unlock methods and uses a mutex that requires the same thread that called Lock to call Unlock.
The problem is that when Lock and Unlock are called from a managed STA thread (using COM interop), the calls come into the COM object on a RPC callback thread but the callback thread that is used is not always the same for both calls. When it is not the same, the Unlock call fails because it can't unlock the mutex.
In other words:
Managed STA thread 1 -> RPC callback (thread 11) -> Lock
Managed STA thread 1 -> RPC callback (thread 12) -> Unlock -> Error
I am trying to evaluate all possible solutions before making any decisions on a fix. As such, I am trying to find out:
1) Is there is a way to prevent a RPC callback thread from being used in the first place? In my testing, if I make the calls to the object from an unmanaged STA thread, the calls seem to come in on the calling thread itself. What is different when the call is coming from .Net that necessitates the use of an RPC callback thread? Is there any way to prevent RPC callbacks from being used? (except for using an MTA calling thread)
2) If not, is there a way to force a consistent RPC callback thread to be used from the same managed STA thread?
This is by design for a free-threaded server. COM takes your word for it and allows stubs to use arbitrary RPC threads. You cannot make any assumptions about the thread identity, the RPC thread is picked from a pool and is recycled. Unfortunately it often picks the same one when the calls are sequenced so it will look like it works fine at first. But trouble starts as soon as more than one concurrent server call is made. There is no option to make it selective, a free-threaded server promises to not care. Nor could that work well in practice, it would either scale horribly or induce deadlock.
You therefore cannot use a mutex to implement locking, it has thread affinity. A semaphore is a good choice.
I'd like to pick someone's brain on this. I have a dedicated save NSManagedObjectContext and GCD queue from which I operate on it. And whenever new data comes into my app I save it on that context and merge the changes into the main context. My problem arises in telling the main thread what just happened. Right after I call save my current context is now up-to-date, but if I fire a method in the main context it's context isn't. If I wait for the NSManagedObjectContextDidSave notification, and I save three times, I now have three queued delegate calls but no way to match them to the notifications coming in. Does anyone know of a good way to get around this?
EDIT
What I ended up doing was creating a new context for each save operation and attaching a block to be called when the save notification arrived. It looks like this, http://pastie.org/2068084
From your answer to my comment above, I see that you pass along the managedObjectContext in the notification. I'm not that confident about asynchronous stuff yet, but I do think that you're violating some concurrency rule, if I correctly interpret this quote from the NSManagedObjectContext Class Reference:
Concurrency
Core Data uses thread (or serialized queue) confinement to protect managed objects and managed object contexts (see “Concurrency with Core Data”). A consequence of this is that a context assumes the default owner is the thread or queue that allocated it—this is determined by the thread that calls its init method. You should not, therefore, initialize a context on one thread then pass it to a different thread. Instead, you should pass a reference to a persistent store coordinator and have the receiving thread/queue create a new context derived from that.
I'd say, try passing along the persistent store coordinator in the notification and recreate the managed object context in the block.
I'm not sure what you mean by, " ...if I fire a method in the main context it's context isn't. If I wait for the NSManagedObjectContextDidSave..." That implies that you are not waiting until the contexts have been merged. If so, that is why you can't access the data, it's just not in the front context.
Do you call mergeChangesFromContextDidSaveNotification: from the front context after it receives the notification?
I have a number of data containers that can send a signal when there are updates in them. The structure looks similar like this:
typedef struct {
int data;
/*...*/
pthread_cond_t *onHaveUpdate;
} Container;
The onHaveUpdate is a pointer to a global condition that is shared by all the containers
In my application, I have a number of these structures and they can concurrently be updated by different threads.
Now, is it possible for me to have a thread that listens to the condition and can perform something on the container that sends the notification?
I know that this can be solved by using one thread per container but it feel like a waste of resources, but I was wondering if this can be done using only one thread for all container?
The problem is that your condition is shared by all containers, so when you send the condition that something has changed, you have no idea what has changed. Since the architecture of your application is not a 100% clear from your context, why not implement a Queue which holds events (and pointers to containers are pushed onto that Queue) and a worker thread which takes events from that waitqueue and performs the work. That way your worker queue needs to wait on the condition that the queue is filled (or run in a non aggressive while-true-fashion), and that way you can remove the condition variable from your container altogether.