I usually write imperative code on Java/Spring MVC, but now my team implement project on WebFlux. I tried to research the topic, but I can't find the answer to the question about locks.
It's normal when we have code that should always be executed by only one thread, or that has locks by some condition (for example, the code should not be executed concurrently for the same entity). These locks can be distributed, for example, through a Redis.
But how is this problem solved in Project Reactor? As far as I understand, it would be a bad idea to use a synchronized block, or ReentrantLock, because they will block threads while we avoid blocking.
It turns out that we need to design the application in such a way that there is no need for locks. Which is not always possible.
Or is there any solution? I will be grateful for any information.
There is no official implementation, here are some resources for reference.
How to trigger Mono execution after another Mono terminates
https://github.com/chenggangpro/reactive-lock
Related
I'm currently looking at having a KMM application backed by SQLdelight for all domain-related operations.
SQLdelight seems to provide really nice interfaces, however it seems like all the write calls (insert/update/delete) are implemented using blocking calls, so I'm worried that it would hurt the responsiveness of the app by blocking the main thread a lot.
Is there a recommended way to perform such operations without blocking the main thread?
The app would have to work on iOS as well, so I can't afford freezing too much.
A bit late to answer but it might be useful for others:
You should use wiwthContext(Dispatchers.Default) assuming you are using the native-mt version of coroutine libraries. That allow you to ensure insert/update/delete are not executed on the main thread.
You also have the possibility of using sqldelight coroutine-extensions library to return a flow from your queries to observe changes in your database.
New to NServiceBus (4.7.5) and just implemented an NSB host.exe hosted service (implementing IWantToRunWhenBusStartsAndStops) that detects changes to database tables and notifies subscribing web apps by publishing events, e.g. "CustomerDataWasUpdatedEvent". In the future we will perform the actual update through messagehandlers receiving commands obviously, but at the moment this publishing service just polls the database etc.
It all works well, however, approaching production, I noticed that David Boike, in his latest edition of "Learning NServiceBus", states that classes implementing
IWantToRunWhenBusStartsAndStops are really mostly for development and rarely used in production. I set up my database change detection in the Start method and it works nicely, does anyone know why this is discouraged?
Here is the comment in the actual book:
https://books.google.se/books?id=rvpzBgAAQBAJ&pg=PA110&lpg=PA110&dq=nservicebus+iwanttorunwhenbusstartsandstops+in+production+david+boike&source=bl&ots=U6sNII0nm3&sig=qIXffOVFhcy-_3qDnSExRpwRlD4&hl=sv&sa=X&ei=lHWRVc2_BKrWywPB65fIBw&ved=0CBsQ6AEwAA#v=onepage&q=nservicebus%20iwanttorunwhenbusstartsandstops%20in%20production%20david%20boike&f=false
The actual quote is:
...it isn't common to have widespread use of in a production system.
Uncommon is not the same thing as discouraged.
That said I do think there is intent here by the author to highlight the fact that further up the page they assert that this is not a good place to be doing lots of coding, as an unhandled exception can cause the whole process to fail.
The author actually does go on to mention a possible use case for when you may want to load a resource(s) to do work within the handler.
Ok, maybe it's just this scenario we have that is a bit uncommon
Agreed - there is nothing fundamentally wrong with your approach. I recently did the same thing as you for wiring up SqlDependency to listen for database events and then publish a message as a result. In these scenarios there is literally nothing else you can do other than to use IWantToRunAtStatup.
Also, David himself often trawls the nservicebus tag, maybe he'll provide a more definitive answer than mine.
I'll copy the answer I gave in the Particular Software Google Group...
I'll quote myself directly here:
An implementation of IWantToRunWhenBusStartsAndStops is a great place to create a quick interface in order to test messages during debugging by allowing you to send messages based on the console input. Apart from this, it isn't common to have widespread use of them in a production system. One possible production use case will be to provision a resource needed by the endpoint at startup and then tear it down when the endpoint stops.
I think if I could add a little bit of emphasis it would be to "widespread use". I'm not trying to say you won't/can't have an IWantToRunWhenBusStartsAndStops in production code or that avoiding them is a best practice. I am trying to say that having a ton of them is probably a code smell.
Above that paragraph in the book, I warn about IWantToRunWhenBusStartsAndStops not having any ambient transactions or try/catch stuff going on. THAT is really the key part. If you end up throwing an exception in an IWantToRunWhenBusStartsAndStops, tyou can run into big problems. If you use something like a .NET Timer and then throw an exception, you can crash your process!
Let me tell you how I screwed up on this in my first-ever NServiceBus system. The system (still in use today, from what I hear) is responsible for ingesting more than 3000 RSS feeds (probably a lot more than that now) into a CMS. So processing each feed, breaking it up into items, resizing images, encoding attached video for mobile ... all those things were handled in NServiceBus message handlers, which was scaled out to multiple servers, and that was all fantastic.
The problem was the scheduler. I implemented that as an IWantToRunWhenBusStartsAndStops (well, actually IWantToRunAtStartup at that time) and it quickly turned into a mess. I kept the whole table worth of feed information in memory so that I could calculate when to fire off the next ProcessFeed command. I was using the .NET Timer class, and IIRC, I eventually had to use threading primitives like ManualResetEvent in order to coordinate the activity. And because I was using .NET Timer, if the scheduler threw an exception, that endpoint failed and had to restart. Lots of weird edge cases and it was always a quagmire of bugs. Plus, this was now a singleton "commander app" so while the feed/item processors could be scaled out, the scheduler could not.
As I got more experienced with NServiceBus, I realized that each feed should have been a saga, starting from a FeedCreated event, controlled through PauseProcessing and ResumeProcessing commands, using timeouts to control the next processing time, and finally (perhaps) ended via a FeedRemoved event. This would have been MUCH more straightforward and everything would have executed inside transactionally-controlled message handlers.
That experience led me to be a little bit distrustful/skeptical of IWantToRunWhenBusStartsAndStops. Not saying it's bad, just something to be aware of. Always be prepared to consider if what you're trying to do couldn't be better accomplished in another way.
I have a loop of several hundred items which need to be processed.
Each item is processed by conditionally setting a global SQLConnection where upon the item is processed using this SQLConnection as part of the processing.
For this reason it is vital that none of these items is allowed to be processed in parallel.
I appreciate that this is not good design and I hope to rectify it as soon as is practical.
However it would seem that despite my best efforts, this code is experiencing some form of multi-threading. Somehow one of these tasks has thrown an exception.
This exception is the violation of a foreign key constraint, but indicates that it was operating against a SQLConnection which it has no business connecting to.
Naturally I have concerns about this, however to my knowledge there is no multi threading code in this app.
I wonder Is it possible to introduce multi threading without explicitly creating new threads
EDIT:
VB.Net 3.5SP1
Console App + Class Libraries
Occasionally Calls out to web services
Makes SQL calls
not much of anything else. No Winforms, no WPF.
Yes - using System.Timers.Timer and/or System.Threading.Timer can cause the effect your describing. Whenever a timer ticks a new work item is queued in the ThreadPool - so essentially you have a multi threading program without explicitly creating new threads.
If the timer is AutoReset (remains enabled after elapsed has been called) you might cause another call to the same handler concurrently.
In addition to the others that have been mentioned: parallel extensions (PLINQ and task parallel library).
Alternatively tasks (ie Task objects) are not called threads, but are. Tasks are commonly found near lambda expressions, check if you have any.
Oh, and async sockets too and all the other async IOs.
BUT:
Instead of trying to avoid multithreading at all cost, wouldn't it be easier to lock ? Sorry if the question is naive, I may miss something.
Could it be that your code is called from a 3rd party library. By using events another library can call your code - from as many threads as it like.
I suggest you check the code that invoke the code that changes and make sure that there's no suspicious calls to your code.
Is there a way to execute a whole objective-c function atomic?
As far as I know, using synchronized only protects a specific peace of code from being executed on multiple threads at the same time. But what I want is stop ALL other threads from doing ANYTHING, as long as I execute the function.
There is a wealth of info in the Threading Programming Guide. It specifically mentions to avoid synchronization (which is funny, cause you cant sometimes) but they offer some suggestions around the problem.
You will have serious problems with your design if you start running your software on multicore. It is a VERY expensive operation to stop all cores from running to run your bit of code. Mutexes, semaphores, run loop events, and atomic operations are the way to go.
Nope. Can't do that.
Or, well, you probably could if you dipped deep enough in the Mach APIs (on Mac OS X anyway).
But you shouldn't do that.
Why do you think you want to do that?
What's the difference between a monitor and a lock?
If a lock is simply an implementation of mutual exclusion, then is a monitor simply a way of making use of the waiting time inbetween method executions?
A good explanation would be really helpful thanks....
regards
For example in C# .NET a lock statement is equivalent to:
Monitor.Enter(object);
try
{
// Your code here...
}
finally
{
Monitor.Exit(object);
}
However, keep in mind that Monitor can also Wait() and Pulse(), which are often useful in complex multithreading situations.
Edit:
In later versions of the .NET framework, this was changed to:
bool lockTaken = false;
try
{
Monitor.Enter(object, ref lockTaken);
// Your code here...
}
finally
{
if (lockTaken)
{
Monitor.Exit(object);
}
}
They're related. For example, in C# the lock statement is a simple try-finally wrapper around entering a Monitor and exiting one when done.
Monitors are compiler-assisted "semi-automatic" locks. They allow one to declare synchronized methods on classes, etc. This is just a different approach to providing mutual exclusion. I found this book to be the most thorough explanation of the concepts, even though it's mostly geared towards OS developers.
A lock ensures mutual exclusion.
A monitor associates the data to be protected and the mutual exclusion and synchronization primitives required to protect accesses to the data.
Synchronization is used e.g. when you need one thread to wait until an event occurs (e.g., wait until another thread places an item in a queue).
Monitors is a programming-language construct that does the same thing as semiphores/locks, but Monitors control the shared data by synchronizing at run time. In contrast, locks protect the shared data by just "spinning" which can lead to poor CPU utilization.
There is no difference, lock generates Monitor.Enter and Monitor.Exit within a try/finally block. Using Monitor over lock allows you to fine tune because it has Pulse and PulseAll. You can also have alternate processing should you be unable to acquire the lock with TryEnter.
Monitor is the concept and Lock is the actual implementation.
As far as I have researched so far, monitor is a set of principles for thread synchronization, while locks are, along with "thread cooperation" facilities like wait and notify, the way monitors are implemented in Java. So effectively, if we try to form the exact relationship between the two notions, locks are one part of the implementation of monitors (the other being wait and notify mechanisms). Please correct me if I'm wrong, but I would really appreciate if the correction is very specific.
Lock focus on only mutual exculsion, but
Moniter provides mutual exclusion automatically.
So we don't need to worry of using mutual exclusion in Monitor.
Instead of ME, we need to consern of sycronzing only when we do programming.
Moniter provides more systematical way of programming.
It, therefor, is more advanced one.