What is a good case/example for using the ScheduledDisposable in Reactive Rx
I like the using the CompositeDisposable and SerialDisposable, but would you need the ScheduledDisposable.
The logic of using the Rx disposables is that code that performs some sort of set up operation can return an IDisposable that anonymously contains the code that will do the associated clean up at a later stage. If this pattern is used consistently then you can compose together many disposables to perform a single clean up operation without any specific knowledge of what is being cleaned up.
The problem is that if that clean up code needs to run on a certain thread then you need some way for Dispose called on one thread to be marshalled to required thread - and that's where ScheduledDisposable comes in.
The primary example is the SubscribeOn extension method which uses ScheduledDisposable to ensure that the "unsubscribe" (i.e. the Dispose) is run on the same IScheduler that the Subscribe was run on.
This is important for the FromEventPattern extension method, for example, that attaches to and detaches from event handlers which must happen on the UI thread.
Here's an example of where you might use ScheduledDisposable directly:
var frm = new SomeForm();
frm.Text = "Operation Started.";
var sd = new ScheduledDisposable(
new ControlScheduler(frm),
Disposable.Create(() =>
frm.Text = "Operation Completed."));
Scheduler.ThreadPool.Schedule(() =>
{
// Long-running task
Thread.Sleep(2000);
sd.Dispose();
});
A little contrived, but it should show a reasonable example of how you'd use ScheduledDisposable.
Related
Background
I have a legacy application where I need to return a List<Item>
There are many different Service classes each belonging to an ItemType.
Each service class calls a few different backend APIs and collects the responses to create a SubType of the Item.
So we can say, each service class implementation returns an Item
All backend API access code is using WebClient which returns Mono of some type, and I can zip all Mono within the service to create an Item
The user should be able to look up many different types of items in one call. This requires many backend calls
So for performance sake, I wanted to make this all asynchronous using reactor, so I introduced Spring Reactive code.
Problem
If my endpoint had to return Flux<Item> then this code work fine,
But this is some service code which is used by other legacy code caller.
So eventually I want to return the List<Item> but When I try to convert my Flux into the List I get an error
"message": "block()/blockFirst()/blockLast() are blocking,
which is not supported in thread reactor-http-nio-3",
Here is the service, which is calling a few other service classes.
Flux<Item> itemFlux = Flux.fromIterable(searchRequestByItemType.entrySet())
.flatMap(e ->
getService(e.getKey()).searchItems(e.getValue()))
.subscribeOn(Schedulers.boundedElastic());
Mono<List<Item>> listMono = itemFlux
.collectList()
.block(); //This line throws error
Here is what the above service is calling
default Flux<Item> searchItems(List<SingleItemSearchRequest> requests) {
return Flux.fromIterable(requests)
.flatMap(this::searchItem)
.subscribeOn(Schedulers.boundedElastic());
}
Here is what a single-item search is which is used by above
public Mono<Item> searchItem(SingleItemSearchRequest sisr) {
return Mono.zip(backendApi.getItemANameApi(sisr.getItemIdentifiers().getItemId()),
sisr.isAddXXXDetails()
?backendApi.getItemAXXXApi(sisr.getItemIdentifiers().getItemId())
:Mono.empty(),
sisr.isAddYYYDetails()
?backendApi.getItemAYYYApi(sisr.getItemIdentifiers().getItemId())
:Mono.empty())
.map(tuple3 -> Item.builder()
.name(tuple3.getT1())
.xxxDetails(tuple3.getT2())
.yyyDetails(tuple3.getT3())
.build()
);
}
Sample project to replicate the problem..
https://github.com/mps-learning/spring-reactive-example
I’m new to spring reactor, feel free to pinpoint ALL errors in the code.
UPDATE
As per Patrick Hooijer Bonus suggestion, updating the Mono.zip entries to always contain some default.
#Override
public Mono<Item> searchItem(SingleItemSearchRequest sisr) {
System.out.println("\t\tInside " + supportedItem() + " searchItem with thread " + Thread.currentThread().toString());
//TODO: how to make these XXX YYY calls conditionals In clear way?
return Mono.zip(getNameDetails(sisr).defaultIfEmpty("Default Name"),
getXXXDetails(sisr).defaultIfEmpty("Default XXX Details"),
getYYYDetails(sisr).defaultIfEmpty("Default YYY Details"))
.map(tuple3 -> Item.builder()
.name(tuple3.getT1())
.xxxDetails(tuple3.getT2())
.yyyDetails(tuple3.getT3())
.build()
);
}
private Mono<String> getNameDetails(SingleItemSearchRequest sisr) {
return mockBackendApi.getItemCNameApi(sisr.getItemIdentifiers().getItemId());
}
private Mono<String> getYYYDetails(SingleItemSearchRequest sisr) {
return sisr.isAddYYYDetails()
? mockBackendApi.getItemCYYYApi(sisr.getItemIdentifiers().getItemId())
: Mono.empty();
}
private Mono<String> getXXXDetails(SingleItemSearchRequest sisr) {
return sisr.isAddXXXDetails()
? mockBackendApi.getItemCXXXApi(sisr.getItemIdentifiers().getItemId())
: Mono.empty();
}
Edit: Below answer does not solve the issue, but it contains useful information about Thread switching. It does not work because .block() is no problem for non-blocking Schedulers if it's used to switch to synchronous code.
This is because the block operator inherited the reactor-http-nio-3 Thread from backendApi.getItemANameApi (or one of the other calls in Mono.zip), which is non-blocking.
Most operators continue working on the Thread on which the previous operator executed, this is because the Thread is linked to the emitted item. There are two groups of operators where the Thread of the output item differs from the input:
flatMap, concatMap, zip, etc: Operators that emit items from other Publishers will keep the Thread link they received from this inner Publisher, not from the input.
Time based operators like delayElements, interval, buffer(Duration), etc. will schedule their tasks on the provided Scheduler, or Schedulers.parallel() if none provided. The emitted items will then be linked to the Thread the task was scheduled on.
In your case, Mono.zip emits items from backendApi.getItemANameApi linked to reactor-http-nio-3, which gets propagated downstream, goes outside both the flatMap in searchItems and in itemFlux, until it reaches your block operator.
You can solve this by placing a .publishOn(Schedulers.boundedElastic()), either in searchItem, searchItems or itemFlux. This will cause the item to switch to a Thread in the provided Scheduler.
Bonus: Since you requested to pinpoint errors: Your Mono.zip will not work if sisr.isAddXXXDetails() is false, as Mono.zip discards any element it could not zip. Since you return a Mono.empty() in that case, no items can be zipped and it will return an empty Mono.
If we have only spring-boot-starter-webflux defined as application dependency, then springbok spin up a `Netty server.
One is not expected to block() in a reactive application using a non-blocking server.
However, once we add spring-boot-starter-web dependency then even with the presence of spring-boot-starter-webflux, springboot spinup a tomcat server. Which is a thread-per-request model and is expected to have blocking calls
So to solve my problem, all I had to do above is, to add spring-boot-starter-web dependency in pom.xml. After that applications is started in Tomcat
with timcat .collectList().block() works in Controller class to return the List<Item>.
Whereas with the Netty server I could return only Flux<Item> not List<Item>, which is expected.
So Rx is single-threaded by default. In the case of the Sample operator it's not. In my ASPNetCore application, I want to be able to have Subscribe run on the main thread (single thread). I tried using ObserveOn passing in SynchronizationContext.Current, but it's null in ASPNetCore.
A possible solution will be to push all items into BlockingCollection then read from it, but that kinda adds another layer of complexity.
Console.WriteLine ($"Main Thread: {Thread.CurrentThread.ManagedThreadId}");
var source = Observable
.Interval (TimeSpan.FromSeconds (1))
.Sample (TimeSpan.FromSeconds (2));
source
.Subscribe (
data => {
Console.WriteLine ($"{Thread.CurrentThread.ManagedThreadId}"); // different thread
// Here each onNext is invoked asynchronously
// But I have a shared resource that I would like to invoke synchronously.
}
);
Any recommendations will be greatly appreciated :).
If you want to schedule it on the main thread, you should specify a scheduler for your operators.
var source = Observable
.Interval(TimeSpan.FromSeconds(1))
.Sample(TimeSpan.FromSeconds(2), Scheduler.CurrentThread);
Please note that this is blocking (since it's doing work on the main thread) - which is similar to your enumerable sample.
Also see this answer by #Enigmativity.
Use Case
Android fragment that consumes items of T from a ReceiveChannel<T>. Once consumed, the Ts should be removed from the ReceiveChannel<T>.
I need a ReceiveChannel<T> that supports consuming items from it. It should function as a FIFO queue.
I currently attach to the channel from my UI like:
launch(uiJob) { channel.consumeEach{ /** ... */ } }
I detach by calling uiJob.cancel().
Desired behavior:
val channel = Channel<Int>(UNLIMITED)
channel.send(1)
channel.send(2)
// ui attaches, receives `1` and `2`
channel.send(3) // ui immediately receives `3`
// ui detaches
channel.send(4)
channel.send(5)
// ui attaches, receiving `4` and `5`
Unfortunately, when I detach from the channel, the channel is closed. This causes .send(4) and .send(5) to throw exceptions because the channel is closed. I want to be able to detach from the channel and have it remain usable. How can I do this?
Channel<Int>(UNLIMITED) fits my use case perfect, except that is closes the channel when it is unsubscribed from. I want the channel to remain open. Is this possible?
Channel.consumeEach method calls Channel.consume method which has this line in documentation:
Makes sure that the given block consumes all elements from the given channel by always invoking cancel after the execution of the block.
So the solution is to simply not use consume[Each]. For example you can do:
launch(uiJob) { for (it in channel) { /** ... */ } }
You can use BroadcastChannel. However, you need to specify a limited size (such as 1), as UNLIMITED and 0 (for rendez-vous) are not supported by BroadcastChannel.
You can also use ConflatedBroadcastChannel which always gives the latest value it had to new subscribers, like LiveData is doing.
BTW, is it a big deal if you new Fragment instance receives only the latest value? If not, then just go with ConflatedBroadcastChannel. Otherwise, none of BroacastChannels may suit your use case (try it and see if you get the behavior you're looking for).
I am writing code to consume WinRT API on Windows 10 using C++ WRL (i.e. without using either C++/CX or C++/CLI). I am using GetAppListEntriesAsync() from Package class to get names of UWP applications installed on a system. GetAppListEntriesAsync() calls a callback method asynchronously. I want to wait till all such asynchronous methods have finished. Unlike C++/CX WRL doesn't have any await keyword to wait on asynchronous tasks.
Details
The following code is used to get the list.
ComPtr<ABI::Windows::Foundation::__FIAsyncOperation_1___FIVectorView_1_Windows__CApplicationModel__CCore__CAppListEntry_t> operation;
package3->GetAppListEntriesAsync(&operation);
package3 is pointer to type ABI::Windows::ApplicationModel::IPackage3:-
ComPtr<ABI::Windows::ApplicationModel::IPackage3> package3;
I invoke the asynchronous operation as follows.
operation->put_Completed(cb.Get())
Where cb holds reference to callback function that GetAppListEntriesAsync() calls asynchronously. (see update2)
There are several such asynchronous operations which are invoked. I want to run a piece of code which should run only after all such asynchronous callbacks have completed. C++/CX has await keyword for doing a wait on asynchronous tasks.
How should I do the same using WRL C++?
I tried using WaitForSingleObject() but it doesn't seem to work.
A very brute approach would be to keep checking status of every asynchronous operation in a while loop.
ComPtr<ABI::Windows::Foundation::IAsyncInfo> ai;
operation.As(&ai);
while (true)
{
ai->get_Status(&status);
if (status == Started)
{
continue;
}
else
{
break;
}
}
I am not sure if this is guaranteed to work, or whether if it will incur performance penalty. The COM apartment type for my code is APTTYPE_STA (note sure if this piece of information is relevant to the question; I wanted to set is to RO_INIT_MULTITHREADED but RoInitialize() fails, however WRL code seems to be working fine).
UPDATE 2:
Here is how cb (callback) is implemented.
auto cb = wrl::Callback<ABI::Windows::Foundation::IAsyncOperationCompletedHandler<ABI::Windows::Foundation::Collections::__FIVectorView_1_Windows__CApplicationModel__CCore__CAppListEntry_t *>>(
[](ABI::Windows::Foundation::__FIAsyncOperation_1___FIVectorView_1_Windows__CApplicationModel__CCore__CAppListEntry_t * ppp, AsyncStatus status) -> HRESULT
{
<do something in callback>;
SetEvent(threadCompleted.Get());
}
Trying to implement IAgileObject.
To make my callback implement IAgileObject I will have to use Implements<> WRL template class. so callback would be something like auto cb = Callback<Implements<IAgileObject,...>> but the problem is that IAsyncOperationCompletedHandler<ABI::Windows::Foundation::Collections::__FIVectorView_1_Windows__CApplicationModel__CCore__CAppListEntry_t *> doesn't derive from IInspectable, hence using Implements on that :-
Callback<Implements<ABI::Windows::Foundation::IAsyncOperationCompletedHandler<ABI::Windows::Foundation::Collections::__FIVectorView_1_Windows__CApplicationModel__CCore__CAppListEntry_t *>,IAgileObject>> gives the following error (Visual Studio 2017).
Error C2338 'I' has to derive from 'IWeakReference', 'IWeakReferenceSource' or 'IInspectable' AsyncTask c:\sw\tools\sdk\winsdk\win10\include\winrt\wrl\implements.h 413
I am very new to NServiceBus, and in one of our project, we want to accomplish following -
Whenever table data is modified in Sql server, construct a message and insert in sql server broker queue
Read the broker queue message using NServiceBus
Publish the message again as another event so that other subscribers
can handle it.
Now it is point 2, that I do not have much clue, how to get it done.
I have referred the following posts, after which I was able to enter the message in broker queue, but unable to integrate with NServiceBus in our project, as the NServiceBus libraries are of older version and also many methods used are deprecated. So using them with current versions is getting very troublesome, or if I was doing it in improper way.
http://www.nullreference.se/2010/12/06/using-nservicebus-and-servicebroker-net-part-2
https://github.com/jdaigle/servicebroker.net
Any help on the correct way of doing this would be invaluable.
Thanks.
I'm using the current version of nServiceBus (5), VS2013 and SQL Server 2008. I created a Database Change Listener using this tutorial, which uses SQL Server object broker and SQLDependency to monitor the changes to a specific table. (NB This may be deprecated in later versions of SQL Server).
SQL Dependency allows you to use a broad selection of all the basic SQL functionality, although there are some restrictions that you need to be aware of. I modified the code from the tutorial slightly to provide better error information:
void NotifyOnChange(object sender, SqlNotificationEventArgs e)
{
// Check for any errors
if (#"Subscribe|Unknown".Contains(e.Type.ToString())) { throw _DisplayErrorDetails(e); }
var dependency = sender as SqlDependency;
if (dependency != null) dependency.OnChange -= NotifyOnChange;
if (OnChange != null) { OnChange(); }
}
private Exception _DisplayErrorDetails(SqlNotificationEventArgs e)
{
var message = "useful error info";
var messageInner = string.Format("Type:{0}, Source:{1}, Info:{2}", e.Type.ToString(), e.Source.ToString(), e.Info.ToString());
if (#"Subscribe".Contains(e.Type.ToString()) && #"Invalid".Contains(e.Info.ToString()))
messageInner += "\r\n\nThe subscriber says that the statement is invalid - check your SQL statement conforms to specified requirements (http://stackoverflow.com/questions/7588572/what-are-the-limitations-of-sqldependency/7588660#7588660).\n\n";
return new Exception(messageMain, new Exception(messageInner));
}
I also created a project with a "database first" Entity Framework data model to allow me do something with the changed data.
[The relevant part of] My nServiceBus project comprises two "Run as Host" endpoints, one of which publishes event messages. The second endpoint handles the messages. The publisher has been setup to IWantToRunAtStartup, which instantiates the DBListener and passes it the SQL statement I want to run as my change monitor. The onChange() function is passed an anonymous function to read the changed data and publish a message:
using statements
namespace Sample4.TestItemRequest
{
public partial class MyExampleSender : IWantToRunWhenBusStartsAndStops
{
private string NOTIFY_SQL = #"SELECT [id] FROM [dbo].[Test] WITH(NOLOCK) WHERE ISNULL([Status], 'N') = 'N'";
public void Start() { _StartListening(); }
public void Stop() { throw new NotImplementedException(); }
private void _StartListening()
{
var db = new Models.TestEntities();
// Instantiate a new DBListener with the specified connection string
var changeListener = new DatabaseChangeListener(ConfigurationManager.ConnectionStrings["TestConnection"].ConnectionString);
// Assign the code within the braces to the DBListener's onChange event
changeListener.OnChange += () =>
{
/* START OF EVENT HANDLING CODE */
//This uses LINQ against the EF data model to get the changed records
IEnumerable<Models.TestItems> _NewTestItems = DataAccessLibrary.GetInitialDataSet(db);
while (_NewTestItems.Count() > 0)
{
foreach (var qq in _NewTestItems)
{
// Do some processing, if required
var newTestItem = new NewTestStarted() { ... set properties from qq object ... };
Bus.Publish(newTestItem);
}
// Because there might be a number of new rows added, I grab them in small batches until finished.
// Probably better to use RX to do this, but this will do for proof of concept
_NewTestItems = DataAccessLibrary.GetNextDataChunk(db);
}
changeListener.Start(string.Format(NOTIFY_SQL));
/* END OF EVENT HANDLING CODE */
};
// Now everything has been set up.... start it running.
changeListener.Start(string.Format(NOTIFY_SQL));
}
}
}
Important The OnChange event firing causes the listener to stop monitoring. It basically is a single event notifier. After you have handled the event, the last thing to do is restart the DBListener. (You can see this in the line preceding the END OF EVENT HANDLING comment).
You need to add a reference to System.Data and possibly System.Data.DataSetExtensions.
The project at the moment is still proof of concept, so I'm well aware that the above can be somewhat improved. Also bear in mind I had to strip out company specific code, so there may be bugs. Treat it as a template, rather than a working example.
I also don't know if this is the right place to put the code - that's partly why I'm on StackOverflow today; to look for better examples of ServiceBus host code. Whatever the failings of my code, the solution works pretty effectively - so far - and meets your goals, too.
Don't worry too much about the ServiceBroker side of things. Once you have set it up, per the tutorial, SQLDependency takes care of the details for you.
The ServiceBroker Transport is very old and not supported anymore, as far as I can remember.
A possible solution would be to "monitor" the interesting tables from the endpoint code using something like a SqlDependency (http://msdn.microsoft.com/en-us/library/62xk7953(v=vs.110).aspx) and then push messages into the relevant queues.
.m