I'm starting with RX and I have a offline app the needs sync data with remote api. What better way to get data from database and send to remote api one by one watching the response of each? Which operators should I use to sequence tasks?
The simplest thing you can do is something like this (kotlin)
getDataFromDb()
.map { doNetworkRequest(it) }
.doOnNext { saveToDb(it) }
.subscribe()
but it really depends on your needs / environment.
You will probably need more mapping in the middle (e.g. to transform the network response to the data that you need to save to the db) and error handling.
Here I assume you don't need the result of saveToDb so I put it as a side effect instead of inside the main observable flow.
Another aspect is how you want to handle the network requests: is it ok to perform them in parallel or not? And maybe use a flatmap
Related
I am using ReactiveRedisOperations to save data objects in Redis and this call returns a Mono as per the api.
I notice that if I don't do anything with this Mono return than this code does not do anything.
Just trying to understand how this works.
I would like below code to save every Object to Redis in this loop, however it does not do so, please share what is missing here.
for (SomeObject obj : list) {
reactiveRedisOperations.opsForHash().put(key, hashKey, obj).map(b -> obj); }
On the other side if i return the Mono result from similar code via a rest service response than it seems to save in Redis correctly, not sure why this is this way. Thanks
This is a quirk of reactive streams, not Lettuce.
Unlike a completable future which begins execution when it's created, a stream won't begin executing (the command isn't sent) until a consumer has subscribed to it.
I believe this is to facilitate backpressure, so a slow consumer isn't flooded with data by the producer.
Some nice reading -> https://blog.knoldus.com/working-with-project-reactor-reactive-streams/
If you return a Mono, to the underlying web framework, which generally will handle subscribe(ing) to this Mono, the respective operation will trigger resulting in whatever side-effects such as data being created in your Redis datastore.
Shall you wish to have your operations executed, you should do the same, i.e. subscribe to the publisher (Mono, or Flux) or return these data wrappers to whatever calling function you would know will handle this for you as in the aforementioned example:
Flux.fromIterable(list)
.flatMap(obj -> reactiveRedisOperations.opsForHash().put(key, hashKey, obj))
.subscribe();
After reading the new spring data casandra documentation (here), it says that there is now Mono<Slice<T>> support for reactive cassandra.
Which is great, because in my team, we wanted to implement some sort of pagination on our reactive response. So we are happy to move from Flux<T> to Mono<Slice<T>> but there is this issue, we do transformations on the data of our flux, with flux.map, but that doesn't seem possible with Slice without blocking.
For example, we have this functionallity:
Flux<Location> resp = repository.searchLocations(searchFields).map(this::transformLocation);
Where transformLocation is a function that receives the database object and returns a more user friendly object with more user friendly data.
How will you achieve that with Mono<Slice<Location>>?
From what I have seen of Slice, you can get the data with getContent, but that returns a list, will that not be like blocking?
You can use the list, got from the getContent() method, to create a flux as you were doing before.
Mono<Slice<Location>> sliceMono; //this is just another mono on which you can operate
Flux<location> intermediateResp = sliceMono.flatMap(slice -> Flux.fromIterable(slice.getContent()));
//you can now transform this intermediateResp flux as you were doing before, can't you?
Won't this serve your purpose or do you require something else?
(The code has been written without the help of IDE, so use it to understand the approach)
I'm using a second datastore with my Ember app, so I can communicate with a separate external API. I have no control over this API.
With a DS.JSONSerializer I can add some missing properties like id:
normalizeResponse(store, primaryModelClass, payload, id, requestType) {
if (requestType == 'query') {
payload.forEach(function(el, index) {
payload[index].id = index
})
}
Now I can do some different tricks for each different requestType. But every response is parsed. Now sometimes a response from one request needs to be parsed differently.
So what I am trying to do is change the normalizeResponse functionality for each different request path (mapped to a fake model using pathForType in an adapter for this store). But the argument store is always the same (obviously) and the argument promaryModelClass is always "unknown mixin" - not sure if this can be any help.
How can I find what model was requested? With this information I could do a switch() in normalizeResponse.
Is there a different way to achieve my goal that does not require me to make a separate adapter for every path/model?
There are over a dozen normalize functions available. Something should work for what I am trying to achieve.
I think this is a great example of a use case of not using ember data.
Assuming that you have models A,B,C that are all working great with ember data, leave those alone.
I'd create a separate service and make raw requests to that different endpoint. So you'd replace this.store.query('thing', {args}) with a separate service that uses ember-ajax (or ember-fetch or whatever). If you need, you can use that service to hold the data that you need (Ember-data is just a service anyway) or you can create models and push them into the store manually.
Without knowing more about your exact situation, hard to give a specific code/advice, but I'd just avoid this problem and write your own custom service.
You can use primaryModelClass.modelName.
I want to use breezejs api for storing data in local storage (indexdb or websql) and also want to sync local data with sql server.
But I am failed to achieve this and also not able to find sample app of this type of application using breezejs, knockout and mvc api.
My requirement is:
1) If internet is available, the data will come from sql server by using mvc web api.
2) If internet is shutdown, the application will retrieve data from cached local storage (indexdb or websql).
3) As soon as internet is on, the local data will sync to sql server.
Please let me know Can I achieve this requirement by using breezejs api or not?
If yes, please provide me some and links and sample.
If no, what other can we use for achieving this type of requirement?
Thanks.
Please help me to meet this requirement.
You can do this, but I would suggest simply using localstorage. Basically, every time you read from the server or save to the server, you export the entities and save that to local storage. THen, when you need to read in the data, if the server is unreachable, you read the data from localstorage and use importentities to get it into the manager and then query locally.
function getData() {
var query = breeze.EntityQuery
.from("{YourAPI}");
manager.executeQuery.then(saveLocallyAndReturnPromise)
.fail(tryLocalRestoreAndReturnPromise)
// If query was successful remotely, then save the data in case connection
// is lost
function saveLocallyAndReturnPromise(data) {
// Should add error handling here. This code
// assumes tis local processing will be successful.
var cacheData = manager.exportEntities()
window.localStorage.setItem('savedCache',cacheData);
// return queried data as a promise so that this detour is
// transparent to viewmodel
return Q(data);
}
function tryLocalRestoreAndReturnPromise(error) {
// Assume any error just means the server is inaccessible.
// Simplified for example, but more robust error handling is
// warranted
var cacheData = window.localStorage.getItem('savedCache');
// NOTE: should handle empty saved cache here by throwing error;
manager.importEntities(cacheData); // restore saved cache
var query = query.using(breeze.FetchStrategy.FromLocalCache);
return manager.executeQuery(query); // this is a promise
}
}
This is a code skeleton for simplicity. You should check catch and handle errors, add an isConnected function to determine connectivity, etc.
If you are doing editing locally, there are a few more hoops to jump through. Every time you make a change to the cache, you will need to export either the whole cache or the changes (probably depending on the size of the cache). When there is a connection, you will need to test for local changes first and, if found, save them to the server before requerying the server. In addition, any schema changes made while offline complicate matters tremendously, so be aware of that.
Hope this helps. A robust implementation is a bit more complex, but this should give you a starting point.
I'm using Netty to write a client application that sends UDP messages to a server. In short I'm using this piece of code to write the stream to the channel:
ChannelFuture future = channel.write(request, remoteInetSocketAddress);
future.awaitUninterruptibly(timeout);
if(!future.isDone()){
//abort logic
}
Everything works fine, but one thing: I'm unable to test the abort logic as I cannot make the write to fail - i.e. even if the server is down the future would be completed successfully. The write operation usually takes about 1 ms so setting very little timeout doesn't help too much.
I know the preffered way would be to use an asynch model instead of await() call, however for my scenario I need it to be synchronous and I need to be sure it get finnished at some point.
Does anyone know how could I simulate an uncompleted future?
Many thanks in advance!
MM
Depending on how your code is written you could use a mock framework such as mockito. If that is not possible, you can also use a "connected" UDP socket, i.e. a datagram socket that is bound to a local address. If you send to a bogus server you should get PortunreachableException or something similar.
Netty has a class FailedFuture that can be used for the purpose of this,
You can for example mock your class with tests that simulate the following:
ChannelFuture future;
if(ALWAYS_FAIL) {
future = channel.newFailedFuture(new Exception("I failed"));
else
future = channel.write(request, remoteInetSocketAddress);