Is there a way I can build a list without using MutableList when I need to call a api synchronously in a loop? I need to call the api synchronously because the response is paged.. so I get 10 results back per response, then I request the next page. Although, having said that, after the first request I know exactly how many pages there are (in the response I get totalPages) so I could theoretically make async requests for the remaining data
The code below works - but can I do the same thing more elegantly? Possibly using async FP, without vars and a MutableList
private fun getAllCustomers(): List<Customer> {
var pageNumber = 0
var totalPages = 1
val allCustomers: MutableList<Customer> = mutableListOf()
while (pageNumber < totalPages) {
val response = customerApi.getCustomers(alphaChar, pageNumber)
totalPages = response.totalPages
pageNumber ++
allCustomers.addAll(response.content)
}
return allCustomers
}
Related
I need to return an array of labels, but I can only return 1 of the labels so far. The error which I get is "Cannot set headers after they are sent to the client". So I tried res.write and placed res.end after my for loop then I get the obvious error of doing a res.end before a res.write. How do I solve this?
for(let i=0;i<arr.length;i++){
request.get(arr[i], function (error, response, body) {
if (!error && response.statusCode == 200) {
myfunction();
async function myfunction(){
const Labels = await Somefunctioncallwhoseresponseigetlater(body)
res.send(Labels);
}
}
});}
New code-
async function getDataSendResponse(res) {
let allLabels = [];
for (let url of arr) {
let body = await got(url).buffer();
var imgbuff= Buffer.from(body,'base64')
const imageLabels = await rekognition.detectLabels(imgbuff);
allLabels.push(...imageLabels);
}
res.send(allLabels);
}
The error I have with this code is
"Resolver: AsyncResolver
TypeError: Cannot destructure property Resolver of 'undefined' or 'null'."
You are trying to call res.send() inside a for loop. That means you'll be trying to call it more than once. You can't do that. You get to send one response for any given http request and res.send() sends an entire response. So, when you try to call it again inside the loop, you can the warning you see.
If what you're trying to do is to send an array of labels, then you need to accumulate the array of labels first and then make one call to res.send() to send the final array.
You don't show the whole calling context here, but making the following assumptions:
Somefunctioncallwhoseresponseigetlater() returns a promise that resolves when it is done
You want to accumulate all the labels you collected in your loop
Your Labels variable is an array
Your http request returns a text response. If it returns something else like JSON, then .text() would need to be changed to .json().
then you can do it like this:
const got = require('got');
async function getDataSendResponse(res) {
let allLabels = [];
for (let url of arr) {
let body = await got(url).buffer();
const labels = await Somefunctioncallwhoseresponseigetlater(body);
allLabels.push(...labels);
}
res.send(allLabels);
}
Note, I'm using the got() library instead of the deprecated request() library both because request() is not deprecated and because this type of code is way easier when you have an http library that supports promises (like got() does).
I am comparatively new to reactive APIs and was curious about what was happening behind the scenes when we return a Flux from a web controller.
According to spring-web documentation
Reactive return values are handled as follows:
A single-value promise is adapted to, similar to using DeferredResult. Examples include Mono (Reactor) or Single (RxJava).
A multi-value stream with a streaming media type (such as application/stream+json or text/event-stream) is adapted to, similar to using ResponseBodyEmitter or SseEmitter. Examples include Flux (Reactor) or Observable (RxJava). Applications can also return Flux or Observable.
A multi-value stream with any other media type (such as application/json) is adapted to, similar to using DeferredResult<List<?>>.
I created two APIs as below:
#GetMapping("/async-deferredresult")
public DeferredResult<List<String>> handleReqDefResult(Model model) {
LOGGER.info("Received async-deferredresult request");
DeferredResult<List<String>> output = new DeferredResult<>();
ForkJoinPool.commonPool().submit(() -> {
LOGGER.info("Processing in separate thread");
List<String> list = new ArrayList<>();
for (int i = 0; i < 10000 ; i++) {
list.add(String.valueOf(i));
}
output.setResult(list);
});
LOGGER.info("servlet thread freed");
return output;
}
#GetMapping(value = "/async-flux",produces = MediaType.APPLICATION_JSON_VALUE)
public Flux<String> handleReqDefResult1(Model model) {
LOGGER.info("Received async-deferredresult request");
List<String> list = new ArrayList<>();
list.stream();
for (int i = 0; i < 10000 ; i++) {
list.add(String.valueOf(i));
}
return Flux.fromIterable(list);
}
So the exception was that both APIs should behave same as multi-value stream(Flux) should have similar behavior to that of a returning a DeferredResult. But in API where deferred result was returned, whole list was printed in one go on browser where as in API where Flux was returned the numbers where printed sequentially(one by one).
What exactly is happening when I am returning Flux from controller ?
When we return a Flux from a service endpoint many things can happen. But I assume you want to know what is happening when Flux observed as stream of events from client of this endpoint.
Scenario One: By adding 'application/json' as the content type of the endpoint Spring will communicate to the client to expect JSON body.
#GetMapping(value = "/async-flux", produces = MediaType.APPLICATION_JSON_VALUE)
public Flux<String> handleReqDefResult1(Model model) {
List<String> list = new ArrayList<>();
for (int i = 0; i < 10000; i++) {
list.add(String.valueOf(i));
}
return Flux.fromIterable(list);
}
The output at the client will be the whole set of numbers in one go. And once the response delivered the connection will be closed. Even though you have used Flux as the response type, you are still bound the laws of how HTTP over TCP/IP works. The endpoint got a HTTP request, execute the logic and respond with HTTP response containing final result.
As a result, you do not see the real value of a reactive api.
Scenario Two: By adding 'application/stream+json' as the content type of the endpoint, Spring starts to treat the resulting events of the Flux stream as individual JSON items. When an item is emitted is gets serialised, the HTTP response buffer is flushed, and the connection from the server to client keep open up until the event sequence get completed.
To get that working we can slightly modify your original code as follows.
#GetMapping(value = "/async-flux",produces = MediaType.APPLICATION_STREAM_JSON_VALUE)
public Flux<String> handleReqDefResult1(Model model) {
List<String> list = new ArrayList<>();
for (int i = 0; i < 10000 ; i++) {
list.add(String.valueOf(i));
}
return Flux.fromIterable(list)
// we have 1 sec delay to demonstrate the difference of behaviour.
.delayElements(Duration.ofSeconds(1));
}
This time we can see the real value of reactive api endpoint where it is able to deliver results to it's client as date get available.
You can find more details about how to build reactive REST APIs at
https://medium.com/#senanayake.kalpa/building-reactive-rest-apis-in-java-part-1-cd2c34af55c6
https://medium.com/#senanayake.kalpa/building-reactive-rest-apis-in-java-part-2-bd270d4cdf3f
Im new to reactive programming paradigm. I have a simple question.
I have a list of elements, for each element i must do a REST call.
I must build a new list based on the response of these calls.
The problem is that my function returns value before the end of the loop...
I don't know how to do ?
Here my piece of code :
TaeaUpdateRequestOutput putTaea(final String dossierId, final TaeaUpdateRequestInput input, final String token) {
final TaeaUpdateRequestOutput output = new TaeaUpdateRequestOutput();
input.getAdhesions().stream().forEach(adhesion -> {
final Mono<TaeaFromMyMB> taeaResponse = doRest(adhesion, TaeaFromMyMB.class, url, token, dossierId);
taeaResponse.subscribe(myMBTaea -> {
final Taea taea = myMBTaea.fromTaeaFromMyMb(adhesion);
output.getListeTaea().add(taea);
});
});
//output is always empty due to async programming. How to wait the the end of the last iteration's subscription?
return output;
}
You subscribed taeaResponsesbut you didn't wait result of the subscription.
Try something like
List<Taea> taeas = Flux.fromIterable(input.getAdhesions())
.flatMap(adhesion -> doRest(adhesion, TaeaFromMyMB.class, url, token, dossierId)
.map(taeaFromMyMB -> taeaFromMyMB.fromTaeaFromMyMb(adhesion)))
.collect(Collectors.toList())
.subscribeOn(Schedulers.parallel())
// wait result here
.block();
Then set it to the output.
Keep in mind that block() waits indefinitely, so you can use reactor.core.publisher.Mono#block(java.time.Duration) to avoid it.
first, I create empty Array(Kotlin) instance in companion object.
companion object {
var strarray: Array<String> = arrayOf()
var objectarray: LinkedHashMap<Int, List<Any>> = LinkedHashMap<Int, List<Any>>()
}
and I expected that I use empty array instance when read textString from CSV File.
fun csvFileToString():String {
val inputStream = File(Paths.get("").toAbsolutePath().toString()
.plus("/src/main/SampleCSVFile_2kb.csv")).inputStream()
val reader = inputStream.bufferedReader()
var iterator = reader.lineSequence().iterator()
var index:Int = 1;
while (iterator.hasNext()){
var lineText:String = iterator.next()
strarray.set(index, lineText)
index++
}
return ""
}
but when I run that source code
a.csvFileToString()
println(CsvParser.strarray)
occured exception
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
strarray.set(index, lineText) <<<<<<<<< because of this line
can I use Array(from kotlin collection) like ArrayList(from java collection)?
You can add a new item to an array using +=, for example: item += item
private var songs: Array<String> = arrayOf()
fun add(input: String) {
songs += input
}
Size of Array is defined at its creation and cannot be modified - in your example it equals 0.
If you want to create Array with dynamic size you should use ArrayList.
arrayOf gives you an array. Arrays have fixed length even in Java.
listOf gives you an immutable list. You cannot add or remove items in this list.
What you're looking for is mutableListOf<String>.
In your current approach, reusing a member property, don't forget to clear the list before every use.
Your code can be further simplified (and improved) like so:
out.clear()
inputStream.bufferedReader().use { reader -> // Use takes care of closing reader.
val lines = reader.lineSequence()
out.addAll(lines) // MutableList can add all from sequence.
}
Now imagine you wanted to consume the output list but needed to parse another file at the same time.
Consider working towards a pure function (no side effects, for now no accessing member properties) and simplifying it even further:
fun csvFileToString(): String { // Now method returns something useful.
val inputStream = File(Paths.get("").toAbsolutePath().toString()
.plus("/src/main/SampleCSVFile_2kb.csv")).inputStream()
inputStream.bufferedReader().use {
return it.lineSequence().joinToString("\n")
}
}
In this case we can totally skip the lists and arrays and just read the text:
inputStream.bufferedReader().use {
return it.readText()
}
I'm assuming that's what you wanted in the first place.
Kotlin has a lot of useful extension functions built-in. Look for them first.
I am trying to develop a search filter and making use of the HTML5 history API to reduce the number of requests sent to the server. If the user checks a checkbox to apply a certain filter I am saving that data in the history state, so that when the user unchecks it I am able to load the data back from the history rather than fetching it again from the server.
When the user checks or unchecks a filter I am changing the window URL to match the filter that was set, for instance if the user tries to filter car brands only of a certain category I change the URL like 'cars?filter-brand[]=1'.
But when mutiple filters are applied I have no way of figuring out whether to load the data from the server or to load it from the history.
At the moment I am using the following code.
pushString variable is the new query string that will be created.
var back = [],forward = [];
if(back[back.length-1] === decodeURI(pushString)){ //check last back val against the next URL to be created
back.pop();
forward.push(currentLocation);
history.back();
return true;
}else if(forward[forward.length-1] === decodeURI(pushString)){
forward.pop();
back.push(currentLocation);
history.forward();
return true;
}else{
back.push(currentLocation); //add current win location
}
You can check if your filters are equivalent.
Comparing Objects
This is a simple function that takes two files, and lets you know if they're equivalent (note: not prototype safe for simplicity).
function objEqual(a, b) {
function toStr(o){
var keys = [], values = [];
for (k in o) {
keys.push(k);
values.push(o[k]);
}
keys.sort();
values.sort();
return JSON.stringify(keys)
+ JSON.stringify(values);
}
return toStr(a) === toStr(b);
}
demo
Using the URL
Pass the query part of the URL (window.location.search) to this function. It'll give you an object you can compare to another object using the above function.
function parseURL(url){
var obj = {}, parts = url.split("&");
for (var i=0, part; part = parts[i]; i++) {
var x = part.split("="), k = x[0], v = x[1];
obj[k] = v;
}
return obj;
}
Demo
History API Objects
You can store the objects with the History API.
window.history.pushState(someObject, "", "someURL")
You can get this object using history.state or in a popState handler.
Keeping Track of Things
If you pull out the toStr function from the first section, you can serialize the current filters. You can then store all of the states in an object, and all of the data associated.
When you're pushing a state, you can update your global cache object. This code should be in the handler for the AJAX response.
var key = toStr(parseUrl(location.search));
cache[key] = dataFromTheServer;
Then abstract your AJAX function to check the cache first.
function getFilterResults(filters, callback) {
var cached = cache[toStr(filters)]
if (cached != null) callback(cached);
else doSomeAJAXStuff().then(callback);
}
You can also use localstorage for more persistent caching, however this would require more advanced code, and expiring data.