Asynchronous completion handling in a function with multiple closures/API requests in swift - api

I just started developing in Swift, so im totally new to closures. I'm also new how to handle asynchronous API request.
I have read a lot of similar question such as, How to get data to return from NSURLSessionDataTask in Swift and How to use completionHandler Closure with return in Swift?. These helped me, but my problem it a little bit different.
In my function I want to first make a API request to get a JSON payload. With some data in this JSON payload I want to make multiple other API request. In this case, I will for each of API request receive a JSON payload, where I want to store some of the data in my own JSON data structure.
The problem is that, for every multiple API request I make I can only return part of my own JSON data in my CompletionHandler - This is only way to return data when making an API request using a closure, as far as I understand.
So instead of getting multiple completion handlers, when calling my function, I just want to receive a single.
The thing is I dont know to how to completion handling several closures in a function, in this case two closures.
I have posted my code below - I know its quite long and maybe not that clean.
However, the point is that when im updating offers to my storeDict this will be empty, due to the offers dict array is getting its information from inside the second closure. This is shown at the bottom of the function.
func getOffersFromWishList(offerWishList: [String], latitude: Double, longitude: Double, radius: Int, completionHandler: ([NSDictionary] -> Void)) {
var master: [NSDictionary] = []
var nearby_params: NSDictionary = ["r_lat": latitude, "r_lng": longitude, "r_radius": radius]
//println(nearby_params)
var store_id_list: [String] = []
// Get all store_ids for store which are nearby (Radius determines how nearby)
singleton_eta.api("/v2/stores", type: ETARequestTypeGET, parameters: nearby_params, useCache: true, completion: { (response, error, fromCache) -> Void in
if error == nil {
let json = JSON(response)
storeArray = json.arrayValue
//println(storeArray)
for store in storeArray {
var storeDict = [String: AnyObject]()
var metaData = [String: String]()
var offers: [NSDictionary] = []
let name = store["branding"]["name"].stringValue
let store_id = store["id"].stringValue
let street = store["street"].stringValue
let city = store["city"].stringValue
let zip_code = store["zip_code"].stringValue
let dealer_id = store["dealer_id"].stringValue
let logo = store["branding"]["logo"].stringValue
metaData = ["name": name, "store_id": store_id, "street": street, "city": city, "zip_code": zip_code, "dealer_id": dealer_id, "logo": logo]
store_id_list.append(store_id)
//println("Butiks ID: \(store_id)")
var offset = 0
let limit = 100
// Loop through the offers for the specific store id - only possible to request 100 offers each time
// A while loop would be more suitable, but I dont know when to stop, as the length of the offerArray can not be counted as it is cant be accessed outside of the closure.
for x in 1...2 {
var store_params: NSDictionary = ["store_ids:": store_id, "limit": limit, "offset": offset]
println(store_params)
// Get offers for a specific store_id
singleton_eta.api("/v2/offers", type: ETARequestTypeGET, parameters: store_params, useCache: true, completion: { (response, error, fromCache) -> Void in
if error == nil {
offerArray = JSON(response).arrayValue
//println( "TypeName0 = \(_stdlib_getTypeName(offerArray))")
//Loop through the recieved offers
for of in offerArray {
let name = of["branding"]["name"].stringValue
let dealer_id = of["dealer_id"].stringValue
let heading = of["heading"].stringValue
let description = of["description"].stringValue
let price = of["pricing"]["price"].stringValue
let image = of["images"]["view"].stringValue
//println(heading)
// Loop through our offerWishList
for owl in offerWishList {
let headingContainsWish = (heading.lowercaseString as NSString).containsString(owl.lowercaseString)
// Check if offer match with our wish list
if(headingContainsWish) {
// Save neccesary meta data about each offer to a tuple array
var offer = Dictionary<String, String>()
offer = ["name": name, "dealer_id": dealer_id, "heading": heading, "description": description, "price": price, "image": image, "offerWishItem": owl]
offers.append(offer)
}
}
}
}
})
//println(storeDict)
offset = offset + limit + 1
}
storeDict.updateValue(metaData, forKey: "meta_data")
storeDict.updateValue(offers, forKey: "offers") // offers is empty due to its appending inside the closure
master.append(storeDict)
}
completionHandler(master)
}
else {
println(error)
}
})
}
Calling the above function
getOffersFromWishList(offerWishList, latitude, longitude, radius) { (master) -> Void in
println(master)
}
This is what the master will print when calling the function, where offers is empty.
{
"meta_data" = {
city = "Kongens Lyngby";
"dealer_id" = d8adog;
logo = "https://d3ikkoqs9ddhdl.cloudfront.net/img/logo/default/d8adog_3qvn3g8xp.png";
name = "d\U00f8gnNetto";
"store_id" = d2283Zm;
street = "Kollegiebakken 7";
"zip_code" = 2800;
};
offers = (
);
}
{
...
}
So my questions, what is the proper way to return data from the second closure to the first closure inside a function? Or am I doing this in the completely wrong way?
The thing is, I need all this data for a tableview and therefore need all the data at once.

A couple of thoughts:
If there's any possibility of returning all of this in a single request to the server, that might offer better performance. Often, the time required to performing the requests on server is inconsequential in comparison to the network latency. If you can avoid needing to make one request, get a response, and then issue more requests, that would be ideal.
Or perhaps you request the locations within some distance in advance, cache that, and then the "show me deals for nearby locations" might not require these two sets of requests.
(I recognize that neither of these may work for you, but it's something to consider if you can. If you can eliminate consecutive requests and focus on largely concurrent requests, you'll have much better performance.)
Let's assume for a second that the above is not an option, and you're stuck with one request to get the nearby locations and another set to get the deals. Then you have a couple of options:
You can definitely go down the road that you're contemplating with a single callback. You can, for example, issue all of your requests, doing a dispatch_group_enter before you initiate each request, do a dispatch_group_leave upon the completion of each request, and then issue a dispatch_group_notify that will be called when each enter call has been offset by a corresponding leave call. So, build your response object as each request finishes, and only when they're done, call the completion closure.
Another approach would be to have a closure that behaves more like an enumeration closure, one that is called as each site's deals come in. That way, the UI can be updated as things come in, rather than waiting for everything. If you're on a slow network, updating the UI as data comes in may be far more tolerable. (E.g., consider ten requests, each which takes 1 second complete on a slow 3G cellular connection: watching them pop in one per second is far more tolerable than seeing nothing for ten seconds).
Having said that, you may want to abandon closures completely. You could consider a delegate-protocol pattern, where you specify a delegate for your request, and then implement protocol methods for each of the responses you get from the server. That way you can update the UI as new responses come in, rather than holding everything up until the last one comes in. But we're recognizing that there are very different types of responses (one is a list of sites, another is the list deals for a given site, a third would be the "I'm all done" and/or "there was an error), so when it starts to get this complicated, it might be better to define a protocol for this interface, and handle it that way.

Related

Spring WebFlux Web Client - Iterating paged REST API

I need to get the items from all pages of a pageable REST API. I also need to start processing items, as soon as they are available, not needing to wait for all the pages to be loaded. In order to do so, I'm using Spring WebFlux and its WebClient, and want to return Flux<Item>.
Also, the REST API I'm using is rate limited, and each response to it contains headers with details on the current limits:
Size of the current window
Remaining time in the current window
Request quota in window
Requests left in current window
The response to a single page request looks like:
{
"data": [],
"meta": {
"pagination": {
"total": 10,
"current": 1
}
}
}
The data array contains the actual items, while the meta object contains pagination info.
My current solution first does a "dummy" request, just to get the total number of pages, and the rate limits.
Mono<T> paginated = client.get()
.uri(uri)
.exchange()
.flatMap(response -> {
HttpHeaders headers = response.headers().asHttpHeaders();
Limits limits = new Limits();
limits.setWindowSize(headers.getFirst("X-Window-Size"));
limits.setWindowRemaining(headers.getFirst("X-Window-Remaining"));
limits.setRequestsQuota(headers.getFirst("X-Requests-Quota");
limits.setRequestsLeft(headers.getFirst("X-Requests-Remaining");
return response.bodyToMono(Paginated.class)
.map(paginated -> {
paginated.setLimits(limits);
return paginated;
});
});
Afterwards, I emit a Flux containing page numbers, and for each page, I do a REST API request, each request being delayed enough so it doesn't get past the limit, and return a Flux of extracted items:
return paginated.flatMapMany(paginated -> {
return Flux.range(1, paginated.getMeta().getPagination().getTotal())
.delayElements(Duration.ofMillis(paginated.getLimits().getWindowRemaining() / paginated.getLimits().getRequestsQuota()))
.flatMap(page -> {
return client.get()
.uri(pageUri)
.retrieve()
.bodyToMono(Item.class)
.flatMapMany(p -> Flux.fromIterable(p.getData()));
});
});
This does work, but I'm not happy with it because:
It does initial "dummy" request to get the number of pages, and then
repeats the same request to get the actual data.
It gets rate limits only with the initial request, and assumes the
limits won't change (eg, that it's the only one using the API) -
which may not be true, in which case it will get an error that it
exceeded the limit.
So my question is how to refactor it so it doesn't need the initial request (but rather get limits, page numbers and data from the first request, and continue through all pages, while updating (and respecting) the limits.
I think this code will do what you want. The idea is to make a flux that make a call to your resource server, but in the process to handle the response, to add a new event on that flux to be able to make the call to next page.
The code is composed of:
A simple wrapper to contains the next page to call and the delay to wait before executing the call
private class WaitAndNext{
private String next;
private long delay;
}
A FluxProcessor that will make HTTP call and process the response:
FluxProcessor<WaitAndNext, WaitAndNext> processor= DirectProcessor.<WaitAndNext>create();
FluxSink<WaitAndNext> sink=processor.sink();
processor
.flatMap(x-> Mono.just(x).delayElement(Duration.ofMillis(x.delay)))
.map(x-> WebClient.builder()
.baseUrl(x.next)
.defaultHeader("Accept","application/json")
.build())
.flatMap(x->x.get()
.exchange()
.flatMapMany(z->manageResponse(sink, z))
)
.subscribe(........);
I split the code with a method that only manage response: It simply unwrap your data AND add a new event to the sink (the event beeing the next page to call after the given delay)
private Flux<Data> manageResponse(FluxSink<WaitAndNext> sink, ClientResponse resp) {
if (resp.statusCode()!= HttpStatus.OK){
sink.error(new IllegalStateException("Status code invalid"));
}
WaitAndNext wn=new WaitAndNext();
HttpHeaders headers=resp.headers().asHttpHeaders();
wn.delay= Integer.parseInt(headers.getFirst("X-Window-Remaining"))/ Integer.parseInt(headers.getFirst("X-Requests-Quota"));
return resp.bodyToMono(Item.class)
.flatMapMany(p -> {
if (p.paginated.current==p.paginated.total){
sink.complete();
}else{
wn.next="https://....?page="+(p.paginated.current+1);
sink.next(wn);
}
return Flux.fromIterable(p.getData());
});
}
Now we just need to initialize the system by calling for the retrieval of the first page with no delay:
WaitAndNext wn=new WaitAndNext();
wn.next="https://....?page=1";
wn.delay=0;
sink.next(wn);

Only a node can be linked! Not "undefined"!

I want to put nodes in Gun set.
const Gun = require('gun');
const _ = require('lodash');
require( "gun/lib/path" );
const gun = new Gun({peers:['http://localhost:8080/gun']});
const watchers = [
{
_id: '123',
_type: 'skeleton',
_source: {
trigger: {
schedule: {
later: 'every 1 sec'
}
}
}
},
{
_id: '456',
_type: 'snowman',
_source: {
trigger: {
schedule: {
later: 'every 1 sec'
}
}
}
}
];
const tasks = gun.get('tasks');
_.forEach(watchers, function (watcher) {
let task = gun.get(`watcher/${watcher._id}`).put(watcher);
tasks.set(task);
});
In the end, I receive only the following message. And script stuck in the terminal.
Only a node can be linked! Not "undefined"!
There is nothing on the listener side:
const tasks = gun.get('tasks');
tasks.map().val(function (task) {
console.log('task', task);
});
What is wrong?
The result is received on the listener side only if I change the watchers objects to simpler ones, for example:
_.forEach(watchers, function (watcher) {
let task = gun.get(`watcher/${watcher._id}`).put({id: '123'});
tasks.set(task);
});
Results:
task { _: { '#': 'watcher/123', '>': { id: 1506953120419 } },
id: '123' }
task { _: { '#': 'watcher/456', '>': { id: 1506953120437 } },
id: '123' }
#trex you correctly reported this as a bug, and we got this fixed here: https://github.com/amark/gun/issues/427 .
When a node is referenced, it should not act as if it is undefined. This was a bug.
However, in the future, some people may attempt to link non-node references. As such, I would like to answer the title of your subject (but note, your actual issue has been fixed, and your code should now work correctly in v0.8.8+).
Why do I get "Only a node can be linked!" error?
Say you have a reference to a thing in gun:
var thing = gun.get('alice').get('age');
You may want to add it to a set (otherwise called a table, or list, or collection) like so:
gun.get('list').set(thing);
You will get a "Only a node can be linked!" error. This is annoying! But here is why:
Because age (or any other example data) is a primitive value, adding it to a table would cause it to lose its context. Instead, we can achieve the exact same end result using the following approach:
var person = gun.get('alice').get('age');
gun.get('list').set(thing);
gun.get('list').map().get('age').on(callback);
Now we get back a list of ages, but those ages will always reflect their latest/current realtime context. Had we just added the age to the table, it would no longer have a realtime context.
This is why only nodes can be linked, because any of the data on that node that you were trying to link can just be linked to by traversing via the node. In this case, it was by doing .get('age') after the list. There are a couple really cool things about this:
Bandwidth is saved. GUN will only load the age property from each item in the list, it will not load the rest of the item. It syncs the data you ask about.
Everything is traversable. No matter where your data is in a graph, whether it is a document, a key/value pair, a table, relational data, or anything else, it will always be traversable from its node in the graph. This is possible because it is always the node that is linked, not the primitive data.
Note: What can be frustrating is that you may not know in advance whether a certain gun reference is a node or a primitive, as you could always allow your users to dynamically change the data on that reference. This would require you to handle the error gracefully and do whatever you can best guess the user intended. You can avoid this problem by enforcing a schema on the data in your app. If your app is deployed, we strongly recommend using a schema.
But what if I want the raw data linked, not a realtime context?
Then all you have to do is pass the actual value of the data, not the reference to it. Like so:
gun.get('list').set(thing);
As always, the chatroom is super friendly so come say hi. Please use StackOverflow for asking questions, but notify us in the chatroom. The chatroom is for quick help, and SO is for long standing questions that others would benefit from.
Thanks for asking this question! I hope this answered it, give us a shout if you have any further questions or concerns.

How can I search within a Podio App using PodioKit and PKTSearchAPI?

I'm trying to do a simple query within all items in a Podio app. But I don't understand how to access the results of the search request. Here's what I have:
let query = PKTSearchQuery.init(text: "search term")
let search = PKTSearchAPI.requestToSearchInAppWithID(1234567, query: query, offset: 0, limit: 30)
requestToSearchInAppWithID creates a PKTRequest object so it seems something needs to be returned via a PKTResponse object. I understand that everything in PodioKit is asynchronous and the examples given here for Objective-C and here for Swift make it clear how responses are received using onComplete, onSuccess, onError completion blocks, but PKTSearchAPI seems to lack a completion block.
Any help would be much appreciated.
The completion blocks you noted from the documentation are accessible on any PKTAsyncTask
To retrieve a PKTAsyncTask given a PKTRequest object, you can use the performRequest method of PKTClient. For example, the following snippet will execute your search request, and then print the results to the console:
let query = PKTSearchQuery.init(text: "search term")
let search = PKTSearchAPI.requestToSearchInAppWithID(1234567, query: query, offset: 0, limit: 30)
let searchTask = PKTClient.currentClient().performRequest(search)
searchTask.onComplete { [weak self] response, error in
let body = response.body as! [String: AnyObject]
let items = body["results"] as? [[NSObject:AnyObject]]
for item in items! {
print("Search result \(item["rank"]!) title: \(item["title"]!)")
}
}
Edit: For additional information about the data you can expect in the search response, see the Podio API Docs for Search

HTML5 history API to reduce server requests

I am trying to develop a search filter and making use of the HTML5 history API to reduce the number of requests sent to the server. If the user checks a checkbox to apply a certain filter I am saving that data in the history state, so that when the user unchecks it I am able to load the data back from the history rather than fetching it again from the server.
When the user checks or unchecks a filter I am changing the window URL to match the filter that was set, for instance if the user tries to filter car brands only of a certain category I change the URL like 'cars?filter-brand[]=1'.
But when mutiple filters are applied I have no way of figuring out whether to load the data from the server or to load it from the history.
At the moment I am using the following code.
pushString variable is the new query string that will be created.
var back = [],forward = [];
if(back[back.length-1] === decodeURI(pushString)){ //check last back val against the next URL to be created
back.pop();
forward.push(currentLocation);
history.back();
return true;
}else if(forward[forward.length-1] === decodeURI(pushString)){
forward.pop();
back.push(currentLocation);
history.forward();
return true;
}else{
back.push(currentLocation); //add current win location
}
You can check if your filters are equivalent.
Comparing Objects
This is a simple function that takes two files, and lets you know if they're equivalent (note: not prototype safe for simplicity).
function objEqual(a, b) {
function toStr(o){
var keys = [], values = [];
for (k in o) {
keys.push(k);
values.push(o[k]);
}
keys.sort();
values.sort();
return JSON.stringify(keys)
+ JSON.stringify(values);
}
return toStr(a) === toStr(b);
}
demo
Using the URL
Pass the query part of the URL (window.location.search) to this function. It'll give you an object you can compare to another object using the above function.
function parseURL(url){
var obj = {}, parts = url.split("&");
for (var i=0, part; part = parts[i]; i++) {
var x = part.split("="), k = x[0], v = x[1];
obj[k] = v;
}
return obj;
}
Demo
History API Objects
You can store the objects with the History API.
window.history.pushState(someObject, "", "someURL")
You can get this object using history.state or in a popState handler.
Keeping Track of Things
If you pull out the toStr function from the first section, you can serialize the current filters. You can then store all of the states in an object, and all of the data associated.
When you're pushing a state, you can update your global cache object. This code should be in the handler for the AJAX response.
var key = toStr(parseUrl(location.search));
cache[key] = dataFromTheServer;
Then abstract your AJAX function to check the cache first.
function getFilterResults(filters, callback) {
var cached = cache[toStr(filters)]
if (cached != null) callback(cached);
else doSomeAJAXStuff().then(callback);
}
You can also use localstorage for more persistent caching, however this would require more advanced code, and expiring data.

Variable in controller is lost when passing between views

I am currently stuck with an issue in my MVC 4 application. I have private variable in controller, that holds instance of a simple class:
private InstallationStatus status = null;
When data get submitted on a view, it gets filled like this:
InstallationStatus installStatus = Install();
if (installStatus != null)
{
status = installStatus;
TempData["installPercent"] = 0;
return View("InstallationProgress", status);
}
This part works as intended, variable is set to the instance as it should be.
After that view periodically checks another variable (using ajax):
<script type="text/javascript">
$(document).ready(function () {
var progress = 0;
$("div.status-message").text("Progress: " + progress + "%");
var statusUpdate = setInterval(function () {
$.ajax({
type: 'GET',
url: "/AppStart/GetInstallProgress",
datatype: "application/html; charset=utf-8",
success: function (data) {
progress = parseInt(data);
if (progress >= 100) {
clearInterval(statusUpdate);
var data = $(this).serialize();
$.ajax({
type: 'POST',
url: "#Url.Action("CompletedStatus", "AppStart")",
success: function () {
window.location = "/Login/Login"
}
});
}
$("div.status-message").text("Progress: " + progress + "%");
}
});
}, 2000);
});
</script>
When it calls "CompletedStatus" action on the controller, variable "status" on the controller is null (the instance set previously is not there?
How do I ensure that its value will persist? It seems to me like whole instance of controller gets lost, but that doesnt really matter to me - the source for "status" is webservice and once I get the instance of InstallationStatus, I cant get it again - I need to keep it.
I tried using TempData to store it but since there can be more than one step between storing it and retrieving it TempData proved unreliable.
The final process is:
Request installation status and navigate to view for installation progress (status will be received when progress will finish).
navigate to view where I will by updating installation progress
using javascript whenever I get callback from server with info about
progress
when installation finishes (status is returned) pass that status to
another view
In the example above I have some dummy code-behind, so the status is returned immediately, but that has no impact on the problem I have.
Currently I can do 1 and 2 and I can call the final view, but I cant pass the status in there because I dont have it on controller anymore and TempData are not reliable for this (sometimes it is still there, sometimes it is not).
Any help will be greatly appreciated.
When it calls "CompletedStatus" action on the controller, variable
"status" on the controller is null (the instance set previously is not
there?
How do I ensure that its value will persist?
private InstallationStatus status = null;
It won't unless it's a static value and that would be a very bad thing to do. Remember that variable values (private members' values) are only scoped within the http request. If you do another request then that's a totally whole new scope for your private variables.
I tried using TempData to store it but since there can be more than
one step between storing it and retrieving it TempData proved
unreliable.
That's because TempData will not have the value you expect it to have once you do another request. One good example of using this TempData is when you want to pass/move some values between a POST and GET, that is when you do a POST and do a redirect. TempData does not fit your case.
Now for a possible solution to your scenario, a good question is: is the installation process called once? Is it unique per user? If it is, which I highly suspect it is, then you need to uniquely identify each request. You can simply use a GUID to identify each request. Save that into your database (better than saving in a session) along with some other information like the status of the installation. Pass that guid back to your client and let them pass it back to the controller and retrieve an update on the status of the installation.