How to call more than 1000 web APIs with a main API with very less response time - asp.net-mvc-4

Question

I have to implement MVC .Net Web api (say "Main" api) which includes two parts.
1) Database call to fetch the record.
2) And more than 1000s of another web api call(response time 100 ms on avg. for each) which will use records returned by above db call.
Also, the Main api will be called in every 3 seconds continuously. I tried implementing using async/await method but didn't find much progress and when trying to test it using Apache Benchmark tool, it throws timeout specified has expired error.
Is there any way to achieve this? Please suggest.
Code snippet
[HttpGet]
public async Task<string> doTaskasync()
{
TripDetails obj = new TripDetails();
GPSCoordinates objGPS = new GPSCoordinates();
try
{
/* uriArray Contains more than 1000 APIs which needs to be exectued. */
string[] uriArray = await dolongrunningtaskasync();
IEnumerable<Task<GPSCoordinates>> allTasks = uriArray .Select(u => GetLocationsAsync(u));
IEnumerable<GPSCoordinates> allResults = await Task.WhenAll(allTasks);
}
catch (Exception ex)
{
return ex.Message ;
}
return "success";
}
Ab.exe test

Answer 1

There is nothing wrong programmatically with your code - except that this is practically a DOS attack on the second (location) API. You should definitely add caching to avoid at least part of the 1000 api calls, especially if you call the main api often (as you wrote).
What I would do is to make the inner api calls a centralized operation instead of making these calls individually inside your web api method. For example you could use a central list for location api calls (tasks) that have been started (but not finished), and another list for results that have been already finished. Both list could be a concurrent dictionary by the unique urls you use for the location api calls.