.NET HttpClient hangs after several requests (unless Fiddler is active) - httpwebrequest

I am using System.Net.Http.HttpClient to post a sequence of requests from a console application to a REST API and to deserialize the JSON responses into strongly-typed objects. My implementation is like this:
using (var client = new HttpClient())
{
var content = new StringContent(data, Encoding.UTF8, "text/html");
var response = client.PostAsync(url, content).Result;
response.EnsureSuccessStatusCode();
return response.Content.ReadAsAsync<MyClass>().Result;
}
However, I am experiencing a problem very similar to one described in this question, whereby everything works fine when the requests are routed via Fiddler, but it hangs after the 4th or 5th request when Fiddler is disabled.
If the cause of the problem is the same, I assume I need to do something more with HttpClient to get it to fully release its resources after each request but I am unable to find any code samples that show how to do this.
Hoping somebody can point me in the right direction.
Many thanks,
Tim

You are not disposing of the HttpResponseMessage object. This can leave open streams with the server, and after some quota of streams with an individual server is filled, no more requests will be sent.
using (var client = new HttpClient())
{
var content = new StringContent(data, Encoding.UTF8, "text/html");
using(var response = client.PostAsync(url, content).Result)
{
response.EnsureSuccessStatusCode();
return response.Content.ReadAsAsync<MyClass>().Result;
}
}

Related

NetCore 3.1 PostAsync CustomHeaders not working

I have several RESTful services that working with each other. In one scenario I want to post some data from one service to another service and I want to attach some information in Header of the request. I saw several cases to do this and in the end I came up with this workaround:
var httpClient = new HttpClient();
httpClient.Timeout = TimeSpan.FromMinutes(3);
var httpRequestMessage = new HttpRequestMessage {
Method = HttpMethod.Post,
RequestUri = new Uri(service2Address),
Content = new StringContent(JsonConvert.SerializeObject(obj))
};
httpRequestMessage.Headers.Add("myCustomHeaderKey", "myCustomHeaderValue");
var response = await httpClient.SendAsync(httpRequestMessage);
var responseString = await response.Content.ReadAsStringAsync();
With these lines of code, a Post request sent, but in service2 when I want to get the headers from request, there is no sign of myCustomHeaderKey in headers collection. I inspect Request.Headers in Visual Studio Watch and even try to get custom header with Request.Headers["myCustomHeaderKey"]. So what's wrong here?
EDIT 1
This implementation in based on this tutorial.
I have developed code like yours. Have created Two Asp.net core 3.1 project with standart template. One service is starting localhost:44320 and other localhost:44300
localhost:44320/PostService wrote the your codes.
Then get this url with browser. localhost:44320/weatherforecast/IncomeService function is like below
Finally i put breakpoint to where get request header. Result is like below
There is a not a problem. Maybe you use change request header middleware. Or if you are using something like nginx. this problem maybe nginx configuration.

How to read the html out of spa website with asp.net.core

As there are no API for this I need to get the HTML of the following website with WebClient response method.
HttpClient client = new HttpClient();
try
{
HttpResponseMessage response = await client.GetAsync("https://www.datawrapper.de/_/UPFwh/");
response.EnsureSuccessStatusCode();
string responseBody = await response.Content.ReadAsStringAsync();
}
catch(HttpRequestException e)
{
}
client.Dispose(true);
The problem is, when I do that I get only the source code of normal javascripts of this single page application and not the real HTML.
Anybody know how to grab the real html with
I profiled the traffic a bit and it looks like the response from that URL you're using is indeed mainly a script, which eventually will load the rest of the website.
Looking through the details the HTML part of the main data seems to be available under a different URL:
https://datawrapper.dwcdn.net/UPFwh/34/
Consider using that instead. Hope this helps!

Avoid HttpWebRequest GET and ProtocolViolationException

Is there anyway to prevent HttpWebRequest.GetRequestStream from throwing an exception when the method is GET? I know that HEAD and GET should not contain a body in valid HTTP, but some servers accept bodies for these methods anyway (e.g., Elasticsearch's _search feature). I have an open source project for doing REST calls and I was wondering if I could circumvent this behavior to avoid surprises.
At some point, I will probably create a version of my library using the new System.Net.Http.HttpClient class. Does this class have the same restrictions?
.Net Core 2.0:
System.Net.Http.HttpClient (Version=4.2.0.0) does not have this restriction if you use the SendAsync method.
var request = new HttpRequestMessage(HttpMethod.Get, "http://example.com/api/_search");
request.Content = new StringContent(jsonSearchCriteria, Encoding.UTF8, "application/json");
var response = await httpClient.SendAsync(request,
HttpCompletionOption.ResponseContentRead);
var jsonResponse = await response.Content.ReadAsStringAsync();

HTTP Post to Onedrive RestAPI

I'm trying to use C# for connecting to Onedrive Rest API platform, The URL that I need to connect is:
"asdfasdfLJLKJLKJK"
and the code I'm trying to run is as below:
using System.Net.Http;
using (var client = new HttpClient())
{
var content_new = new FormUrlEncodedContent(new[]{ new KeyValuePair<string, string>("access_token", "asdKJHKJH")});
var content = new FormUrlEncodedContent(values);
client.BaseAddress = new Uri("https://api.onedrive.com/v1.0");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var response_web = await client.PostAsync("/drive", content_new);
var responseString = await response_web.Content.ReadAsStringAsync();
Console.WriteLine(responseString);
Console.ReadLine();}
But for some odd reason it fails, also would like to know if I can read the data that PostAsync sends to the server? I mean the request URL as that might help troubleshooting too.
Looking at your request you are performing a POST operation on a drive, the OneDrive API does not support that action on the drive node.
If you are looking to create a new folder or upload a file, you'll want to perform that operation on https://api.onedrive.com/drive/root or where ever you'd like the operation to happen in the user's account.
To look up all of the supported actions and example requests see the OneDrive API resource model.

Can't get a token from the Google API

I'm trying to exchange my authorization code for a token with the Google OAuth2 API for my Windows 8 app, but I keep getting HTTP 400 errors.
This is how I perform the request (simplified):
var url = "https://accounts.google.com/o/oauth2/token";
var body = "code=4/LEXF1iAVRZvfCfdQg9r1aFqoYDgV&client_id=904019870963.apps.googleusercontent.com&client_secret=[removed]&redirect_uri=urn:ietf:wg:oauth:2.0:oob&grant_type=authorization_code";
HttpClient httpClient = new HttpClient();
HttpResponseMessage response = await httpClient.PostAsync(new Uri(url), new StringContent(body));
response.EnsureSuccessStatusCode();
Visual Studio usually simply gives me a HTTP 400 bad request error, when I try the same thing in Fiddler I also get an HTTP 400 error, but with this as content:
21
{
"error" : "invalid_request"
}
0
I read all the documentation about Google OAuth, I searched on Google and StackOverflow for this issue, I tried changing all different sorts of things in my code (UrlEncode, etc.), I used the Google API Playground to see what kind of requests it performs and compared it with my own requests (couldn't find a difference except for the return URL, the auth code and the user-agent). No matter what I do, I just can't get this working, I've been stuck for hours already.
Can anyone help me out here?
Read the body content to get the error json you noticed in Fiddler.
HttpClient httpClient = new HttpClient();
var response = httpClient.PostAsync(new Uri(url), new StringContent(body)).Result;
var content = response.Content.ReadAsStringAsync().Result;
content now holds :
{
"error" : "invalid_request"
}
You could project the error in an object, by specifing your error type like: response.Content.ReadAsAsync().Result
For the invalid request part, you should play with UrlEncode. I know you say you tried it, but applying it at the right spots really solves your problem.
var body = "code="+WebUtility.UrlEncode("4/LEXF1iAVRZvfCfdQg9r1aFqoYDgV")+
"&redirect_uri="+WebUtility.UrlEncode("https://yoursite...")+
"&client_id=904019870963.apps.googleusercontent.com" +
"&scope=" +
"&client_secret=********" +
"&grant_type=authorization_code";
HttpClient httpClient = new HttpClient();
var response = httpClient.PostAsync(new Uri(endpoint),
new StringContent(body, Encoding.UTF8, "application/x-www-form-urlencoded")).Result;
This is the code that works for me. Should work for you also.