WebRequest HTTP error code without try-catch (VB.NET) [duplicate] - vb.net

I am in a situation where when I get an HTTP 400 code from the server, it is a completely legal way of the server telling me what was wrong with my request (using a message in the HTTP response content)
However, the .NET HttpWebRequest raises an exception when the status code is 400.
How do I handle this? For me a 400 is completely legal, and rather helpful. The HTTP content has some important information but the exception throws me off my path.

It would be nice if there were some way of turning off "throw on non-success code" but if you catch WebException you can at least use the response:
using System;
using System.IO;
using System.Web;
using System.Net;
public class Test
{
static void Main()
{
WebRequest request = WebRequest.Create("http://csharpindepth.com/asd");
try
{
using (WebResponse response = request.GetResponse())
{
Console.WriteLine("Won't get here");
}
}
catch (WebException e)
{
using (WebResponse response = e.Response)
{
HttpWebResponse httpResponse = (HttpWebResponse) response;
Console.WriteLine("Error code: {0}", httpResponse.StatusCode);
using (Stream data = response.GetResponseStream())
using (var reader = new StreamReader(data))
{
string text = reader.ReadToEnd();
Console.WriteLine(text);
}
}
}
}
}
You might like to encapsulate the "get me a response even if it's not a success code" bit in a separate method. (I'd suggest you still throw if there isn't a response, e.g. if you couldn't connect.)
If the error response may be large (which is unusual) you may want to tweak HttpWebRequest.DefaultMaximumErrorResponseLength to make sure you get the whole error.

I know this has already been answered a long time ago, but I made an extension method to hopefully help other people that come to this question.
Code:
public static class WebRequestExtensions
{
public static WebResponse GetResponseWithoutException(this WebRequest request)
{
if (request == null)
{
throw new ArgumentNullException("request");
}
try
{
return request.GetResponse();
}
catch (WebException e)
{
if (e.Response == null)
{
throw;
}
return e.Response;
}
}
}
Usage:
var request = (HttpWebRequest)WebRequest.CreateHttp("http://invalidurl.com");
//... (initialize more fields)
using (var response = (HttpWebResponse)request.GetResponseWithoutException())
{
Console.WriteLine("I got Http Status Code: {0}", response.StatusCode);
}

Interestingly, the HttpWebResponse.GetResponseStream() that you get from the WebException.Response is not the same as the response stream that you would have received from server. In our environment, we're losing actual server responses when a 400 HTTP status code is returned back to the client using the HttpWebRequest/HttpWebResponse objects. From what we've seen, the response stream associated with the WebException's HttpWebResponse is generated at the client and does not include any of the response body from the server. Very frustrating, as we want to message back to the client the reason for the bad request.

I had similar issues when trying to connect to Google's OAuth2 service.
I ended up writing the POST manually, not using WebRequest, like this:
TcpClient client = new TcpClient("accounts.google.com", 443);
Stream netStream = client.GetStream();
SslStream sslStream = new SslStream(netStream);
sslStream.AuthenticateAsClient("accounts.google.com");
{
byte[] contentAsBytes = Encoding.ASCII.GetBytes(content.ToString());
StringBuilder msg = new StringBuilder();
msg.AppendLine("POST /o/oauth2/token HTTP/1.1");
msg.AppendLine("Host: accounts.google.com");
msg.AppendLine("Content-Type: application/x-www-form-urlencoded");
msg.AppendLine("Content-Length: " + contentAsBytes.Length.ToString());
msg.AppendLine("");
Debug.WriteLine("Request");
Debug.WriteLine(msg.ToString());
Debug.WriteLine(content.ToString());
byte[] headerAsBytes = Encoding.ASCII.GetBytes(msg.ToString());
sslStream.Write(headerAsBytes);
sslStream.Write(contentAsBytes);
}
Debug.WriteLine("Response");
StreamReader reader = new StreamReader(sslStream);
while (true)
{ // Print the response line by line to the debug stream for inspection.
string line = reader.ReadLine();
if (line == null) break;
Debug.WriteLine(line);
}
The response that gets written to the response stream contains the specific error text that you're after.
In particular, my problem was that I was putting endlines between url-encoded data pieces. When I took them out, everything worked. You might be able to use a similar technique to connect to your service and read the actual response error text.

Try this (it's VB-Code :-):
Try
Catch exp As WebException
Dim sResponse As String = New StreamReader(exp.Response.GetResponseStream()).ReadToEnd
End Try

An asynchronous version of extension function:
public static async Task<WebResponse> GetResponseAsyncNoEx(this WebRequest request)
{
try
{
return await request.GetResponseAsync();
}
catch(WebException ex)
{
return ex.Response;
}
}

This solved it for me:
https://gist.github.com/beccasaurus/929007/a8f820b153a1cfdee3d06a9c0a1d7ebfced8bb77
TL;DR:
Problem:
localhost returns expected content, remote IP alters 400 content to "Bad Request"
Solution:
Adding <httpErrors existingResponse="PassThrough"></httpErrors> to web.config/configuration/system.webServer solved this for me; now all servers (local & remote) return the exact same content (generated by me) regardless of the IP address and/or HTTP code I return.

Related

WebClient - how to ignore a specific HTTP error

I'd like to create a Spring WebClient that ignores a specific HTTP error. From the documentation of WebClient.retrieve():
By default, 4xx and 5xx responses result in a WebClientResponseException. To customize error handling, use ResponseSpec.onStatus(Predicate, Function) handlers.
I want all calls through a WebClient instance to ignore the specific HTTP error. That is why onStatus() is of no use to me (it has to be set per response).
The best I could come up with is this:
WebClient webClient = WebClient.builder().filter((request, next) -> {
Mono<ClientResponse> response = next.exchange(request);
response = response.onErrorResume(WebClientResponseException.class, ex -> {
return ex.getRawStatusCode() == 418 ? Mono.empty() : Mono.error(ex);
});
return response;
}).build();
URI uri = UriComponentsBuilder.fromUriString("https://httpstat.us/418").build().toUri();
webClient.get().uri(uri).retrieve().toBodilessEntity().block();
but it does throw the exception instead of ignoring it (the lambda passed to onErrorResume() is never called).
Edited: fixed the mistake pointed out by the first answer.
After extensive debugging of spring-webflux 5.3.4 and with the help of some ideas by Martin Tarjányi, I've come to this as the only possible "solution":
WebClient webClient = WebClient.builder().filter((request, next) -> {
return next.exchange(request).flatMap(res -> {
if (res.rawStatusCode() == HttpStatus.I_AM_A_TEAPOT.value()) {
res = res.mutate().rawStatusCode(299).build();
}
return Mono.just(res);
});
}).build();
URI uri = UriComponentsBuilder.fromUriString("https://httpstat.us/418").build().toUri();
String body = webClient.get().uri(uri).retrieve().toEntity(String.class).block().getBody();
The background: I am migrating some code from RestTemplate to WebClient. The old code looks like this:
RestTemplate restTemplate = ...;
restTemplate.setErrorHandler(new DefaultResponseErrorHandler() {
#Override
public void handleError(ClientHttpResponse response) throws IOException {
if (response.getRawStatusCode() == HttpStatus.I_AM_A_TEAPOT.value()) {
return;
}
super.handleError(response);
}
});
URI uri = UriComponentsBuilder.fromUriString("https://httpstat.us/418").build().toUri();
String body = restTemplate.getForEntity(uri, String.class).getBody();
I believe it is a straightforward and common case.
WebClient is not yet a 100% replacement for RestTemplate.
UPDATE: Turns out this answer doesn't address the core problem of filtering out a specific status code, just addresses a general coding pattern.
The reason onErrorResume lambda is not called is that response.onErrorResume creates a brand new Mono and your code does not use the result (i.e. it's not assigned to the response variable), so in the end a Mono without the onErrorResume operator is returned.
Using Project Reactor it's usually a good practice to avoid declaring local Mono and Flux variables and use a single chain instead. This helps to avoid similar subtle bugs.
WebClient webClient = WebClient.builder()
.filter((request, next) -> next.exchange(request)
.onErrorResume(WebClientResponseException.class, ex -> ex.getRawStatusCode() == 418 ? Mono.empty() : Mono.error(ex)))
.build();

Windows Phone Silverlight request does not update

I'm quite new to the Windows Phone dev and I have to do an application to communicate with a Restful API. Everything works fine to get the informations back from the API but my problem occurs when I try to update the content. For example, I have a profile and I try to update the user's information (change the city let's say). On the server side I can see that my update worked properly but when I go back to my profile in my WP app nothing has changed, the city is still the same as the old one. This is my code :
public MainPage()
{
InitializeComponent();
this.ApplicationBar = this.Resources["HomeBar"] as ApplicationBar;
Requester requester = new Requester();
requester.initGetRequest("/me/", GetResponseCallback, true);
}
private void GetResponseCallback(IAsyncResult asynchronousResult)
{
try
{
HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asynchronousResult);
Stream streamResponse = response.GetResponseStream();
StreamReader streamRead = new StreamReader(streamResponse);
string read = streamRead.ReadToEnd();
GlobalData.GetInstance().user = JsonConvert.DeserializeObject<MeClass>(read);
Dispatcher.BeginInvoke(() =>
{
MessageBox.Show(read);
});
//Create the profile and stuff
streamResponse.Close();
streamRead.Close();
response.Close();
}
catch (WebException webException)
{
HttpStatusCode status = ((HttpWebResponse)webException.Response).StatusCode;
Dispatcher.BeginInvoke(() =>
{
MessageBox.Show(status.ToString());
});
}
}
I figured out that the string 'read' is always equal to the old one, even after the update so this is why the content is not updated but how can the response be exactly the same as before, even if the update worked fine on the server side (if I check in Postman after my update, I can see that my city is the new one). If I restart my app I can see the update.
I can also show you my initGetRequest() :
public void initGetRequest(String endPoint, Action<IAsyncResult> callback, Boolean header)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url + endPoint);
if (header == true)
request.Headers["Authorization"] = GlobalData.GetInstance().Header;
request.BeginGetResponse(new AsyncCallback(callback), request);
}
Thank you for your help !
I finally found why my request was still the same even after the update. The HttpWebRequest uses a cache by default. I only added a small bit of code before calling my request :
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url + endPoint);
if (header == true)
request.Headers["Authorization"] = GlobalData.GetInstance().Header;
request.Headers[HttpRequestHeader.IfModifiedSince] = DateTime.UtcNow.ToString();
request.BeginGetResponse(new AsyncCallback(callback), request);
I had no idea about that cache so I hope this answer will help someone having the same issue !

OneDrive REST API Download file

I'm using the REST API of OneDrive in my WCF Web Service. Everything works well but the Download of a file. I need to get the Stream object of the file downloaded but MemoryStream class gives me an Exception about ReadTimeout and WriteTimeout.
This is the code:
.... some code ....
var rClient = new RestClient("https://apis.live.net/v5.0/");
var rRequest = new RestRequest(rootFile.id + "/content", Method.GET);
rRequest.AddParameter("access_token", data.accessToken);
var rResponse = rClient.Execute(rRequest); // THE RESPONSE IS OK
byte[] array = rResponse.RawBytes;
Stream stream = new MemoryStream(array); // PROBLEM HERE!
return stream;
So when I create the Stream Object the MemoryStream throw 2 Exception on the fields ReadTimeout and WriteTimeout saying that they are not supported for this stream.
I don't know how to solve it
As suggested by Will in the comment, I discovered that the Exception on ReadTimeout and WriteTimeout was not the real problem. The Exception was thrown by a null object in the caller method, after the code posted above.
Below there is the point where the Exception was thrown: the Current object was null.
stream = client.DownloadFile(token);
if (stream != null)
{
**WebOperationContext.Current.OutgoingResponse.ContentType = "text/octet-stream";** //HERE
return stream;
}
I removed the line and everything was fixed

Error 415 from IAV Rest API - Get verbose error message

I have been trying the Instant Account Verification using the REST api but have run into a couple issues. I receive an error 415(Problem Updating Account) when calling either the addTransferAccountForItem or addItemAndStartVerificationDataRequest api. I'm wondering if there is any way to get a more detailed error message to understand what I'm doing wrong when making these calls. The error message is being returned in XML format although it should be returned in JSON.
Here's an example snippet of how I'm making the addItemAndStartVerificationDataRequest call. GDURL is a simple class that holds the url and concatenates all parameters into a string in format "param1=param1Value&param2=param2Value...".
Any nudge in the right direction would be appreciated. Thank you.
The url I am using are:
addItemAndStartVerificationDataRequestURL=
baseUrl+jsonsdk/ExtendedInstantVerificationDataService/addItemAndStartVerificationDataRequest/
addTransferAccountForItem=
baseUrl+jsonsdk/TransferAccountManagement/addTransferAccountForItem/
logger.info("Attempting to add item and start verification");
try{
GDURL iavUrl = new GDURL(restURL + addItemAndStartVerificationDataRequestURL);
iavUrl.addParameter("cobSessionToken", cobrandSessionToken);
iavUrl.addParameter("userSessionToken", userSessionToken);
iavUrl.addParameter("contentServiceId", contentServiceId);
iavUrl.addParameter("accountNumber", accountNumber);
iavUrl.addParameter("routingNumber", routingNumber);
iavUrl.addParameter("credentialFields.enclosedType", "com.yodlee.common.FieldInfoSingle");
iavUrl.addParameter("credentialFields[0].displayName", "UserID");
iavUrl.addParameter("credentialFields[0].fieldType.typeName", "IF_LOGIN");
iavUrl.addParameter("credentialFields[0].helpText", "4710");
iavUrl.addParameter("credentialFields[0].isEditable", "true");
iavUrl.addParameter("credentialFields[0].maxlength", "32");
iavUrl.addParameter("credentialFields[0].name", "LOGIN");
iavUrl.addParameter("credentialFields[0].size", "20");
iavUrl.addParameter("credentialFields[0].value", bankUsername);
iavUrl.addParameter("credentialFields[0].valueIdentifier", "LOGIN");
iavUrl.addParameter("credentialFields[0].valueMask", "LOGIN_FIELD");
iavUrl.addParameter("credentialFields[1].displayName", "Password");
iavUrl.addParameter("credentialFields[1].fieldType.typeName", "IF_PASSWORD");
iavUrl.addParameter("credentialFields[1].helpText", "11976");
iavUrl.addParameter("credentialFields[1].isEditable", "true");
iavUrl.addParameter("credentialFields[1].maxlength", "40");
iavUrl.addParameter("credentialFields[1].name", "PASSWORD");
iavUrl.addParameter("credentialFields[1].size", "20");
iavUrl.addParameter("credentialFields[1].value", bankPassword);
iavUrl.addParameter("credentialFields[1].valueIdentifier", "PASSWORD");
iavUrl.addParameter("credentialFields[1].valueMask", "LOGIN_FIELD");
HttpURLConnection connection = null;
connection = (HttpURLConnection) iavUrl.getURL().openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.addRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.connect();
String s="";
DataOutputStream wr = new DataOutputStream(connection.getOutputStream ());
wr.writeBytes(iavUrl.getParamString());
try (BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(connection.getInputStream()))) {
while(bufferedReader.ready())
s+=bufferedReader.readLine()+"/n";
}
System.out.println("add item response: /n" + s);
}catch(IOException e){
logger.error("error occured", e);
}
The 415(problem updating account) is an error thrown by Yodlee's data agent when it encounters an exception while trying to aggregate the account from end site. This particular error is thrown for situations where the end site terminates the session established by the data agent as the user might have already been logged in to the end site directly.
To know more about error code please refer this document

httpwebrequest in windows phone 8 app throws "The remote server returned an error: NotFound."?

i am developing an windows phone 8 app , in my app i am calling services and downloading some data into my app.
i am testing my app in windows phone 8 emulator, many times it throws this error :
"The remote server returned an error: NotFound."
This is how i have created and used my httpwebrequest :-
public async Task<string> ServiceRequest(string serviceurl, string request, string methodname)
{
string response = "";
try
{
var httpwebrequest = WebRequest.Create(new Uri(serviceurl)) as HttpWebRequest;
httpwebrequest.Method = "POST";
httpwebrequest.Headers["SOAPAction"] = "http://tempuri.org/" + iTestservice + "/" + methodname + "";
httpwebrequest.ContentType = "text/xml";
byte[] data = Encoding.UTF8.GetBytes(request);
using (var requestStream = await Task<Stream>.Factory.FromAsync(httpwebrequest.BeginGetRequestStream, httpwebrequest.EndGetRequestStream, null))
{
await requestStream.WriteAsync(data, 0, data.Length);
}
var httpTask = httpRequest(httpwebrequest);
var completeTask = await Task.WhenAny(httpTask, TaskEx.Delay(300000));
if (completeTask == httpTask)
return await httpTask;
else
httpwebrequest.Abort();
throw new TimeoutException("Service Timed Out");
}
catch (TimeoutException Tex)
{
throw Tex;
}
catch (Exception ex)
{
throw ex;
}
}
public async Task<string> httpRequest(HttpWebRequest request)
{
string received;
using (var response = (HttpWebResponse)(await Task<WebResponse>.Factory.FromAsync(request.BeginGetResponse, request.EndGetResponse, null)))
{
using (var responseStream = response.GetResponseStream())
{
using (var sr = new StreamReader(responseStream))
{
received = await sr.ReadToEndAsync();
}
}
}
return received;
}
i am really not able to figure what was the issue
Please note :-
i tried to open the site (service URL which my app is trying to access) in my emulator browser , it opened correctly , i wasn't facing any issues.
1)is that the problem with my code, if so i request you please correct me ??
2)is this any emulator issue or any connectivity issue ??
3)is this any certification issue opening in emulator ??
Even after a long research to how to fix the issue i was not able to fix it .
Please Help me out.
Thanks in Advance.
I have come across this issue when the webservice requests take longer than 60 seconds. I have read reports that the emulator does not throw this error, but only an actual device. In my experience however, I have had this happen on both emulator and device. This is a default timeout setting that cannot be altered, even if you change it in the config. This issue has been around since WP7.
I've only found this question asked/blogged about once elsewhere. Seems that not a lot of people have picked this up.
AFAIK, Microsoft are aware of this, but don't plan on fixing it. (Proven by the fact that it has been around since WP7, and wasn't fixed for WP8). My guess is that they are of the opinion that web calls shouldn't be taking longer than 60 seconds in any case.
I just resolved it adding empty credentials to https requests, like this
httpwebrequest.Credentials = new NetworkCredential("", "");
here he explains why
http://blog.toetapz.com/2010/11/15/windows-phone-7-and-making-https-rest-api-calls-with-basic-authentication/