i'm trying to request a GET via HTTPS trough a Proxy. The Proxy answers with 400:Bad Request. I sniffed the data with wireshark and i have seen, that the headers are not set. Because of security, i replaced some Values with <> Brackets. Can anybody help?
This is a part of my implementation:
String urlString = ctx.getUrl();
HttpHost target;
CloseableHttpClient httpclient;
HttpClientContext localContext;
try
{
CredentialsProvider credsProvider = new BasicCredentialsProvider();
int proxyport = Integer.parseInt(ctx.getProxyPort());
credsProvider.setCredentials(
new AuthScope(<MyProxyUrl>, <MyProxyPort>),
new UsernamePasswordCredentials(<MyProxyUser>, <MyProxyPassword>));
httpclient = HttpClients.custom().setDefaultCredentialsProvider(credsProvider).build();
target = new HttpHost(urlString, 443, "https");
HttpHost proxy = new HttpHost(<MyProxyUser>, <MyProxyPassword>);
RequestConfig config = RequestConfig.custom().setProxy(proxy).build();
request = new HttpGet("/");
request.setURI(new URI(urlString));
//this method sets different header
HttpProperty.setHeaders(request, ctx);
request.setConfig(config);
HttpResponse response = httpclient.execute(target, request);
I have traced the headers in the request with and printed name/value with the output below:
Header[] headers = request.getAllHeaders();
Header Name:Proxy-Connection
Header Value:close
Header Name:Proxy-Authorization
Header Value:Basic
Header Name:User-Agent
Header Value: MyDevice
Header Name:Accept
Header Value:text/html
At least this is the response:
Content length responesCode: 0 400
From Wireshark sniffing i found, that the headers which are defintly inside the request are not set when sending CONNECT.
I attached this picture:
EDIT:
Thank you Damian Nikodem,
but i found the solution.
The first request is always sent without user headers.
I changed two things, and the proxy authorization works:
request = new HttpGet(urlString);
HttpResponse response = httpclient.execute(request);
I am assuming that you are attempting to communicate with a SCADA or production management system. From what I can see you are sending plain HTTP to your target server via the proxy, while trying to connect via HTTPS.
I couldn't see the response in your wireshark dump, but it most likely contains a error message that looks something like this:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html>
<head>
<title>400 Bad Request</title>
</head>
<body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
Reason: You're speaking plain HTTP to an SSL-enabled server port.<br />
Instead use the HTTPS scheme to access this URL, please.<br />
<blockquote>Hint: <b>https://???/</b></blockquote>
</p>
</body>
</html>
P.S. You should edit your image a little bit more because it shows a LOT of information about your employer/customer..
#Thorgas, I don't understand what you sad about "found the solution". Do you mean that you found out how to add extra headers in the CONNECT with HttpClinet? I'm also confusing about this.
But i found out that using HttpsUrlconnection in android doesn't have this kind of problem.
URLConnection urlConnection = url.openConnection(proxy);
HttpsURLConnection httpsUrlConnection = (HttpsURLConnection) urlConnection;
httpsUrlConnection.setRequestMethod("GET");
httpsUrlConnection.setRequestProperty("HEADER1","HEADER1Content");
httpsUrlConnection.connect();
Related
I thought I'd share something that took me some time to figure out:
I wrote a simple Post method using HttpWebRequest class.
In HttpWebRequest you can't use HttpWebRequest.Headers collection to set your desired headers when there is a dedicated property for it - you must use that dedicated property. ContentType is one of them. So I created my HttpWebRequest like this:
HttpWebRequest httpWebRequest = (HttpWebRequest)webRequest;
httpWebRequest.Method = "POST";
httpWebRequest.KeepAlive = false;
httpWebRequest.ServicePoint.Expect100Continue = false;
httpWebRequest.ContentType = "application/json";
somewhere below I set the body of my request like this:
using (StreamWriter streamWriter = new StreamWriter(streamWebRequest))
{
streamWriter.Write(sJson);
}
and posted the request using:
WebResponse webResponse = httpWebRequest.GetResponse();
But I kept getting a "400 - Bad Request" error, while the same request worked from Postman. After analyzing the request with Fiddler I found that when I send the request from my app, the Content-Type: application/json header is missing. All the other headers were present, except for Content-Type. I thought I'm setting it wrong, so I googled but didn't find a good answer. After much experimentation I found, that if I move the line:
httpWebRequest.ContentType = "application/json"
after this block:
using (StreamWriter streamWriter = new StreamWriter(streamWebRequest))
{
streamWriter.Write(sJson);
}
then the httpWebRequest.ContentType = "application/json" header finally appears in the request. So, for HttpWebRequest make sure you always set your HttpWebRequest's body/content first, before you set the ContentType property.
Hope it helps
My question above already has the answer, but to mark it as "Answered" I had to add this comment:
Make sure you always set your HttpWebRequest's body/content first, before you set the ContentType property.This way the "Content-Type" header will appear in the request.
I tried to generate the token from login request.It is successful in postman tool and success in soapui groovy script.But I couldnot do via rest assured library.Below are the screenshot where the request uses Body - form-data with username and password.
I have tried the using queryparams, formparam but getting the below error.Kindly help me to solve the error.
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.4.4</center>
</body>
</html>
Rest assured code:
Map<String, String> formParams = new HashMap<>();
formParams.put("username", "test");
formParams.put("password", "welcome");
Response response = RestAssured.given().config(RestAssured.config().redirect(redirectConfig().followRedirects(false)).encoderConfig(EncoderConfig.encoderConfig().encodeContentTypeAs("multipart/form-data", ContentType.TEXT)))
.queryParams(formParams)
.post("http://posturl");
You are sending the parameters as a query, not as form params.
Also, looks like you need to follow the redirect, since you're getting a 301, so you need followRedirects(true) instead of false.
You need to do it this way:
Response response = RestAssured.given()
.config(RestAssured.config()
.redirect(new RedirectConfig().followRedirects(true))
.encoderConfig(EncoderConfig
.encoderConfig()
.encodeContentTypeAs("multipart/form-data", ContentType.TEXT)))
.formParams(formParams)
.post("http://posturl");
I'm trying to create a REST API from a SOAP Service using IBM API Connect 5. I have followed all the steps described in this guide (https://www.ibm.com/support/knowledgecenter/en/SSFS6T/com.ibm.apic.apionprem.doc/tutorial_apionprem_expose_SOAP.html).
So, after dragging the web service block from palette, ensuring the correctness of endpoint and publishing the API, I have tried to call the API from the browser. Unfortunately, the API return the following message:
<errorResponse>
<httpCode>500</httpCode>
<httpMessage>Internal Server Error</httpMessage>
<moreInformation>Error attempting to read the urlopen response
data</moreInformation>
</errorResponse>
To testing purpose, I have logged the request and I have tried the request on SOAPUI. The service return the response correctly.
What is the problem?
In my case, the problem was in the backend charset (Content-Type: text/xml;charset=iso-8859-1).
For example, backend returns text/xml in German (or French). Api Connect cannot process character ΓΌ. It needs Content-Type: text/xml;charset=UTF-8.
I had a similar issue, in my case was the accept. if you have an Invoke and the content-type or the accept, is not matching the one of the request, or the response that you got, APIC is getting mad.
Please, check if the formats to send (contentType) and receive (accept) are the same of that your API expected. In my case the error occurs because the API returns a String and my default code is configured to receive a JSON body.
//define a JSON-PLAIN TEXT protocol
private HttpEntity<String> httpEntityWithBody(Object objToParse){
HttpHeaders headers = new HttpHeaders();
headers.set("Authorization", "Bearer " + "xxx token xxx");
headers.set("Accept", MediaType.TEXT_PLAIN_VALUE);
headers.setContentType(MediaType.APPLICATION_JSON);
Gson gson = new GsonBuilder().create();
String json = gson.toJson(objToParse);
HttpEntity<String> httpEntity = new HttpEntity<String>(json, headers);
return httpEntity;
}
//calling the API to APIC...
ParameterizedTypeReference<String> responseType = new
ParameterizedTypeReference<String>(){};
ResponseEntity<String> result =
rest.exchange(builder.buildAndExpand(urlParams).toUri(), HttpMethod.PUT, httpEntityWithBody(myDTO), responseType);
String statusCode = result.getStatusCodeValue();
String message = result.getBody();
I am having problems in retrieving the contents of a http get request in the proper charset.
I tried several pieces of code, such as the following:
HttpClient h = new HttpClient();
//Content-Type: text/html; charset=UTF-8
//p.s. contents are in hebrew.
var resp = h.GetAsync("http://www.wnf.co.il");
var content = resp.Result.Content;
//remove the default Content-Type header
content.Headers.Remove("Content-Type");
content.Headers.Add("Content-Type", "text/html; charset=utf-8");
var res = content.ReadAsStringAsync();
var s = res.Result;
Console.WriteLine(s);
which still does not help, I still get the content in wrong encoding.
This post clarifies that setting the header's request headers charset will not help, it's the response's one that needs to be set. (Besides, you will get an error in trying to add
header "Content-Type" to a request Header.)
But I still could not end up with working retrieval of the content in the proper charset (utf-8).
What am I missing ?
I have been doing similar stuff with hebrew sites for a while, in comparing the response's header in Fiddler from this site and others where I do not have this problem - the only difference I see is indeed this Content-Type header in the response.
The issue is probably due to this bug:
https://connect.microsoft.com/VisualStudio/feedback/details/790174/system-net-http-httpcontent-readasstringasync-does-not-handle-imperfect-content-type-headers
The work-around is to get the response as a byte array and encode it yourself:
var bytes = await content.ReadAsByteArrayAsync();
var s = Encoding.UTF8.GetString(bytes, 0, bytes.Length);
As a side-note, is there a reason you're using .Result instead of await? You are blocking the current thread unnecessarily and setting yourself up for deadlocks.
I got a href from Website by inspect it, and I would like to send http request to update the value. But unfortunately the value isn't updated. And response text is the login page.
The href as below.
<a href="SystemTeam.do?formPostType=Update&ID=001&toggle=false">
My JScript as below.
var baseURL =
"https://xx.xx.xx.xx/admin/SystemTeam.do?formPostType=Update&ID=001&toggle=true";
var xmlHttpRequest = new ActiveXObject("MSXML2.ServerXMLHTTP.3.0");
//Ignore SSL error.
xmlHttpRequest.setOption(2,13056);
xmlHttpRequest.open("GET", baseURL, false);
//Set login user authorization
xmlHttpRequest.setRequestHeader("Authorization", "Basic XXXXX");
xmlHttpRequest.send();
Could anyone give some suggestions?