httpcomponents's ssl connection results in socket is closed - ssl

I am trying to get some data from webserver which works fine with http.
But when I try https(ssl connection), I get the exceptions like below.
I get the http status code 200 and response content length 2230 which is correct.
java.net.SocketException: Socket is closed
at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1483)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:92)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:183)
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:144)
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:121)
My code is like below with apache httpcomponents httpclient(4.2.5) library.
try {
HttpPost httppost = new HttpPost(uri);
HttpHost targetHost = new HttpHost(HOST_NAME, HOST_PORT, PROTOCOL);
InputStreamEntity reqEntity = new InputStreamEntity(new ByteArrayInputStream(request), -1);
String contentType = TSPConstants.CONST_TSA_CONTENT_TYPE_TSREQUEST;
reqEntity.setContentType(contentType);
reqEntity.setChunked(true);
// It may be more appropriate to use FileEntity class in this particular
// instance but we are using a more generic InputStreamEntity to demonstrate
// the capability to stream out data from any arbitrary source
//
// FileEntity entity = new FileEntity(file, "binary/octet-stream");
httppost.setEntity(reqEntity);
//Authentication
httpclient.getCredentialsProvider().setCredentials(
new AuthScope(targetHost.getHostName(), targetHost.getPort()),
new UsernamePasswordCredentials(id, password));
// Create AuthCache instance
AuthCache authCache = new BasicAuthCache();
// Generate BASIC scheme object and add it to the local
// auth cache
BasicScheme basicAuth = new BasicScheme();
authCache.put(targetHost, basicAuth);
// Add AuthCache to the execution context
BasicHttpContext httpContext = new BasicHttpContext();
httpContext.setAttribute(ClientContext.AUTH_CACHE, authCache);
httpContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
//SSL
SSLContext ctx = SSLContext.getInstance("TLS");
X509TrustManager tm = new X509TrustManager() {
public void checkClientTrusted(X509Certificate[] xcs, String string) throws CertificateException { }
public void checkServerTrusted(X509Certificate[] xcs, String string) throws CertificateException { }
public X509Certificate[] getAcceptedIssuers() {
return null;
}
};
ctx.init(null, new TrustManager[]{tm}, null);
SSLSocketFactory ssf = new SSLSocketFactory(ctx, SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
Scheme sch = new Scheme("https", HOST_PORT, ssf);
httpclient.getConnectionManager().getSchemeRegistry().register(sch);
System.out.println("executing request " + httppost.getRequestLine());
httpclient.execute(httppost, httpContext);
HttpResponse response = send(request);
HttpEntity resEntity = response.getEntity();
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
if (resEntity != null) {
System.out.println("Response content length: " + resEntity.getContentLength());
System.out.println("Chunked?: " + resEntity.isChunked());
}
EntityUtils.consume(resEntity);
resEntity.getContent()
} finally {
// When HttpClient instance is no longer needed,
// shut down the connection manager to ensure
// immediate deallocation of all system resources
httpclient.getConnectionManager().shutdown();
}

Basically the answer gave #Avner in the comment.
The problem (for me) was, that the response was closed before the entity was read.
I did something like this, which was wrong:
HttpEntity entity = null;
try (CloseableHttpResponse response = client.execute(request)) {
entity = response.getEntity();
}
read(entity);
The following worked:
try (CloseableHttpResponse response = client.execute(request)) {
HttpEntity entity = response.getEntity();
read(entity);
}
The maybe not so obvious part: The try-with-resources block in the first example closed the stream, before it was read.

Related

HttpClient not sending post data to NancyFX endpoint

I am doing some integration testing of my web API that uses NancyFX end points. I have the xUnit test create a test server for the integration test
private readonly TestServer _server;
private readonly HttpClient _client;
public EventsModule_Int_Tester()
{
//Server setup
_server = new TestServer(new WebHostBuilder()
.UseStartup<Startup>());
_server.AllowSynchronousIO = true;//Needs to be overriden in net core 3.1
_client = _server.CreateClient();
}
Inside a Test Method I tried the following
[Fact]
public async Task EventTest()
{
// Arrange
HttpResponseMessage expectedRespone = new HttpResponseMessage(System.Net.HttpStatusCode.OK);
var data = _server.Services.GetService(typeof(GenijalnoContext)) as GenijalnoContext;
//Get come random data from the DBcontext
Random r = new Random();
List<Resident> residents = data.Residents.ToList();
Resident random_residnet = residents[r.Next(residents.Count)];
List<Apartment> apartments = data.Apartments.ToList();
Apartment random_Apartment = apartments[r.Next(apartments.Count)];
EventModel model = new EventModel()
{
ResidentId = random_residnet.Id,
ApartmentNumber = random_Apartment.Id
};
//Doesnt work
IList<KeyValuePair<string, string>> nameValueCollection = new List<KeyValuePair<string, string>> {
{ new KeyValuePair<string, string>("ResidentId", model.ResidentId.ToString()) },
{ new KeyValuePair<string, string>("ApartmentNumber", model.ApartmentNumber.ToString())}
};
var result = await _client.PostAsync("/Events/ResidentEnter", new FormUrlEncodedContent(nameValueCollection));
//Also Doesnt work
string json = JsonConvert.SerializeObject(model, Formatting.Indented);
var httpContent = new StringContent(json, Encoding.UTF8, "application/json");
var response = await _client.PostAsync("/Events/ResidentEnter", httpContent);
//PostAsJsonAsync also doesnt work
// Assert
Assert.Equal(response.StatusCode, expectedRespone.StatusCode);
}
The NancyFX module does trigger the endpoint and receives the request but without the body
What am I doing wrong? Note that the NancyFX endpoint has no issue transforming a Postman call into a valid model.
The NancyFX endpoint
Alright I fixed it, for those curious the issue was that the NancyFX body reader sometimes does not properly start reading the request body. That is that the stream reading position isn't 0 (the start) all the time.
To fix this you need to create a CustomBoostrapper and then override the ApplicationStartup function so you can set up a before request pipeline that sets the body position at 0
Code below
protected override void ApplicationStartup(TinyIoCContainer container, IPipelines pipelines)
{
base.ApplicationStartup(container, pipelines);
pipelines.BeforeRequest.AddItemToStartOfPipeline(ctx =>
{
ctx.Request.Body.Position = 0;
return null;
});
}

HTTP Requests in Glass GDK

I am implementing a GDK application and need to do in my application some HTTP Post requests. Do I send the HTTP requests the same way as on android phone or there is some other way of doing it? (I have tried the code that I am using on my phone and it's not working for glass.)
thanks for your help in advance.
You can make any post request like in smartphones, but ensure you make the requests using an AsyncTask.
For example:
private class SendPostTask extends AsyncTask<Void, Void, Void> {
#Override
protected Void doInBackground(Void... params) {
// Make your request POST here. Example:
myRequestPost();
return null;
}
protected void onPostExecute(Void result) {
// Do something when finished.
}
}
And you can call that asynctask anywhere with:
new SendPostTask().execute();
And example of myRequestPost() may be:
private int myRequestPost() {
int resultCode = 0;
String url = "http://your-url-here";
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
// add headers you want, example:
// post.setHeader("Authorization", "YOUR-TOKEN");
List<NameValuePair> urlParameters = new ArrayList<NameValuePair>();
nameValuePairs.add(new BasicNameValuePair("id", "111111"));
nameValuePairs.add(new BasicNameValuePair("otherField", "your-other-data"));
try {
post.setEntity(new UrlEncodedFormEntity(urlParameters));
HttpResponse response = client.execute(post);
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + post.getEntity());
System.out.println("Response Code : " +
response.getStatusLine().getStatusCode());
resultCode = response.getStatusLine().getStatusCode();
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
System.out.println(result.toString());
} catch (Exception e) {
Log.e("POST", e.getMessage());
}
return resultCode;
}

Liberty Profile and Apache HttpClient 4.2.1

Not sure if others have seen this, but I can't figure out what the deal is...
I am using Liberty Profile 8.5.5.1 with IBM JDK 7 pxi3270sr5-20130619_01(SR5)
I have a class that needs to make URL requests to another server (PHP based). So I wrote a HttpHelper class to call the apache HttpClient classes. If I invoke my helper from a plain java application I have no problem. When I run the exact same code within Liberty I get a ClassNotFound Error for javax.net.ssl.SSLSocket
Here is my code that calls the apache classes...
public class HttpClientHelper
{
static Logger LOGGER = Logger.getLogger(HttpClientHelper.class.getName());
static String cname = HttpClientHelper.class.getName();
static HttpClientHelper _instance = null;
static PoolingClientConnectionManager _cm = null;
private HttpClientHelper()
{
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(
new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
_cm = new PoolingClientConnectionManager(schemeRegistry);
_cm.setMaxTotal(200);
_cm.setDefaultMaxPerRoute(20);
HttpHost shost = new HttpHost("joomla32.cal2.net", 80);
_cm.setMaxPerRoute(new HttpRoute(shost), 50);
}
protected static HttpClientHelper getInstance() {
if (_instance == null) {
_instance = new HttpClientHelper();
}
return _instance;
}
public static String getUrlBody(String url) {
String method = "getUrlBody(String url)";
LOGGER.entering(cname, method, url);
getInstance();
String val = null;
DefaultHttpClient httpclient = null;
try {
HttpContext context = new BasicHttpContext();
httpclient = new DefaultHttpClient(_cm);
HttpGet httpget = new HttpGet(url);
HttpResponse response = httpclient.execute(httpget, context);
HttpEntity entity = response.getEntity();
System.out.println(response.getStatusLine());
val = EntityUtils.toString(entity);
EntityUtils.consume(entity);
} catch (Exception e) {
LOGGER.logp(Level.WARNING, cname, method, e.getMessage(), url);
}catch(Error e){
LOGGER.logp(Level.INFO,cname,method,e.getMessage());
e.printStackTrace();
}
finally{
LOGGER.exiting(cname, method, val);
}
return val;
}
}
\
The Error gets thrown at the httpclient.execute(httpget,context);
I have tried adding the ssl updates to the IBM JDK but that didn't work.
ssl.SocketFactory.provider=com.ibm.jsse2.SSLSocketFactoryImpl
ssl.ServerSocketFactory.provider=com.ibm.jsse2.SSLServerSocketFactoryImpl
Any help would be appreciated
Figured it out...
My HttpClient Helper class was packaged in a bundle as part of a feature. The bundle didn't import javax.net.ssl so the class couldn't find it. Doh.

Solr 4 with basic authentication

I am trying to connect to solr using solrj. My solr instance runs in jetty and is protected with basic authentication. I found these links that contain relevant information.
http://grokbase.com/t/lucene/solr-user/1288xjjbwx/http-basic-authentication-with-httpsolrserver
Preemptive Basic authentication with Apache HttpClient 4
However, I still get the following exception:
Caused by: org.apache.http.client.ClientProtocolException
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:822)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:352)
... 5 more
Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity.
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:625)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
... 8 more
I have also attached a snippet of the code I am using.
public static void main(String[] args) throws SolrServerException, IOException {
HttpSolrServer server = new HttpSolrServer("http://localhost:8983/solr/");
DefaultHttpClient m_client =(DefaultHttpClient)server.getHttpClient();
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(USERNAME, PASSWORD);
m_client.addRequestInterceptor(new PreemptiveAuthInterceptor(),0);
(((DefaultHttpClient)m_client).getCredentialsProvider()).setCredentials(new AuthScope("localhost",8983), credentials);
SolrInputDocument document = new SolrInputDocument();
document.addField("id",123213);
server.add(document);
server.commit();
}
}
class PreemptiveAuthInterceptor implements HttpRequestInterceptor {
public void process(final HttpRequest request, final HttpContext context) throws HttpException, IOException {
AuthState authState = (AuthState) context.getAttribute(ClientContext.TARGET_AUTH_STATE);
// If no auth scheme avaialble yet, try to initialize it
// preemptively
if (authState.getAuthScheme() == null) {
AuthScheme authScheme = (AuthScheme) context.getAttribute("preemptive-auth");
CredentialsProvider credsProvider = (CredentialsProvider) context.getAttribute(ClientContext.CREDS_PROVIDER);
HttpHost targetHost = (HttpHost) context.getAttribute(ExecutionContext.HTTP_TARGET_HOST);
if (authScheme != null) {
Credentials creds = credsProvider.getCredentials(new AuthScope(targetHost.getHostName(), targetHost.getPort()));
if (creds == null) {
throw new HttpException("No credentials for preemptive authentication");
}
authState.setAuthScheme(authScheme);
authState.setCredentials(creds);
}
}
}
Can anyone point me in the right direction? Thanks !!
i had the same problem when implementing partial documents update. i solved the problem by implementing PreemptiveAuthInterceptor. see below code
PoolingClientConnectionManager cxMgr = new PoolingClientConnectionManager(
SchemeRegistryFactory.createDefault());
cxMgr.setMaxTotal(100);
cxMgr.setDefaultMaxPerRoute(20);
DefaultHttpClient httpclient = new DefaultHttpClient(cxMgr);
httpclient.addRequestInterceptor(
new PreemptiveAuthInterceptor(), 0);
httpclient.getCredentialsProvider().setCredentials(
AuthScope.ANY,
new UsernamePasswordCredentials(solrDto.getUsername(),
solrDto.getPassword()));
HttpSolrServer solrServerInstance = new HttpSolrServer(solrDto.getUrl(),
httpclient);
solrServerInstance.setRequestWriter(new BinaryRequestWriter());
solrServerInstance.setAllowCompression(true);
You also need:
private class PreemptiveAuthInterceptor implements HttpRequestInterceptor {
public void process(final HttpRequest request, final HttpContext context)
throws HttpException, IOException {
AuthState authState = (AuthState) context
.getAttribute(ClientContext.TARGET_AUTH_STATE);
// If no auth scheme avaialble yet, try to initialize it
// preemptively
if (authState.getAuthScheme() == null) {
CredentialsProvider credsProvider = (CredentialsProvider) context
.getAttribute(ClientContext.CREDS_PROVIDER);
HttpHost targetHost = (HttpHost) context
.getAttribute(ExecutionContext.HTTP_TARGET_HOST);
Credentials creds = credsProvider.getCredentials(new AuthScope(
targetHost.getHostName(), targetHost.getPort()));
if (creds == null)
throw new HttpException(
"No credentials for preemptive authentication");
authState.setAuthScheme(new BasicScheme());
authState.setCredentials(creds);
}
}
}
According to the Solr Security - SolrJ section on Solr Wiki you should be able to do the following:
public static void main(String[] args) throws SolrServerException, IOException {
HttpSolrServer server = new HttpSolrServer("http://localhost:8983/solr/");
HttpClientUtil.setBasicAuth(server.getHttpClient(), USERNAME, PASSWORD);
SolrInputDocument document = new SolrInputDocument();
document.addField("id",123213);
server.add(document);
server.commit();
}
You need to add the JAR solr-solrj-4.0.0.jar for HttpClientUtil.
Then use the below code:
HttpSolrServer solrServer = new HttpSolrServer("http://localhost:8080/solr/"+url);
HttpClientUtil.setBasicAuth((DefaultHttpClient) solrServer.getHttpClient(), "USERNAME", "PASSWORD");
That worked for me on Jdk 1.6 and tomcat 6

Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407" via https

I try to connect to a server via https that requires authentication.Moreover, I have an http proxy in the middle that also requires authentication. I use ProxyAuthSecurityHandler to authenticate with the proxy and BasicAuthSecurityHandler to authenticate with the server.
Receiving java.io.IOException: Unable to tunnel through proxy.
Proxy returns "HTTP/1.1 407 Proxy Auth Required"
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:1525)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect (AbstractDelegateHttpsURLConnection.java:164)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:133)
at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.processRequest(HttpURLConnectionHandler.java:97)
I noticed that the implementation of ProxyAuthSecurityHandler is expecting response code 407 however, during debug we never get to the second part due to the IOException thrown.
Code snap:
ClientConfig configuration = new ClientConfig();
configuration.connectTimeout(timeout);
MyBasicAuthenticationSecurityHandler basicAuthProps = new MyBasicAuthenticationSecurityHandler();
basicAuthProps.setUserName(user);
basicAuthProps.setPassword(password);
configuration.handlers(basicAuthProps);
if ("true".equals(System.getProperty("setProxy"))) {
configuration.proxyHost(proxyHost);
if ((proxyPort != null) && !proxyPort.equals("")) {
configuration.proxyPort(Integer.parseInt(proxyPort));
}
MyProxyAuthSecurityHandler proxyAuthSecHandler =
new MyProxyAuthSecurityHandler();
proxyAuthSecHandler.setUserName(proxyUser);
proxyAuthSecHandler.setPassword(proxyPass);
configuration.handlers(proxyAuthSecHandler);
}
restClient = new RestClient(configuration);
// create the createResourceWithSessionCookies instance to interact with
Resource resource = getResource(loginUrl);
// Request body is empty
ClientResponse response = resource.post(null);
Tried using wink client versions 1.1.2 and also 1.2.1. the issue repeats in both.
What I found out is that when trying to pass through a proxy using https url we first send CONNECT and only then try to send the request. The proxy server cannot read any headrs we attach to the request, cause it doesn't have the key to decrypt the traffic.
This means that the CONNECT should already have the user/pass to the proxy to pass this stage.
here is a code snap I used - that works for me:
import sun.misc.BASE64Encoder;
import java.io.*;
import java.net.*;
public class ProxyPass {
public ProxyPass(String proxyHost, int proxyPort, final String userid, final String password, String url) {
try {
/* Create a HttpURLConnection Object and set the properties */
URL u = new URL(url);
Proxy proxy =
new Proxy(Proxy.Type.HTTP, new InetSocketAddress(proxyHost, proxyPort));
HttpURLConnection uc = (HttpURLConnection)u.openConnection(proxy);
Authenticator.setDefault(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
if (getRequestorType().equals(RequestorType.PROXY)) {
return new PasswordAuthentication(userid, password.toCharArray());
}
return super.getPasswordAuthentication();
}
});
uc.connect();
/* Print the content of the url to the console. */
showContent(uc);
} catch (IOException e) {
e.printStackTrace();
}
}
private void showContent(HttpURLConnection uc) throws IOException {
InputStream i = uc.getInputStream();
char c;
InputStreamReader isr = new InputStreamReader(i);
BufferedReader br = new BufferedReader(isr);
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
}
public static void main(String[] args) {
String proxyhost = "proxy host";
int proxyport = port;
String proxylogin = "proxy username";
String proxypass = "proxy password";
String url = "https://....";
new ProxyPass(proxyhost, proxyport, proxylogin, proxypass, url);
}
}
if you are using wink - like I do, you need to set the proxy in the ClientConfig and before passing it to the RestClient set the default authenticator.
ClientConfig configuration = new ClientConfig();
configuration.connectTimeout(timeout);
BasicAuthenticationSecurityHandler basicAuthProps = new BasicAuthenticationSecurityHandler();
basicAuthProps.setUserName(user);
basicAuthProps.setPassword(password);
configuration.handlers(basicAuthProps);
if (proxySet()) {
configuration.proxyHost(proxyHost);
if ((proxyPort != null) && !proxyPort.equals("")) {
configuration.proxyPort(Integer.parseInt(proxyPort));
}
Authenticator.setDefault(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
if (getRequestorType().equals(RequestorType.PROXY)) {
return new PasswordAuthentication(proxyUser), proxyPass.toCharArray());
}
return super.getPasswordAuthentication();
}
});
}
restClient = new RestClient(configuration);
Resource resource = getResource(loginUrl);
// Request body is empty
ClientResponse response = resource.post(null);
if (response.getStatusCode() != Response.Status.OK.getStatusCode()) {
throw new RestClientException("Authentication failed for user " + user);
}
If Ilana Platonov's answer doesn't work, try editing the variables :
jdk.http.auth.tunneling.disabledSchemes
jdk.http.auth.proxying.disabledSchemes