SSL issue on Android 9 Google Pixel One - ssl

I am trying to perform HTTPS requests to a host 10.10.10.1 from Android host with 10.10.10.2 in network without Internet connection - only WiFi 2 peers AP and Android 9 Google Pixel One device.
I've created network_security_config.xml with my cert that is self-signed and has CN=10.10.10.1 and SAN= DNS: 10.10.10.1 PI: 10.10.10.1.
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<base-config cleartextTrafficPermitted="true">
<trust-anchors>
<certificates src="system" />
<certificates src="user" />
<certificates src="#raw/zone"/>
</trust-anchors>
</base-config>
</network-security-config>
I don't receive verification error and observe successful requests incoming to server - data are HTTP request, decrypted and shown on the server log. But the server can't send data back! It sends, but for some reason these data are not being accepted by the Android phone - just ignored.
I see packets are going from the server to the phone and the server repeatedly retries to shutdown SSL socket until error or success (I made such behavior intentionally during surveying) - here is Wireshark dump from WiFi air:
Here is my request from AsyncTask
protected String doInBackground(String... params) {
StringBuilder result = new StringBuilder();
try {
CertificateFactory cf = CertificateFactory.getInstance("X.509");
InputStream caInput = new BufferedInputStream(MainActivity.this.getResources().openRawResource(R.raw.zone));
Certificate ca = cf.generateCertificate(caInput);
String keyStoreType = KeyStore.getDefaultType();
KeyStore keyStore = KeyStore.getInstance(keyStoreType);
keyStore.load(null, null);
keyStore.setCertificateEntry("ca", ca);
String tmfAlgorithm = TrustManagerFactory.getDefaultAlgorithm();
TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlgorithm);
tmf.init(keyStore);
SSLContext ctx = SSLContext.getInstance("TLS");
ctx.init(null, tmf.getTrustManagers(), null);
URL url = new URL("https://10.10.10.1/connect");
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setSSLSocketFactory(ctx.getSocketFactory());
conn.setRequestProperty("param1", params[0]);
conn.setRequestProperty("param2", params[1]);
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setDoInput(true);
mInputStream = conn.getInputStream();
byte[] buffer = new byte[1024];
ByteArrayOutputStream _buf = new ByteArrayOutputStream();
int l;
BufferedInputStream bufin = new BufferedInputStream(mInputStream);
while ((l = bufin.read(buffer,0,1024)) != -1) {
_buf.write(buffer, 0, l);
String rec = _buf.toString("UTF-8");
Log.d("MAIN", "Read: " + rec);
result.append(rec);
}
Log.d("MAIN", "Read finished: " + result.toString());
} catch (Exception e) {
e.printStackTrace();
}
return result.toString();
}
I suspect that Android 9 Network Security does block traffic somehow. I tried to use SSLSockets, change port from 443 to e.g. 1234 - no luck.
In fact my app is being created with Qt and firstly I used Qt stuff, but having no luck - I made fallback to Android Java code within my MainActivity, that I call via JNI from Qt code. Result is the same and I have no ideas more...
Where to dig?
UPD1
When the self-signed certificate is generated with SAN containing DNS:10.10.10.1 only (without IP:10.10.10.1) SSL fails with warnings:
W System.err: javax.net.ssl.SSLPeerUnverifiedException: Hostname 10.10.10.1 not verified:
W System.err: certificate: sha1/gyr2GOhy5lA+ZAHEzh0E2SBEgx0=
W System.err: DN: CN=10.10.10.1,O=Some ltd.,L=Knoxville,ST=TN,C=US
W System.err: subjectAltNames: [10.10.10.1]
W System.err: at com.android.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:201)
W System.err: at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:149)
W ...
And conversely, with SAN IP:10.10.10.1 (without DNS: 10.10.10.1) - works as before - session established, data transferred to server and decrypted, but responses from server to client just ignored by client.
UPD2
I've also tried to use domain name some.device for the 10.10.10.1 device and issued certificate with CN and SAN DNS = some.device. It's resolved by Android 9 client, data is being sent successfully but response is still not being accepting.
Looks like Android bug.

After making additional surveying:
1. Some set of Android devices (builds), including Pixel 1, does not accept TCP session that was not finalized by mutual [FIN,ACK] and received data is not delivered to upper level of stack. Also data may not be accepted if TCP stream was not solid, with many retransmissions and Seq changing.
2. In case of using Qt - Android Network Security Configuration does not affect on communications.
3. This is not TLS related issue.

Related

Restsharp :The SSL connection could not be established

An app is run on netcore2.1.5,win7.
I use restsharp to request the API and need to take client certificates, always return the error:
The SSL connection could not be established, see inner exception.
Authentication failed, see inner exception
The inner exception content:
System.ComponentModel.Win32Exception (0x80090326): 接收到的消息异常,或格式不正确。
English version:
"The message received is abnormal or not formatted correctly"
I user postman to request the API and take the same crt file and response the
result result.
My code on below:
string clientCertfile = #"E:\https\client.crt";
var client = new RestClient("https://apiserver/iocm/app/sec/v1.1.0/login");
X509Certificate2 certificates = new X509Certificate2(clientCertfile);
//X509Certificate2 certificates = GetMyX509Certificate(clientCertfile);
client.ClientCertificates = new X509Certificate2Collection(){ certificates };
client.RemoteCertificateValidationCallback =
new RemoteCertificateValidationCallback(OnRemoteCertificateValidationCallback);
var request = new RestRequest(Method.POST);
request.AddHeader("Content-Type", "application/x-www-form-urlencoded");
request.AddParameter("appId", "rIsyJsnMtOrKfpSO90");
request.AddParameter("secret", "8v0fH0ztunjP1oXT");
IRestResponse response = client.Execute(request);
postman test image

How to connect http server websocket with ssl using vertx?

I have created two classes server and client. Server starting with ssl as below
HttpServer server =
vertx.createHttpServer(new HttpServerOptions().setSsl(true).setKeyStoreOptions(
new JksOptions().setPath("server-keystore.jks").setPassword("wibble")
));
Also one more i.e. client
vertx.createHttpClient(new HttpClientOptions().setSsl(true)).getNow(4443, "localhost", "/", resp -> {
System.out.println("Got response " + resp.statusCode());
resp.bodyHandler(body -> System.out.println("Got data " + body.toString("ISO-8859-1")));
});
While running both On client I am getting "Failed to create SSL connection". Is there any way to configure anything related to ssl?
To enable ssl in vertx you can use keystore.jks file
Then use following configuration :
HttpServerOptions secureOptions = new HttpServerOptions();
if (Configuration.SSL_enabled) {
LOG.debug("Secure Transport Protocol [ SSL/TLS ] has been enabled !!! ");
secureOptions.setSsl(true)
.setKeyStoreOptions(new JksOptions().setPath(Configuration.SSL_filename)
.setPassword(Configuration.SSL_password))
.setTrustStoreOptions(new JksOptions().setPath(Configuration.SSL_filename)
.setPassword(Configuration.SSL_password))
.addEnabledSecureTransportProtocol(Constants.TLS_VERSION_1)
.addEnabledSecureTransportProtocol(Constants.TLS_VERSION_2);
}
vertx.createHttpServer(secureOptions).requestHandler(router::accept).listen(Configuration.port);
I hope this will help you :)

How to get authenticated to Redis Cloud Memcached using Spymemcached?

I am trying to connect to Redis Cloud Memcached but get an error (below). I have checked that the username, password, host, and port are correct in the apps.redislabs.com interface. I able to connect if I disable SASL and connect unauthenticated.
How can I diagnose this?
(Using spymemcached 2.11.6.)
import net.spy.memcached.auth.*;
import net.spy.memcached.*;
...
List<InetSocketAddress> addresses = Collections.singletonList(addr);
AuthDescriptor ad = new AuthDescriptor(new String[] { "CRAM-MD5", "PLAIN" },
new PlainCallbackHandler(user, password));
MemcachedClient mc = new MemcachedClient(new ConnectionFactoryBuilder()
.setProtocol(ConnectionFactoryBuilder.Protocol.BINARY)
.setAuthDescriptor(ad).build(),
AddrUtil.getAddresses(host + ":" + port));
The stacktrace:
net.spy.memcached.MemcachedConnection: Added {QA sa=pub-memcache-14154.us-central1-1-1.gce.garantiadata.com/104.197.191.74:14514, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: SASL auth operation
net.spy.memcached.MemcachedConnection: Reconnecting due to exception on {QA sa=pub-memcache-14154.us-central1-1-1.gce.garantiadata.com/104.197.191.74:14514, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=1}
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:820)
at net.spy.memcached.MemcachedConnection.handleReadsAndWrites(MemcachedConnection.java:720)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:683)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:436)
at net.spy.memcached.MemcachedConnection.run(MemcachedConnection.java:1446)
net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=pub-memcache-14154.us-central1-1-1.gce.garantiadata.com/104.197.191.74:14514, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=1}, attempt 0.
net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for foo.
Lose the "CRAM-MD5" in your AuthDescriptior declaration.
The following works in my tests: (user, pass, and url removed)
AuthDescriptor ad = new AuthDescriptor(new String[] {"PLAIN"}, new PlainCallbackHandler(user, pass));
MemcachedClient mc = null;
try {
mc = new MemcachedClient(
new ConnectionFactoryBuilder()
.setProtocol(ConnectionFactoryBuilder.Protocol.BINARY)
.setAuthDescriptor(ad).build(),
AddrUtil.getAddresses(host + ":" + port ));
} catch (IOException e) {
// handle exception
}
mc.set("foo", 0, "bar");
String value = (String) mc.get("foo");
System.out.println(value);

Inserting client certificates into an HTTPS POST request

I'm building a hardware device which connects to the AWS IOT platform. According to the documentation the authentication with the aws iot platform is done with TLS. I have the Root CA, client key and client certificate files on the device that authorize the access. Is there a way to use these files in the HTTP header while making the POST request? If so, how? So far here is the code for the Energia IDE (based on the Arduino IDE) and using the WiFiClient methods.
if (client.sslConnect(aws_endpoint, 443))
{
Serial.println("\nConnected to AWS endpoint");
String PostData = "{\"value1\" : \"testValue\", \"value2\" : \"Hello\", \"value3\" : \"World!\" }";
request = "POST /things/";
request += thingname;
request += "/shadow";
request += " HTTP/1.1";
Serial.print("Request:\t"); Serial.println(request);
Serial.print("Post data:\t"); Serial.println(PostData);
client.println(request);
client.println("Host: ");
client.println(aws_endpoint);
client.println(":443");
client.println("User-Agent: Energia/1.1");
client.println("Connection: close");
client.println("Content-Type: application/json");
client.print("Content-Length: "); client.println(PostData.length());
client.println();
client.println(PostData);
client.println();
}
else
{
Serial.println("Connection failed");
}
Serial.println();
Serial.println("Server response:");
Serial.println();
// Capture response from the server. (10 second timeout)
long timeOut = 5000;
long lastTime = millis();
while((millis()-lastTime) < timeOut)
{ // Wait for incoming response from server
while (client.available())
{ // Characters incoming from the server
char c = client.read(); // Read characters
Serial.write(c);
}
}
This however, gives an authentication error:
HTTP/1.1 403 Forbidden
content-type: application/json
content-length: 91
date: Tue, 26 Jul 2016 11:46:59 GMT
x-amzn-RequestId: 4d5388a9-e3c4-460a-b674-c3f971f3330d
connection: Keep-Alive
x-amzn-ErrorType: ForbiddenException:
{"message":"Missing Authentication Token","traceId":"4d5388a9-e3c4-460a-b674-c3f971f3330d"}
The TLS client certificates would be sent/used as part of your client.sslConnect() call, not as part of the HTTP request. The TLS handshake (and exchange/validation of client and server certificates) happens before any HTTP message is sent.
This AWS forums post suggests that you may need to be using port 8443 (not port 443), for the shadow API. It looks like the use/requirement of TLS mutual authentication (via certificates), versus the use of AWS SIGv4 headers, is determined by AWS IOT based on the port used.
Hope this helps!

Apache HttpClient able to communicate over HTTPS when DIRECT but not via PROXY error: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated

I have read though many different examples but I am currently having difficulties trying to communicate via a proxy using HTTPS. I have a wrapper to create a Apache HttpClient as seen in the code below.
Currently if I make my call without setting up a proxy it will use my truststore from the SSLSocketFactory and correctly allow the communication via SSL. The only certificate required is a verisign server certificate which does not require authentication.
When I setup a proxy I get an error saying:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
I feel that I must be missing some type of proxy setup which makes the proxy connection use the same SSLSocketFactory?
I tested with -Djavax.net.debug=ssl and I can see a lot more SSL activity when going direct. When I use direct I can see all the keys loaded and sent with the request, when I use the proxy I only see:
httpConnector.receiver.3, setSoTimeout(30000) called
%% No cached client session
*** ClientHello, TLSv1
RandomCookie: GMT: 1307565311 bytes = { 184, 216, 5, 151, 154, 212, 232, 96, 69, 73, 240, 54, 236, 26, 8, 45, 109, 9, 192,
227, 193, 58, 129, 212, 57, 249, 205, 56 }
Session ID: {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_C
BC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH
_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH
_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA
]
Compression Methods: { 0 }
***
httpConnector.receiver.3, WRITE: TLSv1 Handshake, length = 73
httpConnector.receiver.3, WRITE: SSLv2 client hello message, length = 98
httpConnector.receiver.3, handling exception: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
httpConnector.receiver.3, SEND TLSv1 ALERT: fatal, description = unexpected_message
httpConnector.receiver.3, WRITE: TLSv1 Alert, length = 2
httpConnector.receiver.3, called closeSocket()
httpConnector.receiver.3, IOException in getSession(): javax.net.ssl.SSLException: Unrecognized SSL message, plaintext conn
ection?
httpConnector.receiver.3, called close()
httpConnector.receiver.3, called closeInternal(true)
httpConnector.receiver.3, called close()
httpConnector.receiver.3, called closeInternal(true)
2011-12-20 11:11:59,401 [httpConnector.receiver.3] INFO - The JavaScript method AddEvent threw an exception of type class co
m.alarmpoint.integrationagent.soap.exception.SOAPRequestException with message "javax.net.ssl.SSLPeerUnverifiedException: pe
er not authenticated". The exception will be propogated up the call stack.
Can anyone help out please. Here is my code for setting up the proxy and SSLSocketFactory.
var client = httpClientWrapper.getHttpClient();
var proxy = new HttpHost(PROXY_HOST, PROXY_PORT, "https");
client.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
var authpref = new ArrayList();
authpref.add(AuthPolicy.BASIC);
client.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF, authpref);
ServiceAPI.getLogger().debug("KeyStore.getDefaultType() " + KeyStore.getDefaultType());
var trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
var instream = new FileInputStream(new File("conf/my.truststore"));
try {
ServiceAPI.getLogger().debug("getting trustore");
trustStore.load(instream, "changeit".split(''));
} finally {
instream.close();
}
var socketFactory = new SSLSocketFactory(trustStore);
var sch = new Scheme("https", socketFactory, 443);
client.getConnectionManager().getSchemeRegistry().register(sch);
Stack trace:
Caused by: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(Unknown Source)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:390)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:488)
at org.apache.http.conn.scheme.SchemeSocketFactoryAdaptor.connectSocket(SchemeSocketFactoryAdaptor.java:62)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:561)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
Here's a variant on aaron's solution, in Java (vs Groovy). This solution also avoids the HttpClientWrapper class (where does that come from?), and loads the proxy's certificate directly. It is written against HttpClient 4.2 (but I think it should work with 4.0). As an added bonus it includes an example of proxy authentication for a Windows proxy such as Microsoft ForeFront TMG.
It took me long enough to piece this together that I figured I should share it:
HttpParams params = new BasicHttpParams();
DefaultHttpClient.setDefaultHttpParams( params ); // Add the default parameters to the parameter set we're building
DefaultHttpClient client = new DefaultHttpClient( params );
KeyStore trustStore = KeyStore.getInstance( KeyStore.getDefaultType() );
trustStore.load( null );
InputStream certStream = new FileInputStream( "cert-file" );
CertificateFactory cf = CertificateFactory.getInstance("X.509");
X509Certificate cert = (X509Certificate)cf.generateCertificate(certStream);
certStream.close();
trustStore.setCertificateEntry( "proxy-cert", cert );
SSLSocketFactory socketFactory = new SSLSocketFactory(trustStore);
client.getConnectionManager().getSchemeRegistry().register( new Scheme( "https", 443, socketFactory ));
client.getParams().setParameter( ConnRoutePNames.DEFAULT_PROXY,
new HttpHost( "my-proxy", 8080 ));
// These 3 lines are only needed if your proxy is Windows based & requires authentication
AuthScope scope = new AuthScope( "myproxy", 8080, null, AuthPolicy.NTLM );
Credentials credentials = new NTCredentials( "username", "changeit", "WORKSTATION", "MY-DOMAIN" );
client.getCredentialsProvider().setCredentials( scope, credentials );
HttpGet get = new HttpGet( "https://mysite.com/resource" );
String result = client.execute( get, new BasicResponseHandler() );
System.out.println( result );
I solved this. The problem which I found once debugging into HttpClients code was the way my proxy was configured and the scheme's available.
HttpRoute[{tls}->https://someproxy->https://some_endpoint:443]
The problem was that the proxy was setup for https scheme but it was actually running on http. This became a problem when the wrapper did not configure a http scheme. In the end I created the SSLSocketFactory for my truststore and a default http scheme and setup my proxy correctly.
// Setup the Keystore and Schemes for the HttpClient and Proxy
var trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
var instream = new FileInputStream(new File("conf/my.truststore"));
try {
trustStore.load(instream, "changeit".split(''));
} finally {
instream.close();
}
var socketFactory = new SSLSocketFactory(trustStore);
var schHttp = new Scheme("http", PROXY_PORT, PlainSocketFactory.getSocketFactory());
// Create the HttpClient wrapper which will have the truststore SSLSocketFactory and a default http scheme and proxy setup
httpClientWrapper = new HttpClientWrapper("some_endpoint", 443, "/", socketFactory);
var client = httpClientWrapper.getHttpClient();
var proxy = new HttpHost(PROXY_HOST, PROXY_PORT, "http");
client.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
client.getConnectionManager().getSchemeRegistry().register(schHttp);
Have you tried using the global -Dhttp.proxyHost=proxy.host.com -Dhttp.proxyPort=8080 when launching your java process to verify that the SSLSocketFactory isn't falling back to proxyless communications.