In a production setup, randomly a opensips error comes up indicating tls_read failed due to SSL_error_SSL error.
Opensips fails the tls/tcp session and a new session is created and it works fine.
Please provide any pointers on why tls_read would fail with ssl_error_ssl return code.
Opensips code invokes,
ssl = c->extra_data;
ret = SSL_read(ssl, buf, len);
if (ret >0)
{
}
else
{
err = SSL_get_error(ssl, ret);
switch (err) {
case SSL_ERROR_ZERO_RETURN:
LM_INFO("TLS connection to %s:%d closed cleanly\n",
ip_addr2a(&c->rcv.src_ip), c->rcv.src_port);
/*
* mark end of file
*/
c->state = S_CONN_EOF;
return 0;
case SSL_ERROR_WANT_READ:
case SSL_ERROR_WANT_WRITE:
return 0;
case SSL_ERROR_SYSCALL:
LM_ERR("SYSCALL error -> (%d) <%s>\n",errno,strerror(errno));
default:
LM_ERR("TLS connection to %s:%d read failed\n", ip_addr2a(&c->rcv.src_ip), c->rcv.src_port);
LM_ERR("TLS read error: %d\n",err);
c->state = S_CONN_BAD;
tls_print_errstack();
return -1;
}
I want to highlight that TLS connection was established fine and a message is successfully received and send. When the second message is received and SSL_read is invoked there is below error,
2018-05-11T11:23:16.000-04:00 [local2] [err] ffd-alpha-zone1-ccm1.ipc.com /usr/sbin/opensipsInternal[10325]: ERROR:core:_tls_read: TLS connection to 10.204.34.62:51519 read failed
2018-05-11T11:23:16.000-04:00 [local2] [err] ffd-alpha-zone1-ccm1.ipc.com /usr/sbin/opensipsInternal[10325]: ERROR:core:_tls_read: TLS read error: 1
2018-05-11T11:23:16.000-04:00 [local2] [err] ffd-alpha-zone1-ccm1.ipc.com /usr/sbin/opensipsInternal[10325]: ERROR:core:tls_print_errstack: TLS errstack: error:140890B2:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:no certificate returned
In the pcap, there is re-transmission of every tls packet both sides and when this packet is read, there seems the packet is the second portion of fragemented packet.
Thanks,
Related
I am trying to figure out what about my TLS handshake is failing. I am not exactly sure what this error code means. Can someone provide more context here?
2000-00-00T00:00:00-0000 error [[GRPC-LOGG]] : error=handshakeFailed(NIOSSL.BoringSSLError.sslError([Error: 268436496 error:10000410:SSL routines:OPENSSL_internal:SSLV3_ALERT_HANDSHAKE_FAILURE at /Users/username/Library/Developer/Xcode/DerivedData/ios-dc-bocetydygnmhxsdxqxaivnvasghk/SourcePackages/checkouts/swift-nio-ssl/Sources/CNIOBoringSSL/ssl/tls_record.cc:592])) grpc.conn.addr_local=10.220.93.246 grpc.conn.addr_remote=23.98.156.101 grpc_connection_id=C1C6376D-9F74-48AF-9D7A-D903BB68D716/0 [GRPC] grpc client error
I did take a look at the tls_record.cc file; which is reporting SSL3_AL_FATAL.
The tls_record.cc can be seen below.
tls_record.cc
f (alert_level == SSL3_AL_FATAL) {
OPENSSL_PUT_ERROR(SSL, SSL_AD_REASON_OFFSET + alert_descr);
ERR_add_error_dataf("SSL alert number %d", alert_descr);
*out_alert = 0; // No alert to send back to the peer.
return ssl_open_record_error;
}
I am using gRPC-Swift to make this call.
var clientConnection: ClientConnection.Builder
var tlsConfig = TLSConfiguration.makeClientConfiguration()
tlsConfig.certificateVerification = .noHostnameVerification
tlsConfig.trustRoots = .certificates([nioCert!])
let clientConfig = GRPCTLSConfiguration.makeClientConfigurationBackedByNIOSSL(configuration: tlsConfig, hostnameOverride: sniName)
clientConnection = ClientConnection.usingTLS(with: clientConfig, on: eventLoopGroup)
.withTLSCustomVerificationCallback({ ... })
clientConnection.connect(host: hostName, port: port)
When running curl -v https://hostname:port/foo command, this is what I get back from the server:
* Trying 12.43.425.642:443...
* Connected to q003.ed14.ws.samplecloud.dogi (12.43.425.642) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* CAfile: /etc/ssl/cert.pem
* CApath: none
* (304) (OUT), TLS handshake, Client hello (1):
* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to q003.ed14.ws.samplecloud.dogi:443
* Closing connection 0
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to q003.ed14.ws.samplecloud.dogi:443
I have added a ClientError Logger to the gRPC connection and this is what I am getting:
[!! GRPC-CLIENT-ERROR]: handshakeFailed(NIOSSL.BoringSSLError.sslError([Error: 268435581 error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED at /Users/username/Library/Developer/Xcode/DerivedData/ios-dc-bocetydygnmhxsdxqxaivnvasghk/SourcePackages/checkouts/swift-nio-ssl/Sources/CNIOBoringSSL/ssl/handshake.cc:393])) file:[<unknown>] line:[0]]
The error in the log above points back to the tls_record:
if (alert_level == SSL3_AL_FATAL) {
OPENSSL_PUT_ERROR(SSL, SSL_AD_REASON_OFFSET + alert_descr); // << this line
ERR_add_error_dataf("SSL alert number %d", alert_descr);
*out_alert = 0; // No alert to send back to the peer.
return ssl_open_record_error;
}
I think there is an issue with how I am attaching my certificates. When I view the network traffic, I do not see any client certificate showing up in the TLS handshake:
Client Certificates: -
Server Certificates: 3
It seems as though I was attaching my certificates to swift-grpc's server part of the framework and not the client, this is how you attach them for the client-side:
tlsConfig.certificateChain = [.certificate(nioCert!)]
let privateKeyNIO = try? NIOSSLPrivateKey.init(bytes: privateKeyByteAry, format: .der)
tlsConfig.privateKey = NIOSSLPrivateKeySource.privateKey(privateKeyNIO!)
Note that I am still getting the same error as reported above.
Update:
I have confirmed that the client certificates are not showing up in the request. I am not sure why this is the case; I am clearly attaching a client cert.
tlsConfig.certificateChain = [NIOSSLCertificateSource.certificate(nioCert!)]
let privateKeyNIO = try? NIOSSLPrivateKey.init(bytes: privateKeyByteAry, format: .der)
tlsConfig.privateKey = NIOSSLPrivateKeySource.privateKey(privateKeyNIO!)
I am trying to understand the mutual TLS working, I have the following example:
I have a client who wants to connect to server "svc1.example.com"
but the server has a
server certificate with a commonName as "svc1.example.cloud" and a SAN
as "svc.example.test.cloud".
Now when I make a GET request, I get the following:
x509: certificate is valid for svc.example.test.cloud, not svc1.example.com.
So, my question is should I make a the TLS clientConfig changes to include the servername? or should I add a custom verifyPeerCertificate function in the TLS client config, something like below?
Please, let me know, what should be the Servername and what should I check for in the verifyPeerCertificate function.
func customverify(customCName func(*x509.Certificate) bool) func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
if customCName == nil {
return nil
}
return func(_ [][]byte, verifiedChains [][]*x509.Certificate) error {
for _, certs := range verifiedChains {
leaf := certs[0]
if customCName(leaf) {
return nil
}
}
return fmt.Errorf("client identity verification failed")
}
}
func configureClient(certFile, keyFile string) (*http.Client, error) {
certpool, err := addRootCA()
if err != nil {
return nil, err
}
cert, err := tls.LoadX509KeyPair(certFile, keyFile)
if err != nil {
return nil, err
}
transport := ytls.NewClientTransport()
transport.TLSClientConfig.Certificates = []tls.Certificate{cert}
transport.TLSClientConfig.RootCAs = certpool
//transport.TLSClientConfig.ServerName = expectedCName
transport.TLSClientConfig.VerifyPeerCertificate = customverify(func(cert *x509.Certificate) bool {
return cert.Subject.CommonName == "svc1.example.cloud"
})
httpClient := &http.Client{Transport: transport}
return httpClient, nil
}
Since x509: certificate is valid for svc.example.test.cloud, so transport.TLSClientConfig.ServerName = "svc.example.test.cloud"
From https://golang.org/pkg/crypto/tls/#Config
VerifyPeerCertificate, if not nil, is called after normal
certificate verification by either a TLS client or server. It
receives the raw ASN.1 certificates provided by the peer and also
any verified chains that normal processing found. If it returns a
non-nil error, the handshake is aborted and that error results.
If normal verification fails then the handshake will abort before
considering this callback. If normal verification is disabled by
setting InsecureSkipVerify, or (for a server) when ClientAuth is
RequestClientCert or RequireAnyClientCert, then this callback will
be considered but the verifiedChains argument will always be nil.
VerifyPeerCertificate func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error
So if normal verification fails, then VerifyPeerCertificate won't get called. Also if normal verification is passed, i don't think you need this extra check VerifyPeerCertificate.
What I'm trying to do:
Build a package (later usage) that provides a method to execute a get-request to any page through a given socks5 proxy.
My problem:
When ever I try to request a page with SSL (https) I get the following error:
Error executing request Get https://www.xxxxxxx.com: socks connect tcp 83.234.8.214:4145->www.xxxxxxx.com:443: EOF
However requesting http://www.google.com is working fine. So there must be a problem with the SSL connection. Can't imagine why this isn't working as I'm not very experienced with SSL-connections. End of file makes no sense to me.
My current code:
func main() {
// public socks5 - worked when I created this question
proxy_addr := "83.234.8.214:4145"
// With this address I get the error
web_addr := "https://www.whatismyip.com"
// Requesting google works fine
//web_addr := "http://www.google.com"
dialer, err := proxy.SOCKS5("tcp", proxy_addr, nil, proxy.Direct)
handleError(err, "error creating dialer")
httpTransport := &http.Transport{}
httpClient := &http.Client{Transport: httpTransport}
httpTransport.DialTLS = dialer.Dial
req, err := http.NewRequest("GET", web_addr, nil)
handleError(err, "error creating request")
httpClient.Timeout = 5 * time.Second
resp, err := httpClient.Do(req)
handleError(err, "error executing request")
defer resp.Body.Close()
b, err := ioutil.ReadAll(resp.Body)
handleError(err, "error reading body")
fmt.Println(string(b))
}
func handleError(err error, msg string) {
if err != nil {
log.Fatal(err)
}
}
So what am I missing in here to deal with ssl-connections?
Thank you very much.
Edit 1:
In case someone would think this is an issue with whatismyip.com I've done some more tests:
https://www.google.com
EOF error
https://stackoverflow.com
EOF error
https://www.youtube.com/
EOF error
Connection between your program and your socks5 proxy goes not through SSL/TLS
So you should change line
httpTransport.DialTLS = dialer.Dial
to
httpTransport.Dial = dialer.Dial
I checked https://www.whatismyip.com and https://www.google.com.
URLs are downloaded fine.
For test I set up 3proxy service on my server, test your code with fixed line and check 3proxy logs.
All made requests was in proxy server logs.
If you need more help - please let me know, I'll help
Things to notice:
Socks5 proxies need to support SSL connections.
The code from the question won't work with this answer as the proxy (used in the code) isn't supporting SSL connections.
I want to install SSL(Comodo wildcard certificate, ex: "*.test.com")
in Vapor Web framework, the "servers.json" I got is:
{
"default": {
"port": "$PORT:443",
"host": "api.test.com",
"securityLayer": "tls",
"tls": {
"certificates": "chain",
"certificateFile": "/path/ssl-bundle.crt",
"chainFile": "/path/ssl-bundle.crt",
"privateKeyFile": "/path/key.pem",
"signature": "signedFile",
"caCertificateFile": "/path/AddTrustExternalCARoot.crt"
}
}
}
I already make sure that "public/private" key matches already using openssl command. And about the certificateFile part like "ssl-bundle.crt", I also tried "*.test.com.crt" with the "key.pem" as well(still pass the validation using openssl, the only difference is one is test.com's certificate, the other is bundle certificate, combined by correct orders already.). Besides, all certs and key's format are correct as well. And I also make sure the cert/key files location is correct so that the Vapor can find these files. But I still can't launch the server correctly, and always display the error.
I try to locate the exact location in xcode, but I can only see it fails in this method: "tls_accept_fds()", which is in tls_server.c of CLibreSSL library.
Also, I saw the error message the xcode displayed to me:
After use debug mode to trace, I can only know that it seems the program throws the error in "SSL_set_rfd()" or "SSL_set_rfd()", but I don't know exactly. The xcode only shows this to me, and I can't find any other error messages in the debug console. As result, so far I can only make sure that the error should be in this block:
int
tls_accept_fds(struct tls *ctx, struct tls **cctx, int fd_read, int fd_write)
{
struct tls *conn_ctx = NULL;
// I pass this block
if ((ctx->flags & TLS_SERVER) == 0) {
tls_set_errorx(ctx, "not a server context");
goto err;
}
// I pass this block
if ((conn_ctx = tls_server_conn(ctx)) == NULL) {
tls_set_errorx(ctx, "connection context failure");
goto err;
}
// I pass this block
if ((conn_ctx->ssl_conn = SSL_new(ctx->ssl_ctx)) == NULL) {
tls_set_errorx(ctx, "ssl failure");
goto err;
}
// I pass this block
if (SSL_set_app_data(conn_ctx->ssl_conn, conn_ctx) != 1) {
tls_set_errorx(ctx, "ssl application data failure");
goto err;
}
// The error occurs here, in SSL_set_rfd or SSL_set_wfd, it will then go to err part: "*cctx = NULL;", not even go into the if block.
if (SSL_set_rfd(conn_ctx->ssl_conn, fd_read) != 1 ||
SSL_set_wfd(conn_ctx->ssl_conn, fd_write) != 1) {
tls_set_errorx(ctx, "ssl file descriptor failure");
goto err;
}
*cctx = conn_ctx;
return (0);
err:
tls_free(conn_ctx);
*cctx = NULL;
return (-1);
}
So, the above is all the info I got right now, and I can't find the solution on the internet for several days already...
Could anyone give me any hint about how to install SSL in Vapor web framework? I can correctly install the SSL in Apache, Nginx, Tomcat, etc already. But never success in Vapor, it seems like C library issue, but I don't know the real reason why it fails, thank you very much for any possible help.
The bug has been found and fixed here: https://github.com/vapor/tls/pull/27
So I created this test case (a mish mash of existing alamofire test cases):
func testHTTPBasicAuthenticationWithValidCredentialsSelfSignedSuccess() {
// Given
let expectation = expectationWithDescription("\(URLString) 200")
var request: NSURLRequest?
var response: NSHTTPURLResponse?
var data: NSData?
var error: NSError?
setRootCertificateAsLoneAnchorCertificateForTrust(serverTrust)
let policies = [SecPolicyCreateBasicX509()]
SecTrustSetPolicies(serverTrust, policies)
// When
Alamofire.request(.GET, URLString)
.authenticate(user: user, password: password)
.response { responseRequest, responseResponse, responseData, responseError in
request = responseRequest
response = responseResponse
data = responseData
error = responseError
expectation.fulfill()
}
waitForExpectationsWithTimeout(defaultTimeout, handler: nil)
// Then
XCTAssertNotNil(request, "request should not be nil")
XCTAssertNotNil(response, "response should not be nil")
XCTAssertTrue(data?.length > 0, "Data not found.")
XCTAssertNil(error, "error should be nil")
}
The Root CA is a self-signed cert for an existing sight that works.
But I get this error back:
Test Suite 'Selected tests' started at 2015-08-12 12:46:37.512 Test
Suite 'StageAuthentication' started at 2015-08-12 12:46:37.514 Test
Case '-[Alamofire_iOS_Tests.StageAuthentication
testHTTPBasicAuthenticationWithValidCredentialsSelfSignedSuccess]'
started. 2015-08-12 12:46:37.663 xctest[3641:12220875]
NSURLSession/NSURLConnection HTTP load failed
(kCFStreamErrorDomainSSL, -9813)
/Users/wynne_b/Alamofire/Tests/QuestAuthentication.swift:309: error:
-[Alamofire_iOS_Tests.StageAuthentication testHTTPBasicAuthenticationWithValidCredentialsSelfSignedSuccess] :
XCTAssertNotNil failed - response should not be nil
/Users/wynne_b/Alamofire/Tests/QuestAuthentication.swift:310: error:
-[Alamofire_iOS_Tests.StageAuthentication testHTTPBasicAuthenticationWithValidCredentialsSelfSignedSuccess] :
XCTAssertTrue failed - Data not found.
/Users/wynne_b/Alamofire/Tests/QuestAuthentication.swift:311: error:
-[Alamofire_iOS_Tests.StageAuthentication testHTTPBasicAuthenticationWithValidCredentialsSelfSignedSuccess] :
XCTAssertNil failed: "Error Domain=NSURLErrorDomain Code=-1202 "The
certificate for this server is invalid. You might be connecting to a
server that is pretending to be “portal.care180.com” which could put
your confidential information at risk."
UserInfo={NSLocalizedDescription=The certificate for this server is
invalid. You might be connecting to a server that is pretending to be
“portal.care180.com” which could put your confidential information at
risk., NSLocalizedRecoverySuggestion=Would you like to connect to the
server anyway?, _kCFStreamErrorDomainKey=3,
NSUnderlyingError=0x7ae21c60 {Error Domain=kCFErrorDomainCFNetwork
Code=-1202 "(null)"
UserInfo={_kCFStreamPropertySSLClientCertificateState=0,
_kCFNetworkCFStreamSSLErrorOriginalValue=-9813, _kCFStreamErrorCodeKey=-9813, _kCFStreamErrorDomainKey=3, kCFStreamPropertySSLPeerTrust=,
kCFStreamPropertySSLPeerCertificates={type = immutable, count = 1, values = ( 0 :
)}}}, _kCFStreamErrorCodeKey=-9813,
NSErrorFailingURLStringKey=https://portal.care180.com/services/init.json,
NSErrorPeerCertificateChainKey={type =
immutable, count = 1, values = ( 0 : )},
NSErrorClientCertificateStateKey=0,
NSURLErrorFailingURLPeerTrustErrorKey=,
NSErrorFailingURLKey=https://portal.care180.com/services/init.json}" -
error should be nil Test Case
'-[Alamofire_iOS_Tests.StageAuthentication
testHTTPBasicAuthenticationWithValidCredentialsSelfSignedSuccess]'
failed (0.156 seconds). Test Suite 'StageAuthentication' failed at
2015-08-12 12:46:37.671. Executed 1 test, with 3 failures (0
unexpected) in 0.156 (0.157) seconds Test Suite 'Selected tests'
failed at 2015-08-12 12:46:37.672. Executed 1 test, with 3 failures
(0 unexpected) in 0.156 (0.160) seconds Program ended with exit code:
1
Sorry for being dense: what am I doing wrong? Or is there an Alamofire test that does this with a different cert and host?
I confused the root and the leaf. My bad.