CAP_NET_ADMIN causes SSL to break in rust binary - ssl

I am working on a rust networking application. And I download a package from gcloud storage (using an https://... URL). I will eventually need the capabilities CAP_NET_ADMIN and CAP_NET_RAW.
This is my rust program:
pub fn download_runner_binary(bin_dir: PathBuf) -> Result<()> {
let uri = "https://example.com/foo.tar";
let response = reqwest::blocking::get(uri)?;
let tar_path = bin_dir.join("foo.tar");
let tar_path2 = tar_path.clone();
let mut dest = File::create(tar_path)?;
io::copy(&mut Cursor::new(&mut response.bytes()?), &mut dest)?;
// now seek the beginning
let dest_f = File::open(tar_path2)?;
// now extract the files here
let mut archive = Archive::new(dest_f);
archive.unpack(bin_dir)?;
Ok(())
}
It compiles fine, and I can run it just fine. Subsequently I do:
sudo setcap 'CAP_NET_RAW+eip CAP_NET_ADMIN+eip' ./target/debug/foo
and when I getcap:
getcap ./target/debug/foo
./target/debug/foo = cap_net_admin,cap_net_raw+eip
At this stage, when I run my program:
(base) ➜ wallet git:(s/permissions) ✗ ./target/debug/foo init
Initializing job runner
Error: NetworkError: `error sending request for url (https://example.com/): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914: (unable to get local issuer certificate)`
Caused by:
0: error sending request for url (https://example.com/foo.tar): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914: (unable to get local issuer certificate)
1: error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914: (unable to get local issuer certificate)
2: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914: (unable to get local issuer certificate)
3: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914:
And I can't pull things down from any SSL url. Once I remove these capabilities with setcap -r ./target/debug/foo, it is all good.
I then wrote a small python script:
#!/usr/bin/env python
import sys
import requests
resp = requests.get('https://example.com/foo.tar')
print(resp.text)
Setting the same capabilities on this script and running ./test.py completes with no issues.
Is reqwest doing something weird? Is there some issue with certificates? Is it some issue with my local SSL setup?
Any help would be appreciated.

Related

Chef Client Upgrade 16 => 17 Kitchen Converge Failure SSL Verification

Business is working on upgrading chef client throughout the orgs from 16 to 17 (17.9.26 specifically). Within our cookbooks we perform a chef http_request to test the availability of the resource within Test Kitchen/elsewhere and validate the ssl certificate. With the client upgrade we started to see a SSL verification failure indicating the CA-bundle that we're dropping out there cannot be validated. Pretty weird since nothing changed in the source for this except that we're now on client version 17 instead of 16. If I go out to the node that is built via test kitchen (we deploy test instances to AWS ec2 instances) and attempt to curl localhost (should be about the same thing as the chef http_request I guess) it will fail, but then if I specify the ca-bundle that Chef should be using by default the curl returns correct response.
`
[root#ip-REDACTED ~]# curl https://localhost:9200
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
[root#ip-REDACTED ~]# curl https://localhost:9200 --cacert /opt/chef/embedded/ssl/certs/cacert.pem
{
"name" : "ip-REDACTED",
"cluster_name" : "REDACTED-unit-green",
"clusEDACTEDter_uuid" : "R",
"version" : {
"number" : "6.4.3",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "fe40335",
"build_date" : "2018-10-30T23:17:19.084789Z",
"build_snapshot" : false,
"lucene_version" : "7.4.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
[root#ip-REDACTED ~]#
`
I can't really provide all of my cookbooks/recipes, but according to: https://docs.chef.io/chef_client_security/ the location of that ca-bundle file is correct and as mentioned and shown in the above snip it validates fine when specified. It also ran through kitchen correctly before switching client versions. Really just out here hoping that someone might have experienced similar and knows what they did to fix it.
Thanks in advance, and if I've been unclear about anything just let me know in the comments, I will try to clarify.
EDIT: Forgot to include the actual exception that comes from Chef test kitchen run:
`
================================================================================
Error executing action `put` on resource 'http_request[elasticsearch_license]'
================================================================================
OpenSSL::SSL::SSLError
----------------------
SSL Error connecting to https://localhost:9200/_xpack/license - SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate)
Resource Declaration:
---------------------
# In /tmp/kitchen/cache/cookbooks/csg_stathub_linux_643/recipes/register_license.rb
31: http_request 'elasticsearch_license' do
32: url "#{node['REDACTED']['es']['endpoint']['local_api']}/_xpack/license"
33: action :nothing
34: message lazy { ::File.read("/etc/elasticsearch/#{license_file}") }
35: provider Chef::Provider::HttpRequest
36: retries 10
37: retry_delay 5
38: headers({ 'Authorization' => "Basic #{elastic_userpass64}", 'Content-Type' => 'application/json' })
39: subscribes :put, 'http_request[elasticsearch_cluster_health]', :immediately
40: only_if { ::File.exist?("/etc/elasticsearch/#{license_file}") }
41: only_if { elasticsearch_service_running? }
42: end
43:
`

tls unsigned certificate when using terraform

The microstack.openstack project recently enabled/required tls authentication as outlined here. I am working on deploying an openstack cluster to microstack using a terraform example here. As a result of the change, I receive an unknown signed cert error when trying to create an openstack network client data source.
data "openstack_networking_network_v2" "terraform" {
name = "${var.pool}"
}
The error I get when calling terraform plan:
Error: Error creating OpenStack networking client: Post "https://XXX.XXX.XXX.132:5000/v3/auth/tokens": OpenStack connection error, retries exhausted. Aborting. Last error was: x509: certificate signed by unknown authority
with data.openstack_networking_network_v2.terraform,
on datasources.tf line 1, in data "openstack_networking_network_v2" "terraform":
1: data "openstack_networking_network_v2" "terraform" {
Is there a way to ignore the certificate error, so that I can successfully use terraform to create the openstack cluster? I have tried updating the generate-self-signed parameter, but I haven't seen any change in behavior:
sudo snap set microstack config.tls.generate-self-signed=false
I think insecure provider parameter is what you are looking for:
(Optional) Trust self-signed SSL certificates. If omitted, the OS_INSECURE environment variable is used.
Try:
provider "openstack" {
insecure = true
}
Disclaimer: I haven't tried that.
The problem was that I did not source the admin-openrc.sh file that I had downloaded from the horizon web page:
$ source admin-openrc.sh
I faced the same problem, if it could help, here my contribution :
sudo snap get microstack config.tls
Key Value
config.tls.cacert-path /var/snap/microstack/common/etc/ssl/certs/cacert.pem
config.tls.cert-path /var/snap/microstack/common/etc/ssl/certs/cert.pem
config.tls.compute {...}
config.tls.generate-self-signed true
config.tls.key-path /var/snap/microstack/common/etc/ssl/private/key.pem
In terraform directory, do :
cat /var/snap/microstack/common/etc/ssl/certs/cacert.pem : copy paste -> cacert.pem
cat /var/snap/microstack/common/etc/ssl/certs/cert.pem : copy/paste -> cert.pem
cat /var/snap/microstack/common/etc/ssl/private/key.pem : copy/past -> key.pem
And create a file in your terraform directory main.tf :
provider "openstack" {
user_name = "admin"
tenant_name = "admin"
password = "pass" (get with sudo snap get microstack config.credentials.keystone-password)
auth_url = "https://host_ip:5000/v3"
#insecure = true (uncomment & comment cacert_file + key line)
cacert_file = "/terraform_dir/cacert.pem"
#cert = "/terraform_dir/cert.pem" (if needed)
key = "/terraform_dir/private.pem"
region = "microstack" (or regionOne)
}
To finish terraform plan/apply

Jmeter testing integration with IBM dtapower

Need your help in setting the SSL manager in Jmeter for performance testing with IBM datapower.
I tried the below steps to Add cert.
• Added (* .jks /*.p12 ) file in the jmeter GUI > Options > SSL Manager.
• I tried the setting the jks file in system.properties file too.
Path : *\jMETER\apache-jmeter-3.0\apache-jmeter-3.0\bin\system.properties
# Truststore properties (trusted certificates)
#javax.net.ssl.trustStore=/path/to/[jsse]cacerts
#javax.net.ssl.trustStorePassword
#javax.net.ssl.trustStoreProvider
#javax.net.ssl.trustStoreType [default = KeyStore.getDefaultType()]
# Keystore properties (client certificates)
# Location
javax.net.ssl.keyStore=****.jks -- Added
#
#The password to your keystore
javax.net.ssl.keyStorePassword=****-- Added
#
#javax.net.ssl.keyStoreProvider
#javax.net.ssl.keyStoreType [default = KeyStore.getDefaultType()]
I dont see the SSL handshake jMETER and datapower even after i followed ablove steps. Getting below error from datapower.
12:47:26 AM ssl error 51751363 10.123.98.73 0x806000ca valcred (###_CVC_Reverse_Server): SSL Proxy Profile '###_SSLPP_Reverse_Server': connection error: peer did not send a certificate
12:47:26 AM mpgw error 51751363 10.123.98.73 0x80e00161 source-https (###_HTTPS_FSH_CON_****): Request processing failed: Connection terminated before request headers read because of the connection error occurs, from URL: 10.123.98.73:58394
12:47:26 AM ssl error 51751363 10.123.98.73 0x8120002f sslproxy (####_SSLPP_Reverse_Server): SSL library error: error:140890C7:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not
Can you please advice how to send the cert(.jks/ .p12) file from jmeter.
Change "Implementation" of your HTTP Request sampler(s) to Java. The fastest and the easiest way of doing this is using HTTP Request Defaults.
If you're using .p12 keystores you will need an extra line in the system.properties file like:
javax.net.ssl.keyStoreType=pkcs12
JMeter restart is required to pick the properties up.
See How to Set Your JMeter Load Test to Use Client Side Certificates article for more information.

Service account Google oAuth: file_get_contents .p12 failed to open stream: No such file or directory

I am trying to run Google oAuth Service Account by specifying keyfilelocation as below:
$client_id = 'xccsssds-ib5c5qmr7n34aifss95ue7qouplh7v3r.apps.googleusercontent.com'; //Client ID
$service_account_name = 'sdfdsfsxxxxxxxxxsdfdsf-ib5c5qmr7n34aifss95ue7qouplh7v3r#developer.gserviceaccount.com'; //Email Address
$key_file_location = 'My Cloud Project-e2a5b459dfd9.p12'; //key.p12
But my server is throwing error like:
file_get_contents(My Cloud Project-e2a5b459dfd9.p12): failed to open stream: No such file or directory in XXXXXXXXXX
Uncaught exception 'Google_Auth_Exception' with message 'Unable to parse the p12 file. Is this a .p12 file? Is the password correct? OpenSSL error: '
Path of the p12 file correct and gave file permissions correctly but it was throwing this error.

SSL certificate validation for a proxy connection using python requests

I'm trying to connect to a site, e.g www.yahoo.com via proxy using python requests library. So, I define proxy settings as:
HOST = 'host'
PORT = 1234 # random port I have used here
USER = 'user'
PASS = 'password'
PROXY = "%s:%s#%s:%d" % (USER, PASS, HOST, PORT)
PROXY_DICT = {
"http" : 'http://' + PROXY,
"https" : 'https://' + PROXY,
}
I use the following line of code:
requests.get('http://www.yahoo.com', proxies=proxy_dict)
This doesn't raise an exception but the response text is an error page from the proxy saying "Ensure you have installed the certificate". I have a certificate "certificate.crt", which runs fine when used with chrome browser. And the certificate is self-signed. I have tried a couple of things which raise errors.
When used the crt file as a verify param, following error:
SSLError: [Errno bad ca_certs: 'certificate.crt'] []
When used the crt file as a cert param, following error:
Error: [('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL routines', 'SSL_CTX_use_certificate_file', 'PEM lib')]
I managed to get a .pem file(I'm not sure but, it might have been generated using a key and a crt file) as well. When using it with cert param, it doesn't throw error, but the response text is again having the text "...Ensure that the certificate is installed..."
When used .pem file with verify param, following error:
SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
Now when I refer to requests docs, I see I can use two parameters verify and cert. What shall I use here? And how?