Python 'requests' [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) - ssl

I have a problem verifiying a HTTPS endpoint when providing a specific certificate path to the 'verify' option; setting 'verify' to true DOES work however:
import requests
def run_tests():
url="https://www.google.com"
certfilename="google.crt"
generate_cert_file( certfilename )
response = requests.get( url, verify=False )
print "URL:%s, Verify=False. Result: %s"%(url, response.status_code )
response = requests.get( url, verify=True )
print "URL:%s, Verify=True. Result: %s"%(url, response.status_code )
response = requests.get( url, verify=certfilename )
print "URL:%s, Verify=%s. Result: %s"%(url, certfilename, response.status_code )
def generate_cert_file( filename ):
cert_text=('''\
-----BEGIN CERTIFICATE-----
MIIEgDCCA2igAwIBAgIIIxZ+YmNtB9AwDQYJKoZIhvcNAQELBQAwSTELMAkGA1UE
BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl
cm5ldCBBdXRob3JpdHkgRzIwHhcNMTUwOTA5MjI1MzM5WhcNMTUxMjA4MDAwMDAw
WjBoMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwN
TW91bnRhaW4gVmlldzETMBEGA1UECgwKR29vZ2xlIEluYzEXMBUGA1UEAwwOd3d3
Lmdvb2dsZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDqH1dz
jOr385g9DXVZowTLkhjCoYGQ/zOLzslSFowYqZnlMiE7iZ8Vm9/iJfhStSMoA6gZ
87uqLENCi3NK+pk4+n1SIPRZS0jorb1jsyenMMy6Fxb9sQe3ndqZpWBtsW5ezW9t
7q5HWA1cNm1KSi2fmwZpDrSaN0TYQdFbRAz6cPf6J/P66qyEC7OInFgLsy4FDdp8
itpjCo/gnk2gz1vpWjiRYLCkz/dlFY3M3kR/s7YcJa7UP7BZ+QmmDfN3y4mxmEfn
Cg8rB25ZbD+oQ+Hua3/oMrx4r3lliou3yrD08/ABEqs16EEPX5wFHtI4CQYAUt5E
rEDl36bpvFD0QkfLAgMBAAGjggFLMIIBRzAdBgNVHSUEFjAUBggrBgEFBQcDAQYI
KwYBBQUHAwIwGQYDVR0RBBIwEIIOd3d3Lmdvb2dsZS5jb20waAYIKwYBBQUHAQEE
XDBaMCsGCCsGAQUFBzAChh9odHRwOi8vcGtpLmdvb2dsZS5jb20vR0lBRzIuY3J0
MCsGCCsGAQUFBzABhh9odHRwOi8vY2xpZW50czEuZ29vZ2xlLmNvbS9vY3NwMB0G
A1UdDgQWBBQHiMS+8X3+WcfbMQymf9yX9XA1yDAMBgNVHRMBAf8EAjAAMB8GA1Ud
IwQYMBaAFErdBhYbvPZotXb1gba7Yhq6WoEvMCEGA1UdIAQaMBgwDAYKKwYBBAHW
eQIFATAIBgZngQwBAgIwMAYDVR0fBCkwJzAloCOgIYYfaHR0cDovL3BraS5nb29n
bGUuY29tL0dJQUcyLmNybDANBgkqhkiG9w0BAQsFAAOCAQEAUvY5/JbZLfahjC4F
IZU0jVtGRBDa6tXLhCHAdwLHYLQdHn74oQ8Y1vxL0AYd7V2SBAgrDjCoK9bDQbQi
UZ7xwG9K2O75bS2qYc/OlZLJr0Gdfs71ZpQzlv14SGBXvwPuD6noj+hiZjqICd6t
3Rd5oIFiZvkhNDdN6Uag28rIfR8HECdlkbZNZeLZnyoIaxsprANvlEkY0wEbn1K9
kww3/SYKbdPJ8VQSUOtWgGsO1RPFaE4PP7hCg8Q062+mmCs0ZMQSvrzzv7JQsO5J
EkdG6691HdVA6z/rRGGX8E6assTnJLmVMxRayV+Do07KvywLONTInUWue8heHUSX
EHJZbg==
-----END CERTIFICATE-----\
''')
with open(filename, "wb") as output:
output.write(cert_text)
if __name__=='__main__':
run_tests()
Am I doing something wrong here ? (I embedded the cert inline to make the code easier to run without having to provide a separate cert file)
I'm fetched 'requests' down from the git repository - the newest version TAG in the history is V2.7.0, and the latest commit is "46ff1a9a543cc4d33541aa64c94f50f0a698736e"
EDIT: I actually had the wrong certificate here (thanks Steffen Ullrich for pointing this out) : but I have now confirmed I have the correct cert/endpoint: and I get the same error.
I retrieved the cert like this:
openssl s_client -connect www.google.com:443
And just copied the cert details into the python program.
The issue is actually happening for my own in-house systems as well - using self-signed certs (which is my real use-case).
Alternatively : where does the 'verify=True' option actually look for trusted certs/CAs ? (On Java it would be 'cacerts' - not sure what the equivalent here is for Python/requests ?).
I'm on a Windows platform here.

You are using the certificate which is only valid for www.google.co.uk, but access www.google.com. Thus the certificate can not match at all. And I'm not sure if using the host certificate instead of an issuer certificate (i.e. root CA or intermediate CA) will work at all.

Can you try this:
s = Session()
req = Request('POST', 'https://www.google.com')
prepped = s.prepare_request(req)
resp = s.send(prepped, verify=False, cert=CERT_PATH)
if resp.status_code == 200:...

Related

Opening and checking a Pem file in SWI-Prolog

How do I open a Pem file to check a) That the 'Not before' and 'Not after' dates are okay and b) That there is a chain of certs in the pem file to a route certificate authority?
I have tried:
:-use_module(library(http/http_client)).
url('http://fm4dd.com/openssl/source/PEM/certs/512b-rsa-example-cert.pem').
url_data(Url,D):-
http_get(Url,D,[to(string)]).
url_data1(Url,Certificate):-
http_get(Url,D,[to(stream(Stream))]),
load_certificate(Stream, Certificate),
close(Stream).
url_data/1 works in that it returns the pem file as a string. But url_data1/1 does not work. It is intended to return each certificate(s) as a list of terms.
* Update *
I have:
url_data1(Url,Certs):-
http_open(Url,Stream,[]),
all_certs(Stream,Certs),
forall(member(C,Certs),my_validate(C)),
close(Stream).
all_certs(Stream,[C1|Certs]):-
catch(load_certificate(Stream,C1),_,fail),
all_certs(Stream,Certs),!.
all_certs(_Stream,[]).
my_validate(C):-
memberchk(to_be_signed(Signed),C),
memberchk(key(Key),C),
memberchk(signature(Signature),C),
memberchk(signature_algorithm(A),C),
algo_code(A,Code),
rsa_verify(Key,Signed,Signature,[type(Code)]).
algo_code('RSA-SHA256',sha256).
algo_code('RSA-SHA1',sha1).
Which fails. What are the correct arguments?
You can use http_open/3 in combination with load_certificate/2:
?- url(Url),
http_open(Url, Stream, []),
load_certificate(Stream, Certificate),
maplist(portray_clause, Certificate).
Yielding:
version(0).
notbefore(1345613214).
notafter(1503293214).
serial('0DFA').
subject(['C'='JP', 'ST'='Tokyo', 'O'='Frank4DD', 'CN'='www.example.com']).
hash("071CB94F0CC8514D024124708EE8B2687BD7D9D5").
signature("14B64CBB817933E671A4DA516FCB081D8D60ECBC18C7734759B1F22048BB61FAFC4DAD898DD121EBD5D8E5BAD6A636FD745083B60FC71DDF7DE52E817F45E09FE23E79EED73031C72072D9582E2AFE125A3445A119087C89475F4A95BE23214A5372DA2A052F2EC970F65BFAFDDFB431B2C14A9C062543A1E6B41E7F869B1640").
signature_algorithm('RSA-SHA1').
etc.
Check the issuer_name/1 element to obtain the issuer. You can use load_certificate/2 again to read further certificates from the file.
Note that a much more typical way to validate the certificate chain is to establish a secure connection (via HTTPS), and then to use ssl_peer_certificate/2 or ssl_peer_certificate_chain/2 on the stream to obtain the peer certificate and certificate chain.
To validate the chain, you must verify the signature/1 fields, which contain the digital signatures of the to_be_signed/1 portions of the certificate, signed by the respective issuer.
You can use library(crypto) to verify the signatures.

How to generate the Apple certificate properly?

I've followed the tutorial on this website but if I want to save the results i get a message that prevents me from saving the results:
1 error prohibited this sender from being saved:
APNS certificate or private key is not valid
But why do I get this error?
I've tried a couple of methods, placing it with -----BEGIN CERTIFICATE----- and placing it without it, but nothing works. I've created a couple of certificates but each one is false according too the error.
Have I forgot something to do?
placing it with -----BEGIN CERTIFICATE----- and placing it without it, but nothing works.
You must copy and paste the whole content of the files, including -----BEGIN CERTIFICATE----- and ----- END CERTIFICATE-----.
But why do I get this error?
You have probably done an error in one of the steps: that error means that the certificates are not valid. They are not valid in general (e.g. malformed, etc.): this is not something specific related to push notifications, APNs or Pushpad.
If you know Ruby, you can see what is the exception raised by that certificates by running the following snippet:
private_key = OpenSSL::PKey.read apns_private_key
certificate = OpenSSL::X509::Certificate.new apns_certificate
pkcs12 = OpenSSL::PKCS12.create(nil, nil, private_key, certificate)
pkcs12.to_der
Otherwise contact support#pushpad.xyz and attach your cert/private key so that I can try to help.

Clojure SSL specify local cert to use as a client side certificate

This comes from the python documentation for Python "Requests" http library
"You can also specify a local cert to use as client side certificate, as a single file (containing the private key and the certificate) or as a tuple of both file’s path":
>>> requests.get('https://kennethreitz.com', cert=('/path/server.crt', '/path/key'))
<Response [200]>
http://docs.python-requests.org/en/latest/user/advanced/
What's a good way to do the same thing in Clojure ? I looked at clj-http and http-kit but did not see an example
Have you seen async-http-client?
It has specific tests for cert handling you can view here. The API docs are here, particularly relevant would be the namespace http.async.client.cert.
From that test, a typical example of loading keystore and certificate is:
(def ks-file "test-resources/keystore.jks")
(def cert-file "test-resources/certificate.crt")
(def password "secret")
(defn load-test-certificate [] (load-x509-cert cert-file))
(defn load-test-keystore [] (load-keystore (resource-stream ks-file) password))

SSL certificate validation for a proxy connection using python requests

I'm trying to connect to a site, e.g www.yahoo.com via proxy using python requests library. So, I define proxy settings as:
HOST = 'host'
PORT = 1234 # random port I have used here
USER = 'user'
PASS = 'password'
PROXY = "%s:%s#%s:%d" % (USER, PASS, HOST, PORT)
PROXY_DICT = {
"http" : 'http://' + PROXY,
"https" : 'https://' + PROXY,
}
I use the following line of code:
requests.get('http://www.yahoo.com', proxies=proxy_dict)
This doesn't raise an exception but the response text is an error page from the proxy saying "Ensure you have installed the certificate". I have a certificate "certificate.crt", which runs fine when used with chrome browser. And the certificate is self-signed. I have tried a couple of things which raise errors.
When used the crt file as a verify param, following error:
SSLError: [Errno bad ca_certs: 'certificate.crt'] []
When used the crt file as a cert param, following error:
Error: [('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL routines', 'SSL_CTX_use_certificate_file', 'PEM lib')]
I managed to get a .pem file(I'm not sure but, it might have been generated using a key and a crt file) as well. When using it with cert param, it doesn't throw error, but the response text is again having the text "...Ensure that the certificate is installed..."
When used .pem file with verify param, following error:
SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]
Now when I refer to requests docs, I see I can use two parameters verify and cert. What shall I use here? And how?

Use SOAP::Lite based on https, certificate verify failed

I constructed a apache mod_perl web service based on SSL.Of course, From my browser, I can access the web service using https (Of cource,I add my self-signed CA cert to brower's trust list) access the web service,but when using SOAP::Lite , I failed.
This is my source code:
$ENV{HTTPS_CERT_FILE} = '/etc/pki/tls/mycerts/client.crt';
$ENV{HTTPS_KEY_FILE} = '/etc/pki/tls/mycerts/client.key';
#$ENV{HTTPS_CA_FILE} = '/etc/pki/tls/mycerts/ca.crt';
#$ENV{HTTPS_CA_DIR} = '/etc/pki/tls/mycerts/ca.key';
#$ENV{HTTPS_VERSION} = 3;
$ENV{SSL_ca_file}='/etc/pki/tls/mycerts/ca.crt';
$ENV{SSL_ca_pah}='/etc/pki/tls/mycerts/';
#$ENV{SSL_cert_file}='/etc/pki/tls/mycerts/client.key';
#$ENV{SSL_key_file}='/etc/pki/tls/mycerts/client.crt';
$ENV{PERL_LWP_SSL_CA_FILE}='/etc/pki/tls/mycerts/ca.crt';
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME}=1;
#$ENV{PERL_LWP_SSL_CA_PATH}='/etc/pki/tls/mycerts/';
use SOAP::Lite;
my $name = "wuchang";
print "\n\nCalling the SOAP Server to say hello\n\n";
print SOAP::Lite
-> uri('http://localhost/mod_perl_rules1')
-> proxy('https://localhost/mod_perl_rules1')
-> result;
I get the response:
500 Can't connect to localhost:443 (certificate verify failed) at /root/Desktop/test.pl line 18
I really cannot debug this.I don't know if my certificate format is incorrect.I use openssl to generate my cert,including client cert ,server cert and my self-signed ca cert and I make CA sign the client and server cert.I really don't know what is going wrong/.
Simply tell it not to check the certificate. Set SSL Verify to zero like this:
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME}=0;