I have a rake task that pulls and parses JSON data over an SSL connection from an external API.
I use a gem that wraps this external API and have no problems running locally, but the task fails when run on heroku with #<Curl::Err::SSLCaertBadFile: Curl::Err::SSLCaertBadFile>
I installed the piggyback SSL add-on, hoping it might fix it, but no dice.
Any ideas?
UPDATE
I managed to fix it by disabling ssl verification on the curl request previously set by the following two fields:
request.ssl_verify_peer
request.ssl_verify_host
I don't know enough about SSL to know exactly why the error was caused by these settings in a heroku environment or what the implications of disabling this are, aside from reduced security.
It is a bad idea to disable certificate checking. See http://www.rubyinside.com/how-to-cure-nethttps-risky-default-https-behavior-4010.html, http://jamesgolick.com/2011/2/15/verify-none..html and associated references for more on that topic.
The issue is that your HTTP client doesn't know where to find the CA certificates bundle on heroku.
You don't mention what client you are using, but here is an example for using net/https on heroku:
require "net/https"
require "uri"
root_ca_path = "/etc/ssl/certs"
url = URI.parse "https://example.com"
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = (url.scheme == "https")
if (File.directory?(root_ca_path) && http.use_ssl?)
http.ca_path = root_ca_path
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
http.verify_depth = 5
end
request = Net::HTTP::Get.new(url.path)
response = http.request(request)
Here is an example using Faraday:
Faraday.new "https://example.com", ssl: { ca_path: "/etc/ssl/certs" }
Good luck.
Related
I have the below function in my code, which works perfectly fine when I'm not behind any proxy. In fact, without even mentioning the certifi default CA certificate, it works fine if I pass verify=TRUE, I guess, because it works in the same way.
def reverse_lookup(lat, long):
cafile=certifi.where()
params={'lat' : float(lat), 'lon' : float(long), 'format' : 'json',
'accept-language' : 'en', 'addressdetails' : 1}
response = requests.get("https://nominatim.openstreetmap.org/reverse", params=params, verify=cafile)
#response = requests.get("https://nominatim.openstreetmap.org/reverse", params=params, verify=True) <-- this works as well
result = json.loads(response.text)
return result['address']['country'], result['address']['state'], result['address']['city']
When I run the same code from within my enterprise infrastructure (where I'm behind proxy), I make some minor changes in the code mentioning the proxy as parameter in requests method:
def reverse_lookup(lat, long):
cafile=certifi.where()
proxies = {"https" : "https://myproxy.com"}
params={'lat' : float(lat), 'lon' : float(long), 'format' : 'json',
'accept-language' : 'en', 'addressdetails' : 1}
response = requests.get("https://nominatim.openstreetmap.org/reverse", params=params, verify=cafile, proxies=proxies)
result = json.loads(response.text)
return result['address']['country'], result['address']['state'], result['address']['city']
But it gives me one out of these 3 SSL errors at different times, if I set verify=True or verify=certifi.where():
CERTIFICATE_VERIFY_FAILED
UNKNOWN_PROTOCOL
WRONG_VERSION_NUMBER
Only time it works is when I completely bypass the SSL verification with verify=False
My questions are:
Since I'm sending the https request via proxy, is it ok if I bypass SSL verification ?
How to make the default context of SSL verification work in this case, when I'm behind proxy ?
Any help is appreciated. Code tested in both Python 2.7.15 and 3.9
Since I'm sending the https request via proxy, is it ok if I bypass SSL verification ?
Do you need the protection offered by HTTPS, i.e. encryption of the application data (like passwords, but also the full URL) to protect against sniffing or modifications by a malicious man in the middle? If you don't need the protection, then you can bypass certificate validation.
How to make the default context of SSL verification work in this case, when I'm behind proxy ?
The proxy is doing SSL interception and when doing this issues a new certificate for this site based on an internal CA. If this is expected (i.e. not an attack) then you need to import the CA from the proxy as trusted with verify='proxy-ca.pem'. Your IT department should be able to provide you with the proxy CA.
But it gives me one out of these 3 SSL errors at different times, if I
set verify=True or verify=certifi.where():
CERTIFICATE_VERIFY_FAILED
UNKNOWN_PROTOCOL
WRONG_VERSION_NUMBER
It should only give you CERTIFICATE_VERIFY_FAILED. The two other errors indicate wrong proxy settings, typically setting https_proxy to https://... instead of http://... (which also can be seen in your code).
I'm deploying a flask app on Heroku using a Redis premium plan. I get the following error: 'SSL Certification Verify Failed'. Attempted fixes:
Downgrading to Redis 5
Passing ssl_cert_reqs=None to the Redis constructor in redis-py
A solution to this problem could be:
Explain how to disable TLS certification on heroku redis premium plans
Explain how to make TLS certification work on heroku redis premium plans
From Heroku's docs, this may be a hint: 'you must enable TLS in your Redis client’s configuration in order to connect to a Redis 6 database'. I don't understand what this means.
I solved my problem by adding ?ssl_cert_reqs=CERT_NONE to the end of REDIS_URL in my Heroku config.
You can disable TLS certification on Heroku by downgrading to Redis 5 and passing ssl_cert_reqs=None to the Redis constructor.
$ heroku addons:create heroku-redis:premium-0 --version 5
from redis import ConnectionPool, Redis
import os
connection_pool = ConnectionPool.from_url(os.environ.get('REDIS_URL'))
app.redis = Redis(connection_pool=connection_pool, ssl_cert_reqs=None)
My mistake was not doing both at the same time.
An ideal solution would explain how to configure TLS certification for Redis 6.
The docs are actually incorrect, you have to set SSL to verify_none because TLS happens automatically.
From Heroku support:
"Our data infrastructure uses self-signed certificates so certificates
can be cycled regularly... you need to set the verify_mode
configuration variable to OpenSSL::SSL::VERIFY_NONE"
I solved this by setting the ssl_params to verify_none:
ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE } }
For me it was where I config redis (in a sidekiq initializer):
# config/initializers/sidekiq.rb
Sidekiq.configure_client do |config|
config.redis = { url: ENV['REDIS_URL'], size: 1, network_timeout: 5,
ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE } }
end
Sidekiq.configure_server do |config|
config.redis = { url: ENV['REDIS_URL'], size: 7, network_timeout: 5,
ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE } }
end
On Heroku (assuming Heroku Redis addon), the redis TLS route already has the ssl_cert_reqs param sorted out. A common oversight that can cause errors in cases like this on heroku is: using REDIS_URL over REDIS_TLS_URL.
Solution:
redis_url = os.environ.get('REDIS_TLS_URL')
This solution works with redis 6 and python on Heroku
import os, redis
redis_url = os.getenv('REDIS_URL')
redis_store = redis.from_url(redis_url, ssl_cert_reqs=None)
In my local development environment I do not use redis with the rediss scheme, so I use a function like this to allow work in both cases:
def get_redis_store():
'''
Get a connection pool to redis based on the url configured
on env variable REDIS_URL
Returns
-------
redis.ConnectionPool
'''
redis_url = os.getenv('REDIS_URL')
if redis_url.startswith('rediss://'):
redis_store = redis.from_url(
redis_url, ssl_cert_reqs=None)
else:
redis_store = redis.from_url(redis_url)
return redis_store
If using the django-rq wrapper and trying to deal with this, be sure to not use the URL parameter with SSL_CERTS_REQS. There is an outstanding issue that describes this all, but basically you need to specify each connection param instead of using the URL.
I'm having an error with gmail gem while trying to send a mail, this is working fine on local, and was working fine on heroku, but now im moving this app to a VPS server. This is the error:
e = g.compose do
to 'test#gmail.com'
subject 'testasea'
body 'test'
end
=> #<Mail::Message:25450040, Multipart: false, Headers: <From: .......>
e.deliver!
=> OpenSSL::SSL::SSLError: hostname does not match the server certificate
I've added this into an initializer file, without any luck:
ActionMailer::Base.smtp_settings = {
:enable_starttls_auto => true,
:openssl_verify_mode => 'none' # I've tested with 0 and false,
}
I tried to monkey path the class
OpenSSL::SSL::SSLSocket.class_eval do
def post_connection_check(hostname)
return true
end
end
with no luck, when I do that i receive a 535 Incorrect authentication data, however I know data is ok because i can do
g.inbox.count :read
And it returns me the right number.
I would like to know:
the incorrect certificate is the one my server (smtp client) is sending? or the one that is received by gmail smtp server?
why it works in local?
Why if i monkey path the class I received an authentication error?
Is there any workaround? i dont care if is not safe, is just a tenting application,.
This is only a guess, but if you are in a WHM VPS there is a function that restricts outgoing SMTP connections, you can find it in Tweak Settings.
Restrict outgoing SMTP to root, exim, and mailman (FKA SMTP Tweak)
It redirects all SMTP connections, If this is enabled you will receive your server self-signed ssl certificate, and if you bypass it using the monkey patch or setting configuration to dont check ssl certificate you will probably found an authentication error as you are in fact connecting to the LOCAL SMTP server.
Just disable it and test again.
When I update my Rails gems I find this errors (only in production, in development environment working good):
Curl::Err::SSLCACertificateError
Seems that is an SSL Certificate Authority Error, but why only in production is not working? And what can I do to resolve the problem?
My curb gems is v.0.8.3 with rails 3.2.8
(the mayor update was the rake, that now is v.10.0.2, but I don't know if this influence the good working of the curb gem).
FYI, this is the code that raise the error:
loginData = { :login => "myuser", :password => "mypass" }
loginJson = ActiveSupport::JSON.encode(loginData)
req = Curl::Easy.http_post("https://mysite.com", loginJson
) do |curl|
curl.headers['Content-type'] = 'application/json'
end
It sounds like you're using a self-signed SSL certificate on the server, which is fine. I do the same thing for internal services. You'll just want to make your client aware of the custom SSL certificate as well, that way it knows that it can be trusted.
Something like this should do the trick:
req = Curl::Easy.http_post("https://mysite.com", loginJson) do |curl|
curl.headers['Content-type'] = 'application/json'
curl.cacert = "/path/to/ca.crt"
curl.cert = "/path/to/cert.pem"
end
Of course, you'll probably want to extract these string constants into a config file.
I find an alternative solution, when I request a "curl" I put this additional propriety:
curl.ssl_verify_peer = false
I don't know if in term of security is the best solution, but for the moment it works...
How do I turn off certificate verification in a gem like jiraSOAP. the verification is inbuilt, but I really don't need that verification since the server & client are my properties. It should be doable to tweak the gem to stop this verification.
https://github.com/Marketcircle/jiraSOAP/blob/master/lib/jiraSOAP.rb
This cert verification error is stopping me moving on to my project. I have tried other solutions from forums, to upgrade my /etc/ssl/cert, adding OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE to my project. But none of them worked.
require 'jiraSOAP'
jira = JIRA::JIRAService.new "https://server/rpc/soap/jirasoapservice-v2?wsdl"
jira.login username, password
/usr/local/rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:799:in `connect': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (OpenSSL::SSL::SSLError)
I was able to solve it modifying the Handsoap gem to ignore cert check only for my host.
File to modify in the gems path -
handsoap-1.1.8/lib/handsoap/http/drivers/net_http_driver.rb
Modify after this new http client -
http_client = Net::HTTP.new(url.host, url.port)
Add this -
#Added these new lines below
if url.host == "server" #ignore cert check only for this host
http_client.verify_mode = OpenSSL::SSL::VERIFY_NONE
end