I am using the Prometheus Client library to push metrics to pushgateway. Frequently I am getting below error while pushing the metrics. How can I find the root cause of this issue?
push_to_gateway(
File "/usr/local/lib/python3.8/dist-packages/prometheus_client/exposition.py", line 285, in push_to_gateway
_use_gateway('PUT', gateway, job, registry, grouping_key, timeout, handler)
File "/usr/local/lib/python3.8/dist-packages/prometheus_client/exposition.py", line 358, in _use_gateway
handler(
File "/usr/local/lib/python3.8/dist-packages/prometheus_client/exposition.py", line 217, in handle
resp = build_opener(HTTPHandler).open(request, timeout=timeout)
File "/usr/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 1369, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/usr/lib/python3.8/urllib/request.py", line 1330, in do_open
r = h.getresponse()
File "/usr/lib/python3.8/http/client.py", line 1332, in getresponse
response.begin()
File "/usr/lib/python3.8/http/client.py", line 303, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.8/http/client.py", line 272, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without respons
We use Kubernetes internally for deploying services, and here I am trying to Push metrics to push gateway using its ingress. Changing it to use Kubernetes service names instead of ingress reduced these errors significantly but this is not a portable solution incase if service is relocated to other cluster. Solution which worked for me is to do retry using python decorator functions
Related
Running python version 3.9.1 on arch linux with OpenSSL version 1.1.1i and pyopenssl version 1.1.1i I get the following error when trying to use an https proxy with the requests module:
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 696, in urlopen
self._prepare_proxy(conn)
File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 964, in _prepare_proxy
conn.connect()
File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 359, in connect
conn = self._connect_tls_proxy(hostname, conn)
File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 496, in _connect_tls_proxy
return ssl_wrap_socket(
File "/usr/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 424, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
File "/usr/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 466, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock)
File "/usr/lib/python3.9/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib/python3.9/ssl.py", line 1040, in _create
self.do_handshake()
File "/usr/lib/python3.9/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/lib/python3.9/site-packages/urllib3/util/retry.py", line 573, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.9/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.9/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)')))
The code I am running is:
import requests
proxy = {
'https' : 'https://proxyip:proxyport'
}
requests.get("https://google.com", proxies=proxy)
No matter what https proxy I try, I get the same error. I have also reinstalled both openssl and python with no change. Any suggestions?
... line 496, in _connect_tls_proxy
Your code is trying to use the (new) support for accessing the proxy itself over HTTPS. This is done because you've explicitly given that URL as the proxy as https://... and not http://...:
'https' : 'https://proxyip:proxyport'
^^^^^^
It is very likely that the proxy itself does not support TLS connections to the proxy. Commonly HTTP proxies have a plain HTTP connections to the proxy only. They still can proxy HTTPS traffic this way, since the client will simply issue a CONNECT request to the proxy to create a tunnel and then use end-to-end TLS between client and server.
Accessing a proxy by HTTPS will add an additional layer of TLS between client and proxy, which is not supported by most proxies. Therefore, you likely need plain HTTP proxy instead:
'https' : 'http://proxyip:proxyport'
^^^^^^
Note that in older versions of the requests library both access with http:// and https:// worked. These older versions had no support for HTTPS to the proxy and simply used plain HTTP even if https:// would be specified.
Add login.microsoftonline.com;management.azure.com as exceptions will work.
it was fixed with my case by this command:
python3 -m pip install urllib3==1.22
I'm trying to user google sheets API service, which requires an HTTPS connection. I'm getting the following error
Exception Type: InsecureTransportError at my_site/google/success/
Exception Value: (insecure_transport) OAuth 2 MUST utilize https.
I am using Heroku, and on my settings it says AMC Status: ok. I verified that I'm using HTTPS by running curl -vI https://my_site/google/success
which returned:
SSL certificate verify ok
From my perspective it seems that I am using HTTPS, but I am getting this error. What could I be doing wrong? Surely I have something misconfigured Anything else I need to provide from troubleshooting? Here is the Full traceback:
traceback (most recent call last):
File "/app/.heroku/python/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/app/.heroku/python/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/app/google_api/views.py", line 56, in authorize_success
flow.fetch_token(authorization_response=code)
File "/app/.heroku/python/lib/python3.8/site-packages/google_auth_oauthlib/flow.py", line 286, in fetch_token
return self.oauth2session.fetch_token(self.client_config["token_uri"], **kwargs)
File "/app/.heroku/python/lib/python3.8/site-packages/requests_oauthlib/oauth2_session.py", line 239, in fetch_token
self._client.parse_request_uri_response(
File "/app/.heroku/python/lib/python3.8/site-packages/oauthlib/oauth2/rfc6749/clients/web_application.py", line 203, in parse_request_uri_response
response = parse_authorization_code_response(uri, state=state)
File "/app/.heroku/python/lib/python3.8/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 256, in parse_authorization_code_response
raise InsecureTransportError()
import os
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'
I am trying to get a connection with the Nessus server with the bellow command in python but it failed with an error message can you tell me what can be the cause. I have checked my network connection it is fine.
requests.post( 'https://164.99.175.30:8834/'+ '/session',data={'username':'admin','password':'micro#123'},verify=False)```
error message
Traceback (most recent call last):
File "nessus.py", line 425, in <module>
login()
File "nessus.py", line 111, in login
res = requests.post(url + '/session',data={'username':username,'password':password},verify=verify)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost', port=8834): Max retries exceeded with url: /session (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f46f2d6d410>: Failed to establish a new connection: [Errno 111] Connection refused',))
The nessus api is depreciated as of version 7.x, this is the best source I could find.
EDIT: I have found a better source directly from tenable
What has been removed from Nessus 7:
There is a restriction in scan API capabilities.
The ability to manage scans via API and CLI has been removed in v7. All Nessus Pro scanning operations must be done through the user interface.
So currently the ability of the Nessus API is as follows:
Removed the ability to run scans or reports and create new objects
The Read features, where the ability to pull scan data so GET /scan/scan ID now works again and this aids with some of the integration processes.
https://community.tenable.com/s/article/The-differences-between-Nessus-6-and-Nessus-7
This is only for Nessus pro versions
I am trying python twitter API for tweets streaming . My code after execution showing me this . Any idea how can I remove it?
Traceback (most recent call last): File
"C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py",
line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\connection.py",
line 80, in create_connection
raise err File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\connection.py",
line 70, in create_connection
sock.connect(sa) TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond
after a period of time, or established connection failed because
connected host has failed to respond
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
"C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py",
line 603, in urlopen
chunked=chunked) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py",
line 344, in _make_request
self._validate_conn(conn) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py",
line 843, in _validate_conn
conn.connect() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py",
line 316, in connect
conn = self._new_conn() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py",
line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError:
: Failed to establish a new connection: [WinError
10060] A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
"C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py",
line 449, in send
timeout=timeout File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py",
line 641, in urlopen
_stacktrace=sys.exc_info()[2]) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\retry.py",
line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError:
HTTPSConnectionPool(host='stream.twitter.com', port=443): Max retries
exceeded with url: /1.1/statuses/filter.json?delimited=length (Caused
by NewConnectionError(': Failed to establish a new connection:
[WinError 10060] A connection attempt failed because the connected
party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
"C:\Users\Administrator\Desktop\twitter\python
twitter_lucidprograming.py", line 57, in
twitter_streamer.stream_tweets(fetched_tweets_filename, hash_tag_list) File "C:\Users\Administrator\Desktop\twitter\python
twitter_lucidprograming.py", line 24, in stream_tweets
stream.filter(track=hash_tag_list) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\tweepy\streaming.py",
line 474, in filter
self._start(is_async) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\tweepy\streaming.py",
line 389, in _start
self._run() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\tweepy\streaming.py",
line 320, in _run
six.reraise(*exc_info) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\six.py",
line 693, in reraise
raise value File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\tweepy\streaming.py",
line 273, in _run
proxies = self.proxies) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py",
line 533, in request
resp = self.send(prep, **send_kwargs) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py",
line 646, in send
r = adapter.send(request, **kwargs) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py",
line 516, in send
raise ConnectionError(e, request=request) requests.exceptions.ConnectionError:
HTTPSConnectionPool(host='stream.twitter.com', port=443): Max retries
exceeded with url: /1.1/statuses/filter.json?delimited=length (Caused
by NewConnectionError(': Failed to establish a new connection:
[WinError 10060] A connection attempt failed because the connected
party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond'))
I am in China, Twitter is blocked accessing twitter through VPN. furthermore I want to know
1- once when we get authentication keys it is important to be longed in on twitter and developer account or no need of it.
2- if VPN can be issue how I can resolve it?
3- Might be a chance I didnot install tweepy right, When I run command on cmd "pip install tweepy" it says requirement already satisfied. How I can check tweepy installed well?
I think my university recently changed a firewall or filtering protocol with regards to Twitter's streaming API. However, for me to provide them more information, I'm going to have to figure out the details.
Ok, below is the simplest possible program to collect data from Twitter's stream:
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
ckey="[OMITED]"
csecret="[OMITED]"
atoken="[OMITED]"
asecret="[OMITED]"
class listener(StreamListener):
def on_data(self, data):
print(data)
return(True)
def on_error(self, status):
print status
auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken, asecret)
twitterStream = Stream(auth, listener())
twitterStream.filter(track=["car"])
When I run this program on my laptop/home desktop, it works fine. However, I have a university computer setup for data collection and it produces the following error:
Traceback (most recent call last):
File "test.py", line 25, in <module>
twitterStream.filter(track=["car"])
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 228, in filter
self._start(async)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 172, in _start
self._run()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 106, in _run
conn.connect()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1278, in connect
server_hostname=server_hostname)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 352, in wrap_socket
_context=self)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 579, in __init__
self.do_handshake()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 808, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
Now, the thing is, I can open up Safari and browse the web. Further, data collection like this use to work (as of a month ago). The error says the SSL cert is failing verification (but strangely the original OAuth seems to work -- it is the filter command kicking up the error). Is there a way to get it to spit out if that is because an intermediate cert has been put in place? Or maybe a port is being blocked? Basically, is there a way to get a lot more details so I can go the university and say: 'you are blocking X'?
Any suggestions?
I've resolved the situation. It was an error relating to SSL sites that were signed with Digicerts.