I'm trying to user google sheets API service, which requires an HTTPS connection. I'm getting the following error
Exception Type: InsecureTransportError at my_site/google/success/
Exception Value: (insecure_transport) OAuth 2 MUST utilize https.
I am using Heroku, and on my settings it says AMC Status: ok. I verified that I'm using HTTPS by running curl -vI https://my_site/google/success
which returned:
SSL certificate verify ok
From my perspective it seems that I am using HTTPS, but I am getting this error. What could I be doing wrong? Surely I have something misconfigured Anything else I need to provide from troubleshooting? Here is the Full traceback:
traceback (most recent call last):
File "/app/.heroku/python/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/app/.heroku/python/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/app/google_api/views.py", line 56, in authorize_success
flow.fetch_token(authorization_response=code)
File "/app/.heroku/python/lib/python3.8/site-packages/google_auth_oauthlib/flow.py", line 286, in fetch_token
return self.oauth2session.fetch_token(self.client_config["token_uri"], **kwargs)
File "/app/.heroku/python/lib/python3.8/site-packages/requests_oauthlib/oauth2_session.py", line 239, in fetch_token
self._client.parse_request_uri_response(
File "/app/.heroku/python/lib/python3.8/site-packages/oauthlib/oauth2/rfc6749/clients/web_application.py", line 203, in parse_request_uri_response
response = parse_authorization_code_response(uri, state=state)
File "/app/.heroku/python/lib/python3.8/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 256, in parse_authorization_code_response
raise InsecureTransportError()
import os
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'
Related
I am trying to send message from iot device to iot hub through iot edge gateway. The iot edge gateway uses self signed certificate. Although I have added root CA certificate to iot device, I always get error that it can not verify certificate. Please help advise how I can disable certificate verification.
Code: https://github.com/Azure/azure-iot-sdk-python/blob/main/samples/async-edge-scenarios/send_message_downstream.py
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.9/site-packages/azure/iot/device/common/mqtt_transport.py", line 396, in connect
rc = self._mqtt_client.connect(
File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 914, in connect
return self.reconnect()
File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1073, in reconnect
sock.do_handshake()
File "/usr/lib/python3.9/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.9/site-packages/azure/iot/device/iothub/aio/async_clients.py", line 33, in handle_result
return await callback.completion()
File "/home/pi/.local/lib/python3.9/site-packages/azure/iot/device/common/async_adapter.py", line 91, in completion
return await self.future
File "/home/pi/.local/lib/python3.9/site-packages/azure/iot/device/common/pipeline/pipeline_stages_mqtt.py", line 206, in _run_op
self.transport.connect(password=password)
File "/home/pi/.local/lib/python3.9/site-packages/azure/iot/device/common/mqtt_transport.py", line 409, in connect
raise exceptions.TlsExchangeAuthError() from e
azure.iot.device.common.transport_exceptions.TlsExchangeAuthError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/pi/connect.py", line 54, in <module>
asyncio.run(main())
File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/home/pi/connect.py", line 35, in main
await device_client.connect()
File "/home/pi/.local/lib/python3.9/site-packages/azure/iot/device/aio/patch_documentation.py", line 14, in connect
return await super(IoTHubDeviceClient, self).connect()
File "/home/pi/.local/lib/python3.9/site-packages/azure/iot/device/iothub/aio/async_clients.py", line 231, in connect
await handle_result(callback)
File "/home/pi/.local/lib/python3.9/site-packages/azure/iot/device/iothub/aio/async_clients.py", line 45, in handle_result
raise exceptions.ClientError("Error in the IoTHub client due to TLS exchanges.") from e
azure.iot.device.exceptions.ClientError: Error in the IoTHub client due to TLS exchanges.
I tried to look for a way to disable certificate verification but I can not find out
It's not possible to disable the certificate verification - it's required for a downstream device to connect to an IoT Edge gateway.
Did you configure the gateway with the certificate by editing the config.toml file? For more information, see https://learn.microsoft.com/azure/iot-edge/how-to-create-transparent-gateway?view=iotedge-1.4&tabs=iotedge#configure-certificates-on-device
Running python version 3.9.1 on arch linux with OpenSSL version 1.1.1i and pyopenssl version 1.1.1i I get the following error when trying to use an https proxy with the requests module:
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 696, in urlopen
self._prepare_proxy(conn)
File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 964, in _prepare_proxy
conn.connect()
File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 359, in connect
conn = self._connect_tls_proxy(hostname, conn)
File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 496, in _connect_tls_proxy
return ssl_wrap_socket(
File "/usr/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 424, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
File "/usr/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 466, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock)
File "/usr/lib/python3.9/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib/python3.9/ssl.py", line 1040, in _create
self.do_handshake()
File "/usr/lib/python3.9/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/lib/python3.9/site-packages/urllib3/util/retry.py", line 573, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.9/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.9/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)')))
The code I am running is:
import requests
proxy = {
'https' : 'https://proxyip:proxyport'
}
requests.get("https://google.com", proxies=proxy)
No matter what https proxy I try, I get the same error. I have also reinstalled both openssl and python with no change. Any suggestions?
... line 496, in _connect_tls_proxy
Your code is trying to use the (new) support for accessing the proxy itself over HTTPS. This is done because you've explicitly given that URL as the proxy as https://... and not http://...:
'https' : 'https://proxyip:proxyport'
^^^^^^
It is very likely that the proxy itself does not support TLS connections to the proxy. Commonly HTTP proxies have a plain HTTP connections to the proxy only. They still can proxy HTTPS traffic this way, since the client will simply issue a CONNECT request to the proxy to create a tunnel and then use end-to-end TLS between client and server.
Accessing a proxy by HTTPS will add an additional layer of TLS between client and proxy, which is not supported by most proxies. Therefore, you likely need plain HTTP proxy instead:
'https' : 'http://proxyip:proxyport'
^^^^^^
Note that in older versions of the requests library both access with http:// and https:// worked. These older versions had no support for HTTPS to the proxy and simply used plain HTTP even if https:// would be specified.
Add login.microsoftonline.com;management.azure.com as exceptions will work.
it was fixed with my case by this command:
python3 -m pip install urllib3==1.22
I am trying to get a connection with the Nessus server with the bellow command in python but it failed with an error message can you tell me what can be the cause. I have checked my network connection it is fine.
requests.post( 'https://164.99.175.30:8834/'+ '/session',data={'username':'admin','password':'micro#123'},verify=False)```
error message
Traceback (most recent call last):
File "nessus.py", line 425, in <module>
login()
File "nessus.py", line 111, in login
res = requests.post(url + '/session',data={'username':username,'password':password},verify=verify)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost', port=8834): Max retries exceeded with url: /session (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f46f2d6d410>: Failed to establish a new connection: [Errno 111] Connection refused',))
The nessus api is depreciated as of version 7.x, this is the best source I could find.
EDIT: I have found a better source directly from tenable
What has been removed from Nessus 7:
There is a restriction in scan API capabilities.
The ability to manage scans via API and CLI has been removed in v7. All Nessus Pro scanning operations must be done through the user interface.
So currently the ability of the Nessus API is as follows:
Removed the ability to run scans or reports and create new objects
The Read features, where the ability to pull scan data so GET /scan/scan ID now works again and this aids with some of the integration processes.
https://community.tenable.com/s/article/The-differences-between-Nessus-6-and-Nessus-7
This is only for Nessus pro versions
I've been trying to set up my EndpointsV2 backend (billing has been enabled) with Firebase authentication but am always ending up with either an error when I attempt to generate the OpenAPI spec or a 401 Unauthorized when accessing the API.
My current configuration is as follows (errors out on OpenAPI generation):
main.py:
firebase_issuer = endpoints.Issuer(
issuer='https://securetoken.google.com/[GAE/FIREBASE_APP_ID]',
jwks_uri=('https://www.googleapis.com/service_accounts/v1/metadata/x509/'
'securetoken#system.gserviceaccount.com'))
issuers = {
"firebase": firebase_issuer
}
#endpoints.api("user", "v1", api_key_required=True,
issuers=issuers)
class UserApi(remote.Service):
#models.UserModel.query_method(
user_required=True,
path="info",
name="user.info",
http_method="GET")
def info(self, query):
return query.filter(
models.UserModel.owner == endpoints.get_current_user())
Error on OpenAPI Gen:
$ python lib/endpoints/endpointscfg.py get_openapi_spec main.UserApi --hostname [SERVICE-NAME].endpoints.[GAE_APP_ID].appspot.com
No handlers could be found for logger "endpoints.apiserving"
Traceback (most recent call last):
File "lib/endpoints/endpointscfg.py", line 633, in <module>
main(sys.argv)
File "lib/endpoints/endpointscfg.py", line 629, in main
args.callback(args)
File "lib/endpoints/endpointscfg.py", line 487, in _GenOpenApiSpecCallback
application_path=args.application)
File "lib/endpoints/endpointscfg.py", line 332, in _GenOpenApiSpec
application_path=application_path)
File "lib/endpoints/endpointscfg.py", line 214, in GenApiConfig
services, hostname=hostname))
File "D:\Users\Will\Documents\Projects\CardApp\backend\lib\endpoints\openapi_generator.py", line 973, in pretty_print_config_to_json
descriptor = self.get_openapi_dict(services, hostname)
File "D:\Users\Will\Documents\Projects\CardApp\backend\lib\endpoints\openapi_generator.py", line 959, in get_openapi_dict
return self.__api_openapi_descriptor(services, hostname=hostname)
File "D:\Users\Will\Documents\Projects\CardApp\backend\lib\endpoints\openapi_generator.py", line 872, in __api_openapi_descriptor
security_definitions)
File "D:\Users\Will\Documents\Projects\CardApp\backend\lib\endpoints\openapi_generator.py", line 691, in __method_descriptor
service.api_info.audiences, security_definitions)
File "D:\Users\Will\Documents\Projects\CardApp\backend\lib\endpoints\openapi_generator.py", line 722, in __x_security_descriptor
_INVALID_AUTH_ISSUER % default_auth_issuer)
api_exceptions.ApiConfigurationError: No auth issuer named google_id_token defined in this Endpoints API.
The error leads me to think that if I simply swap "firebase" for "google_id_token"in issuers that it may work but I have tried that and just gotten a 401.
I've also started to see this error in GAE Logging:
Cannot decode and verify the auth token. The backend will not be able to retrieve user info (/base/data/home/apps/s~[GAE/FIREBASE_PROJECT]/20170318t232908.399940418908006950/lib/google/api/control/wsgi.py:588)
Traceback (most recent call last):
File "/base/data/home/apps/s~[GAE/FIREBASE_PROJECT]/20170318t232908.399940418908006950/lib/google/api/control/wsgi.py", line 585, in __call__
service_name)
File "/base/data/home/apps/s~[GAE/FIREBASE_PROJECT]/20170318t232908.399940418908006950/lib/google/api/auth/tokens.py", line 83, in authenticate
"allow provider id: " + provider_id)
UnauthenticatedException: The requested method does not allow provider id: google_id_token
The idToken I am sending with my testing (done through Postman using the header Authorization: Bearer [idToken]) was generated using pyrebase and I checked jwt.io to confirm that it is in fact a valid token.
My initial thought is that I haven't set up the audiences correctly, but I'm not sure what it should be (though I have tried appending a ClientID from API Manager > Credentials).
Is it possible to authenticate with firebase using endpoints-proto-datastore on the endpointsv2 framework? If so, what do I need to do differently to get that working? If not, would the code that I have work right away without using endpoints-proto-datastore?
Edit:
Looks like this is an issue with the endpoints library itself
https://github.com/cloudendpoints/endpoints-python/issues/32
I think my university recently changed a firewall or filtering protocol with regards to Twitter's streaming API. However, for me to provide them more information, I'm going to have to figure out the details.
Ok, below is the simplest possible program to collect data from Twitter's stream:
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
ckey="[OMITED]"
csecret="[OMITED]"
atoken="[OMITED]"
asecret="[OMITED]"
class listener(StreamListener):
def on_data(self, data):
print(data)
return(True)
def on_error(self, status):
print status
auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken, asecret)
twitterStream = Stream(auth, listener())
twitterStream.filter(track=["car"])
When I run this program on my laptop/home desktop, it works fine. However, I have a university computer setup for data collection and it produces the following error:
Traceback (most recent call last):
File "test.py", line 25, in <module>
twitterStream.filter(track=["car"])
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 228, in filter
self._start(async)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 172, in _start
self._run()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 106, in _run
conn.connect()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1278, in connect
server_hostname=server_hostname)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 352, in wrap_socket
_context=self)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 579, in __init__
self.do_handshake()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 808, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
Now, the thing is, I can open up Safari and browse the web. Further, data collection like this use to work (as of a month ago). The error says the SSL cert is failing verification (but strangely the original OAuth seems to work -- it is the filter command kicking up the error). Is there a way to get it to spit out if that is because an intermediate cert has been put in place? Or maybe a port is being blocked? Basically, is there a way to get a lot more details so I can go the university and say: 'you are blocking X'?
Any suggestions?
I've resolved the situation. It was an error relating to SSL sites that were signed with Digicerts.