I am trying to do http request to get information for a job that was submitted by another script. This script has the job id and the project. I read on Oauth20 and saw that using Application Default Credentials is the recommended way.
I have exported the default auth:
export GOOGLE_APPLICATION_CREDENTIALS= /test/proj-service-key-file.json
Here is my code:
from google.oauth2.service_account import Credentials
from google.cloud import bigquery
import requests
from oauth2client.client import GoogleCredentials
from google.auth.transport.requests import AuthorizedSession
import google.auth
credentials = GoogleCredentials.get_application_default()
session = google.auth.transport.requests.AuthorizedSession(credentials)
job_url="https://www.googleapis.com/bigquery/v2/projects/" + self.project + "/jobs/" + job_id + "?maxResults=1"
job_query_results = session.request('GET',job_url)
I am getting the following error:
self.credentials.before_request(
AttributeError: '_JWTAccessCredentials' object has no attribute 'before_request'
Any suggestions is appreciated.
Try deleting the space in your export statement:
export GOOGLE_APPLICATION_CREDENTIALS=/test/proj-service-key-file.json
I got it to work. I had to set up the scopes explicity and that did the trick:
scopes = ['https://www.googleapis.com/auth/cloud-platform',
'https://www.googleapis.com/auth/devstorage.full_control',
'https://www.googleapis.com/auth/bigquery']
credentials = credentials.create_scoped(scopes)
http = httplib2.Http()
if credentials.access_token_expired:
credentials.refresh(http)
http = credentials.authorize(http)
response, content = http.request(job_url)
Related
Trying an example to hit a rest API , but seeing a 400 status code. Is this the correct way to call an API using form params?
import javax.json.JsonObject;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.core.Form;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.client.Invocation;
import javax.ws.rs.HttpMethod;
import javax.ws.rs.core.Response;
...
public JsonObject getUserProfile() throws Exception {
Form userform = new Form();
userform.param("grant_type", "password")
.param("client_id", "cexxx")
.param("username", "theuser")
.param("password", "password")
.param("scope", "user.read openid profile offline_access");
Client client = ClientBuilder.newClient();
String serverUrl = "https://login.microsoftonline.com/547xx/oauth2/v2.0/token";
WebTarget target = client.target(serverUrl);
final Invocation.Builder invocationBuilder = target.request();
invocationBuilder.header("Content-Type", "application/x-www-form-urlencoded");
final Response response = invocationBuilder.method(
HttpMethod.POST,
Entity.entity(userform, MediaType.APPLICATION_FORM_URLENCODED),
Response.class);
System.out.println("r.getStatus()=" + response.getStatus());
...
}
The same works on Postman:
Thanks Giorgi for the hint.
Actually, the code we have above to programmatically make an API call works. The issue is that we are seeing error from server side.
Using this, we are able to see the error message from server:
System.out.println(response.readEntity(String.class));
{"error":"invalid_grant","error_description":"AADSTS50034: The user account sx does not exist in the 547..b32 directory. To sign into this application, the account must be added to the directory.\r\nTrace ID: 4616...3c00\r\nCorrelation ID: 617...a01\r\nTimestamp: 2020-09-29 22:25:41Z","error_codes":[50034],"timestamp":"2020-09-29 22:25:41Z","trace_id":"461...c00","correlation_id":"617..a01","error_uri":"https://login.microsoftonline.com/error?code=50034"}
I have a Vue frontend that uses Axios to POST to my Flask API. I've registered flask_cors to my Flask instance but I'm still get a CORS error.
flask_cors registered in app/__init__.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from config import Config
from flask_marshmallow import Marshmallow
from flask_cors import CORS
app = Flask(__name__)
app.config.from_object(Config)
cors = CORS(app)
db = SQLAlchemy(app)
migrate = Migrate(app, db)
ma = Marshmallow(app)
from . import routes, models
app/routes.py
from flask import request, jsonify, current_app
from . import app, db
from .models import SetAnon
#app.route('/sets', methods=['POST'])
def sets():
data = request.get_json()
_set = SetAnon(
col1=data['somedata']
)
db.session.add(_set)
db.session.commit()
return "set saved", 201
Vue frontend is making POST request with axios:
import axios from 'axios'
const API_URL = 'http://127.0.0.1:5000'
export function saveSet(set) {
return axios.post(`${API_URL}/sets/`, set)
}
Getting this error in browser console
xhr.js?ec6c:172 OPTIONS http://127.0.0.1:5000/sets/ 404 (NOT FOUND)
Access to XMLHttpRequest at 'http://127.0.0.1:5000/sets/' from origin 'http://localhost:8080' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.
Looks like an issue with trailing backslash. Your routes.py defines the route as '/sets' however the frontend is calling '/sets/'.
Change the routes.py to this and it should work -
from flask import request, jsonify, current_app
from . import app, db
from .models import SetAnon
#app.route('/sets/', methods=['POST']) # Added trailing backslash
def sets():
data = request.get_json()
_set = SetAnon(
col1=data['somedata']
)
db.session.add(_set)
db.session.commit()
return "set saved", 201
``I am trying to implement token authentication in my Django Rest Framework application. I am able to receive a token from the django server but when I send a request with 'Authorization' header, I receive the following error:
{
"detail": "Authentication credentials were not provided."
}
I have also checked the request.META and there is no key with name "HTTP_AUTHORIZATION". I am running my django app on development server.
Views.py
# Create your views here.
from rest_framework.authentication import SessionAuthentication, BasicAuthentication
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
from rest_framework.views import APIView
from rest_framework.decorators import api_view,authentication_classes,permission_classes
from django.contrib.auth.decorators import permission_required,login_required
import google.cloud.storage
import uuid
import psycopg2
#api_view(['GET'])
#authentication_classes((SessionAuthentication, BasicAuthentication))
#permission_classes((IsAuthenticated,))
def auth_test(request):
return Response("it's great to have you here with token auth!!")
#api_view(['GET'])
def free_test(request):
#print(request.META['HTTP_AUTHORIZATION'])
return Response("entry without token auth !!")
What is the reason that my authorization credentials are not received by django server?
POSTMAN SCREENSHOT WHERE I HAVE PASSED THE AUTHORIZATION HEADER
You are trying to access your view using Token auth method, and you are not mentioned that in authentication_classes. Try the following code
from rest_framework.authentication import TokenAuthentication
#api_view(['GET'])
#authentication_classes((SessionAuthentication, TokenAuthentication, BasicAuthentication))
#permission_classes((IsAuthenticated,))
def auth_test(request):
return Response("it's great to have you here with token auth!!")
If someone could point me in the right direction I would be very grateful. I am trying to use the requests module to interact with Reddit's APIs, but for some reason I am getting HTTP status 403 codes. I am able to successfully log in I believe (since I get HTTP 200), but I cannot perform a successful API request. I am very new to this so please use small words. Thanks.
import requests
import json
from pprint import pprint
# URLs
url_base = r'https://www.reddit.com'
url_login = '/api/login'
url_me = '/api/v1/me'
# Credentials
user = '__kitten_mittens__'
passwd = 'password'
params = {'user': user,
'passwd': passwd,
'api_type': 'json',}
headers = {'user-agent': '/u/__kitten_mittens__ practice api bot'}
# Set up the session/headers
client = requests.session()
client.headers = headers
response = client.post(url_base + url_login, data = params)
j = json.loads(response.content.decode('utf-8'))
mh = j['json']['data']['modhash']
print("The modhash for {USER} is {mh}".format(USER=user, mh=mh))
# THIS CODE GIVES HTTP STATUS CODE 403
response = client.get(url_base + url_me)
me_json = json.loads(r.content.decode('utf-8'))
pprint(me_json)
EDIT: I fixed the missing single quote from the password field. That was not part of the problem.
For the "normal" oauth2 dance, I get to specify the user and get a corresponding token.
This allows me to make API calls masquerading as that user, i.e. on his behalf.
It can also allow the user to make calls masquerading as me.
A use case is bigquery where I don't have to grant table access to the user and I can specify my own preferred level of control.
Using the simplified OAuth2Decorator, I don't seem to have this option.
Am I right to say that?
Or is there a work-around?
In general, what is the best practice? To use the proper oauth (comprising of Flow, Credentials and Storage)? Or to use OAuth2Decorator.
Thank you very much.
You can certainly use an OAuth2Decorator
Here is an example:
main.py
import bqclient
import httplib2
import os
from django.utils import simplejson as json
from google.appengine.api import memcache
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from oauth2client.appengine import oauth2decorator_from_clientsecrets
PROJECT_ID = "xxxxxxxxxxx"
DATASET = "your_dataset"
QUERY = "select columns from dataset.table"
CLIENT_SECRETS = os.path.join(os.path.dirname(__file__),'client_secrets.json')
http = httplib2.Http(memcache)
decorator = oauth2decorator_from_clientsecrets(CLIENT_SECRETS,
'https://www.googleapis.com/auth/bigquery')
bq = bqclient.BigQueryClient(http, decorator)
class MainHandler(webapp.RequestHandler):
#decorator.oauth_required
def get(self):
data = {'data': json.dumps(bq.Query(QUERY, PROJECT_ID))}
template = os.path.join(os.path.dirname(__file__), 'index.html')
self.response.out.write(render(template, data))
application = webapp.WSGIApplication([('/', MainHandler),], debug=True)
def main():
run_wsgi_app(application)
if __name__ == '__main__':
main()
bqclient.py that gets imported in your main.py which handles BigQuery actions
from apiclient.discovery import build
class BigQueryClient(object):
def __init__(self, http, decorator):
"""Creates the BigQuery client connection"""
self.service = build('bigquery', 'v2', http=http)
self.decorator = decorator
def Query(self, query, project, timeout_ms=10):
query_config = {
'query': query,
'timeoutMs': timeout_ms
}
decorated = self.decorator.http()
queryReply = (self.service.jobs()
.query(projectId=project, body=query_config)
.execute(decorated))
jobReference=queryReply['jobReference']
while(not queryReply['jobComplete']):
queryReply = self.service.jobs().getQueryResults(
projectId=jobReference['projectId'],
jobId=jobReference['jobId'],
timeoutMs=timeout_ms).execute(decorated)
return queryReply
where all your authentication details are kept in a json file client_secrets.json
{
"web": {
"client_id": "xxxxxxxxxxxxxxx",
"client_secret": "xxxxxxxxxxxxxxx",
"redirect_uris": ["http://localhost:8080/oauth2callback"],
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token"
}
}
finally, don't forget to add these lines to your app.yaml:
- url: /oauth2callback
script: oauth2client/appengine.py
Hope that helps.
I am not sure I completely understand the use case, but if you are creating an application for others to use without their having to authorize access based on their own credentials, I would recommend using App Engine service accounts.
An example of this type of auth flow is described in the App Engine service accounts + Prediction API article.
Also, see this part and this part of the App Engine Datastore to BigQuery codelab, which also uses this authorization method.
The code might look something like this:
import httplib2
# Available in the google-api-python-client lib
from apiclient.discovery import build
from oauth2client.appengine import AppAssertionCredentials
# BigQuery Scope
SCOPE = 'https://www.googleapis.com/auth/bigquery'
# Instantiate and authorize a BigQuery API client
credentials = AppAssertionCredentials(scope=SCOPE)
http = credentials.authorize(httplib2.Http())
bigquery_service = build("bigquery", "v2", http=http)
# Make some calls to the API
jobs = bigquery_service.jobs()
result = jobs.insert(projectId='some_project_id',body='etc, etc')