BigQuery Authentication Problems - google-bigquery

I'm starter with BigQuery.
And my Problem comes with Authentication.
The credential File has been created and store at:
/home/ltloi/.config/gcloud/application_default_credentials.json
Content:
{
"type": "service_account",
"project_id": "xxx",
"private_key_id": "xxx",
"private_key": "-----BEGIN PRIVATE KEY-----xxx\n-----END PRIVATE KEY-----\n",
"client_email": "xxx",
"client_id": "xxx",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/bigqueryservices1%40neon-deployment-141106.iam.gserviceaccount.com"
}
Sourcecode:
// just authentication with google big query
public static Bigquery createAuthorizedClient() throws IOException {
// Create the credential
HttpTransport transport = new NetHttpTransport();
JsonFactory jsonFactory = new JacksonFactory();
GoogleCredential credential = GoogleCredential.getApplicationDefault(transport, jsonFactory);
// Depending on the environment that provides the default credentials (e.g. Compute Engine, App
// Engine), the credentials may require us to specify the scopes we need explicitly.
// Check for this case, and inject the Bigquery scope if required.
if (credential.createScopedRequired()) {
credential = credential.createScoped(BigqueryScopes.all());
}
return new Bigquery.Builder(transport, jsonFactory, credential)
.setApplicationName("Bigquery Samples")
.build();
}
The Problems Issue:
Exception in thread "main" java.io.IOException: Error reading credential file from location
/home/ltloi/.config/gcloud/application_default_credentials.json
Error:
java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getCredentialUsingWellKnownFile(DefaultCredentialProvider.java:251)
at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getDefaultCredentialUnsynchronized(DefaultCredentialProvider.java:117)
at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getDefaultCredential(DefaultCredentialProvider.java:91)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:213)
at com.google.cloud.bigquery.samples.GettingStarted.createAuthorizedClient(GettingStarted.java:53)
at com.google.cloud.bigquery.samples.GettingStarted.main(GettingStarted.java:135)
When I try debug and the code shows that comes with SSL Issue form HttpClient.
Experts please help resolve it. I have try to add SSL of
https://accounts.google.com/o/oauth2/auth
https://www.googleapis.com/oauth2/v1/certs
But Can NOT resolve.
I also install Cloud SDK and But the also CAN NOT resolve.
{
"client_id": "XXXX",
"client_secret": "XXX",
"refresh_token": "XXXX",
"type": "authorized_user"
}
Note: I have Done gcloud beta auth application-default login gcloud beta auth application-default print-access-token
Thanks & Regards,

Were the application default credentials created using the following command?
gcloud beta auth application-default login
I think there are two additional quick tests that you could do to get closer to the problem source. First, you can try this to verify that your credentials are fine:
gcloud beta auth application-default print-access-token
Then, you could copy and paste this brief python script to verify that it works (from here: https://cloud.google.com/bigquery/authentication)
# Grab the application's default credentials from the environment.
credentials = GoogleCredentials.get_application_default()
# Construct the service object for interacting with the BigQuery API.
bigquery_service = build('bigquery', 'v2', credentials=credentials)

Related

.net core web API not getting authenticated even after generating bearer token from Azure AD using AzureADDefaults.BearerAuthenticationScheme

I am developing a .net core Web API and I am trying to authenticate it using AZURE AD authentication.
I am following below configurations.:
1.In Startup.cs I have added authentication scheme as :AzureADDefaults.BearerAuthenticationScheme
services.AddAuthentication(AzureADDefaults.BearerAuthenticationScheme)
.AddAzureADBearer(options => { Configuration.Bind("AzureAd", options); });
2.In configure method of startup.cs I have added:
app.UseAuthentication();
3.In app.settings.json I have added following properties:
"AzureAd": {
"Instance": "https://login.microsoftonline.com/",
"ClientId": "<MY client ID>",
"TenantId": "<My Tenant ID>",
"Issuer": "https://login.microsoftonline.com/<My Tenant ID>/v2.0",
"Domain": "<My Domain>",
"ConfigView": "MVC",
"CallbackPath": "/signin-oidc",
"ClientSecret": "<My Client Secret>"
}
I have added Authorize attribute on top of my controller
I have generated my Bearer token using following code:
static void Main(string[] args)
{
Program obj = new Program();
IRestResponse ARMtokenResponse = obj.GetARMAuthToken();
dynamic response = JsonConvert.DeserializeObject(ARMtokenResponse.Content);
Console.WriteLine(response["access_token"].ToString());
Console.ReadKey();
}
private IRestResponse GetARMAuthToken()
{
var client = new RestClient("https://login.microsoftonline.com/<MY TENANT ID>/oauth2/token"); //tenantid
client.Timeout = -1;
var request = new RestRequest(Method.POST);
request.AddHeader("Content-Type", "application/x-www-form-urlencoded");
request.AddParameter("grant_type", "client_credentials");
request.AddParameter("client_id", "<My Client ID>");
request.AddParameter("client_secret", "<MY CLIENT SECRET>");
request.AddParameter("resource", "https://management.azure.com/");
IRestResponse response = client.Execute(request);
return response;
}
Further I am using this token generated, in postman/console app for calling the API but getting error in response header: Bearer error="invalid_token", error_description="The audience is invalid"
Please help me in this. I am stuck here
There are several issues in your codes :
You should acquire access token for web api , not acquiring access token for Azure Rest API (https://management.azure.com/) , your web api can't validate Azure Rest API's access token .
When acquiring token you are using Azure AD V1.0 endpoint , but when validating token you are using Azure AD V2.0 endpoint (Issuer) .
For Azure AD V1.0 , you can refer to code sample : Call a web API in an ASP.NET Core web app using Azure AD.
For Azure AD V2.0 , you can refer to code samples : Enable your Web Apps to sign-in users and call APIs with the Microsoft identity platform for developers , and follow the 4-WebApp-your-API scenario .

Kubernetes auth does not work when using non default auth path

I am using spring vault to access Vault from Spring boot app running in Kubernetes.
Version
<dependency>
<groupId>org.springframework.vault</groupId>
<artifactId>spring-vault-core</artifactId>
<version>2.1.3.RELEASE</version>
</dependency>
Config
vault:
uri: https://xxx.xxx.com:8200
authentication: KUBERNETES
kubernetes:
role: abc
kubernetes-path: path/to/k8s
service_account_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
Error
o.s.v.a.VaultLoginException: Cannot login using Kubernetes: invalid role name \"abc\";
When I try to login using curl with the same role and token, its success:
VAULT_LOGIN="{\"role\":\"$SA_ROLE\", \"jwt\":\"$SA_JWT_TOKEN\"}"
curl --request POST --data "$VAULT_LOGIN" https://xxx.xxx.com:8200/v1/auth/path/to/k8s/login
It is a bug in spring vault. It does not support custom auth backend path. Pleae find the issue here: https://github.com/spring-projects/spring-vault/issues/462
As a workaround we can fix this by overriding kubeAuthentication method.
#Override
protected ClientAuthentication kubeAuthentication() {
String role = getEnvironment().getProperty("vault.kubernetes.role");
Assert.hasText(role, "Vault Kubernetes authentication: role must not be empty");
String tokenFile = getEnvironment().getProperty("vault.kubernetes.service-account-token-file");
if (!StringUtils.hasText(tokenFile)) {
tokenFile = KubernetesServiceAccountTokenFile.DEFAULT_KUBERNETES_SERVICE_ACCOUNT_TOKEN_FILE;
}
KubernetesJwtSupplier jwtSupplier = new KubernetesServiceAccountTokenFile(
tokenFile);
String path = getEnvironment().getProperty("vault.kubernetes.kubernetes-path");
KubernetesAuthenticationOptions authenticationOptions = KubernetesAuthenticationOptions
.builder() //
.role(role) //
.path(path)
.jwtSupplier(jwtSupplier) //
.build();
return new KubernetesAuthentication(authenticationOptions, restOperations());
}

Non-interactive authorization with Google OAuth2

I have created a Java application that performs syncing of MS Active Directory with Google Groups. It is non-interactive application that suppose to be executed by the cronjob at night time.
It does work perfectly on my laptop (DEV environment).
My problem is that on the first run it pops up browser window with the dialog asking to authorize access to the Google API. After I click on "Allow" button it happily proceeds to the end.
Now I need to move it to production server which runs CentOS and does not have a browser.
When I run my application in this environment it prints the following message:
Please open the following address in your browser:
https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=520105804541-c7d7bfki88qr8dbkv6oahp22i3oq1jq0.apps.googleusercontent.com&redirect_uri=http://localhost:50089/Callback&response_type=code&scope=https://www.googleapis.com/auth/admin.directory.group.member%20https://www.googleapis.com/auth/admin.directory.orgunit%20https://www.googleapis.com/auth/admin.directory.device.mobile.action%20https://www.googleapis.com/auth/admin.directory.orgunit.readonly%20https://www.googleapis.com/auth/admin.directory.userschema%20https://www.googleapis.com/auth/admin.directory.device.chromeos%20https://www.googleapis.com/auth/admin.directory.notifications%20https://www.googleapis.com/auth/admin.directory.device.chromeos.readonly%20https://www.googleapis.com/auth/admin.directory.device.mobile%20https://www.googleapis.com/auth/admin.directory.group.readonly%20https://www.googleapis.com/auth/admin.directory.group.member.readonly%20https://www.googleapis.com/auth/admin.directory.userschema.readonly%20https://www.googleapis.com/auth/admin.directory.user.alias%20https://www.googleapis.com/auth/admin.directory.user.security%20https://www.googleapis.com/auth/admin.directory.device.mobile.readonly%20https://www.googleapis.com/auth/admin.directory.user%20https://www.googleapis.com/auth/admin.directory.user.readonly%20https://www.googleapis.com/auth/admin.directory.user.alias.readonly%20https://www.googleapis.com/auth/admin.directory.group%20https://www.googleapis.com/auth/apps.groups.settings
There is no browser on this machine but if I try to run it on another box I am getting error 400 - invalid request.
The reason is clear, because it redirecting to localhost:50089 and nobody is listening on that port on another machine.
Going through the multiple documents and samples on developers.google.com I found a way to bypass the dialog by creating the service account and generating a primary key for it.
try {
GoogleCredential cr = GoogleCredential
.fromStream(new FileInputStream(keyFile))
.createScoped(SCOPES);
Directory directory = new Directory.Builder(httpTransport, jsonFactory, cr)
.setApplicationName(config.getProperty("google.application.name")).build();
} catch (IOException ex) {
LOG.error("Failed to establish access credentials : ", ex.getMessage());
}
It does read the key and does create Directory instance, but any request to it results in 403 response code
{ "code" : 403, "errors" : [ { "domain" : "global", "message" :
"Not Authorized to access this resource/api", "reason" : "forbidden"
} ], "message" : "Not Authorized to access this resource/api" }"
But I have delegated all the privileges for this account to the service account. Don't know what else I can do.
If anyone can help me out of this jam, I appreciate it greatly.
I found solution. The problem is that Google generated JSON key does not set service account user. You have to do it manually like this:
GoogleCredential cr = GoogleCredential
.fromStream(new FileInputStream(keyFile))
.createScoped(SCOPES);
GoogleCredential.Builder builder = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountScopes(SCOPES)
.setServiceAccountId(cr.getServiceAccountId())
.setServiceAccountPrivateKey(cr.getServiceAccountPrivateKey())
.setServiceAccountPrivateKeyId(cr.getServiceAccountPrivateKeyId())
.setTokenServerEncodedUrl(cr.getTokenServerEncodedUrl())
.setServiceAccountUser(user);
Directory directory = new Directory.Builder(
httpTransport, jsonFactory, builder.build())
.setApplicationName(config.getProperty("google.application.name")).build();

Getting Insufficient scope for this resource using Spring Boot and OAuth2

I'm trying to run a small proof of concept for a REST API using Spring Boot 1.2.2 and secured with OAuth2. In order to do that I created two projects, one for the authentication server and another one for the REST service.
When running the projects, everything goes right. I get a token from the authentication server and then using that token, I managed to call the service and get the expected result.
The issue happens when I try to validate the scopes in the service. As soon as I add the line:
#PreAuthorize("#oauth2.hasScope('webshop')")
to my service, in the code:
#RestController
public class ProductService {
#RequestMapping("/product/{productId}")
#PreAuthorize("#oauth2.hasScope('webshop')")
public Product getProduct(#PathVariable int productId) {
return new Product(productId, "name", 123);
}
}
I get this response:
{
"error": "insufficient_scope",
"error_description": "Insufficient scope for this resource",
"scope": "webshop"
}
The authentication server is configured as:
#Configuration
#EnableAuthorizationServer
protected static class OAuth2Config extends AuthorizationServerConfigurerAdapter {
#Autowired
private AuthenticationManager authenticationManager;
#Override
public void configure(AuthorizationServerEndpointsConfigurer endpoints) throws Exception {
endpoints.authenticationManager(authenticationManager);
}
#Override
public void configure(ClientDetailsServiceConfigurer clients) throws Exception {
clients.inMemory()
.withClient("acme")
.secret("acmesecret")
.authorizedGrantTypes("authorization_code", "refresh_token", "implicit", "password", "client_credentials")
.scopes("webshop");
}
}
This is the actual call to get the token:
juan#ubuntu:~/ms/core/product-service$ curl -s acme:acmesecret#localhost:9999/uaa/oauth/token -d grant_type=client_credentials -d scope=webshop
{
"access_token": "ac3a9768-4ebb-44e3-a49f-21e2f117d0b4",
"token_type": "bearer",
"expires_in": 43199,
"scope": "webshop"
}
And then, to call the service:
juan#ubuntu:~/ms/core/product-service$ TOKEN=ac3a9768-4ebb-44e3-a49f-21e2f117d0b4
juan#ubuntu:~/ms/core/product-service$ curl 'http://localhost:8080/product/32' -H "Authorization: Bearer $TOKEN" -s
{
"error": "insufficient_scope",
"error_description": "Insufficient scope for this resource",
"scope": "webshop"
}
I feel that I am just missing something simple because of lack of a concept. If the client has been created in memory and is supposed to be allowed to do "webshop" and the token has been generated successfully, why is the error generated?
Thanks in advance for pointing to some direction or help.
This is caused by the UserInfoTokenServices class failing to extract the scopes from the request and therefore not adding them to a new OAuth2Request.
Specifically this method
private OAuth2Authentication extractAuthentication(Map<String, Object> map) {
Object principal = getPrincipal(map);
List<GrantedAuthority> authorities = this.authoritiesExtractor
.extractAuthorities(map);
OAuth2Request request = new OAuth2Request(null, this.clientId, null, true, null,
null, null, null, null);
UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(
principal, "N/A", authorities);
token.setDetails(map);
return new OAuth2Authentication(request, token);
}
neglects to extract the scopes map from the oauth2Request map entry, and then passes a null for the scopes collection into the constructor for the Oauth2Request scope parameter.
This means that even though you might have authorised your user to a particular scope, when the authserver is queried via the security.oauth2.resource.userInfoUri endpoint,the authorised scopes are ignored in the calling code, so the PreAuthorize check fails.
This looks like a bug to me; I've raised an issue on github.

Google OAuth2 Service Accounts API Authorization

I'm trying to authenticate my server app through Google's service account authentication but, for some reason, it is just not pushing through.
In the API console, I already created the project, enabled the service I need (Admin SDK), and created a Service Account and Web Application API Access.
When I do use the web application access credentials I am able to authenticate and retrieve user records. But using service account authentication would keep giving me a login required message.
"error": { "errors": [ { "domain": "global", "reason": "required", "message": "Login Required", "locationType": "header", "location": "Authorization" } ], "code": 401, "message": "Login Required" }
I forgot to add, I am testing this with the PHP client library.
public function init() {
$client = new Google_Client();
if (isset($_SESSION['access_token'])) {
$client->setAccessToken($_SESSION['access_token']);
}
$key = file_get_contents(App::SERVICE_KEY_FILE);
$client->setAssertionCredentials(new Google_AssertionCredentials(
App::SERVICE_ACCOUNT_NAME,
App::SERVICE_API_SCOPES,
$key)
);
$client->setClientId(App::SERVICE_CLIENT_ID);
debug($client, 'CLIENT');
if ($client->getAccessToken()) {
$this->access_token = $_SESSION['access_token'] = $client->getAccessToken();
debug($_SESSION['access_token'], 'TOKEN');
} else {
debug('NO TOKEN');
}
$this->client = $client;
}
As you can see, the code is basically about the same as the Google example. Am I missing an extra step?
One last thing, when I authenticate using the web app then access my service account script, the service account script can pick up the web app script's session and push through with the user record retrievals. Does that mean the Admin SDK API explicitly needs user interaction through web app authentication?
Instead of service account, I instead opted to use installed applications API Access.
This ruby gem actually helped my figure this out - https://github.com/evendis/gmail_cli
I was playing with it on the console and just followed the authorization steps in the readme, and found that installed applications is more simple when doing server admin apps.
Being a newb, I think I missed the important part the refresh token plays in the entire process. Going via the installed application approach helped me figure that out.
My config file now contains the client id, client secret, api scope, redirect uri, authorization code, and the refresh token; my initialization code now looks like:
public function init() {
$client = new Google_Client();
$client->setClientId(App::CLIENT_ID);
$client->setClientSecret(App::CLIENT_SECRET);
$client->setScopes(App::API_SCOPES);
$client->setRedirectUri(App::REDIRECT_URI);
if (!$client->getAccessToken()) {
$client->refreshToken(App::REFRESH_TOKEN);
}
$this->access_token = $client->getAccessToken();
$this->client = $client;
}