I'm attempting to connect BigQuery to Looker. I am pulling sample data from a Google Sheets document to a BigQuery dataset; this part is working fine, as my internal BigQuery queries are running just fine for this dataset. Using this documentation from the Looker forums, I tried to create a service account key to connect my BigQuery dataset to Looker. Unfortunately, the documentation is slightly out of date: Google now asks which service account (compute engine default service account, app engine default service account, or a new service account that can have any of multiple roles) you want to attach the key to.
Thus far, I have tried using P12 keys created for the compute engine default service account, the app engine default service account, as well as a new Project Owner service account. When I create the connection in Looker, the admin page confirms that the connection "can connect, can cancel queries, can run simple select query" (I need it to do more complex things, but am just trying to connect at all right now). Using the SQL Runner to test a simple select 10 query out, I was able to query the public datasets, e.g. hacker_news or usa_names. However, whenever I tried to run the same query on my personal sample dataset, I received this error:
Failed to retrieve data - The job encountered an internal error during execution and was unable to complete successfully.
The permissions for the base Google Sheet that the BigQuery project is pulling from are set to be viewable by my coworkers who have the link. I have also been adding each service account I test as an editor (which I assume has the highest permissions). At this point, I am creating new service accounts with each of the different possible roles to see if it's a permissions issue from the role perspective. Nothing has worked so far, so any insight would be helpful!
UPDATE: I have created a new table within the same BigQuery dataset. The new table was created using a CSV file, which was simply a download of my previous table in Google Sheets. I updated the connection to Looker. When I wrote a select 10 query pulling from the new table, it worked fine and ran very quickly. This seems to imply that the problem is something about the permissions between Google Sheets and Google BigQuery.
I've been wanting to do something like this myself for a bit, saw this question, and decided to dig in.
First thing I found was this "documentation" over in the looker discourse:
https://discourse.looker.com/t/live-spreadsheets-in-databases/2698/7
In there, it describes the steps necessary to get this working.
Two important things that you are probably missing, based on your description of events so far (since it sounds like you've already attached the sheet to your dataset and are able to query it from the BigQuery UI):
Make sure you share the Google Sheet with the service account you are using to connect Looker to BigQuery. This is the Username from the Connections tab of the Admin page in Looker.
Make sure you have enabled the Drive and Sheets APIs for your google project. You can do that via The API Library. Just search for "Drive" (or "Sheets"), click on the name, and then click on the "Enable" button from the API detail page.
Once I did the above, I had to wait a few minutes before things started working. I'll go out on a limb and guess that this was because Looker needed to cycle it's internal connection pool before the permissions would reset and work. So you may need to run a few failing queries, or wait out the connection pool before this will go into effect.
Hope that helps.
Related
I have a view that is connected to a google sheet via connected sheets.
I'm trying to let a user refresh the data by giving them access in GCP.
I've tried giving access at the project, dataset and view levels. But every time they get the error: "Query failed, no access to the connected BigQuery table"
I'm giving the role of bigquery.user and bigquery.dataviewer.
What could be causing this?
Please, make sure if the user you wish to give access to the BigQuery data in Google Sheets has:
An Enterprise Plus or G Suite Enterprise for Education account
Access to BigQuery
A project with billing setup in BigQuery
A BigQuery Job Creator role on the selected billing project
BigQuery Data Viewer role on the datasets containing the selected table
According to the documentation:
If you share a sheet with someone who doesn't meet the criteria
above, they'll be able to see analysis created with Connected Sheets
and perform regular Sheets operations, but they won't be able to
refresh it or create their own connected sheet.
Additionally, have a look for the another SO thread and Using Connected Sheets documentation.
I have a View (Table A) in Big Query which was created from a Google Sheet. It updates live which is perfect.
I have then connected that View to another View (Table B) in Bigquery. Let’s call this combined View, Table C.
In Tableau Desktop I try to connect to Table C, but it comes up with an authentication issue because Tableau cannot pass on authentication to Google Sheets.
Has anyone found a solution or workaround? Using service accounts, or even cloud functions or a scheduled query which saves the results of Table A as a table every time the google sheet is saved.
This has been asked before in the following link, but hasn’t received a step-by-step solution and I do not have enough stackoverflow reputation to comment:
BigQuery Credential Problems when Accessing Google Sheets Federated Table
https://community.tableau.com/thread/207871
I am trying the new 'Transfers' function in google BigQuery.
I am using the option: 'Scheduled Query'
It works with a simple query, but when I am trying another query that is normally working based on a view, that is based on a join between two tables (on table based on a google sheet shared with me) none of the more complicated Transfers I created are working.
I get the following error message:
Failed to start job for table 'xxx' with error PERMISSION_DENIED: Access Denied: BigQuery BigQuery: No OAuth token with Google Drive scope was found.
Is it because one of the source tables is based on a google sheet?
I tried to copy the source table to another table, but when I do this BigQuery automatically deletes this table.
Any ideas?
The problem is with the view which queries Google Drive data. In order to resolve your problem you need to request Google Drive scopes. Quoting directly from documentation:
Accessing data hosted within Google Drive requires an additional OAuth
scope, both when defining the federated source as well as during query
execution.
In the documentation page linked above you'll also find ways to do this via command line, api and web UI.
I need to grant access to Big Query to 30 Phd Students in a University on Google Cloud Platform.
Can I give them standalone access to each of them? i.e. One student cannot see other's work unless it is granted.
Creating projects = number of students is not too cost effective.
so can I give 30 access controls to a single project ?
The students need to have full access to Big query (Create, edit, join, download, run) to their respective databases.
The document is indeed confusing. Don't grant any project-level permissions. Just as Katie Sinatra said, go to the dataset Web UI, in the arrow drop's "Share dataset", add email and grant "Can edit". At the time of this writing, after you do the above, the user won't be able to see the dataset in the Web UI, but s/he can still do query in the Web UI just fine if s/he specifies the table correctly, i.e. `project.dataset.table`. (i tested it.) The user can also manually add the dataset to be displayed in the web UI. Here is how to do it, https://cloud.google.com/bigquery/bigquery-web-ui#displayprojects.
What i am still confusing is: after you do the above, when the user do a query, who is paying. My guess is the user is paying. If you want the dataset's original owner (the project under which the dataset is created) to be billed, then my guess is that you need to grant project-level BigQuery Job User permission to the user/email in addition to the above. Then, the user will be able to select the project in the GCP console and hit "BigQuery" to go to the big query Web UI, and be billed under the project. (by the way, if you do this, the dataset can be seen by the user in the Web UI.)
As JL-HN said, it is documented but it is a bit confusing. To give access to an specific dataset, you only need to go the dataset, in the arrow drop it down and click on "Share dataset". Then you only need to add the email of the student that will handle that dataset.
our organization has been using SAS BI Dashboard for several months now for internal use within our own organization. Now, we are working on a project where roughly 100 people in other, outside organizations will need to log on to our BI Dashboard site to view an individualized dashboard for their organization. We plan to use row-level permissions in an Information Map to control who is allowed to see what in terms of the data behind the dashboard indicators.
How would you recommend creating roughly 100 individual log-ons for outside users?
Is there a way to automate the process rather than manually creating all the accounts?
If I create the log-on name and password for each outside user, how/where would I store that in Management Console?
Any help would be appreciated - our office is small enough that we do not have a dedicated IT person or fully-trained SAS administrator, so I'm in over my head. Thanks!
As an ex SAS consultant, I can tell you briefly how I have solved this problem.
First, creating the users in batch should be easy. There are tons of scripts out there that will teach you this. I would recommend to create them in your LDAP server (probably Active Directory), to have them in a central place. That way, you can treat them the same way as you do the internal users.
To get them into the metadataserver, you should take a look at the macros that SAS provides for this:
The following macros are the core components used to import and synchronize user accounts from Active Directory to SAS metadata: %MDUIMPC , %MDUIMPLB , %MDUEXTR , %MDUCMP , %MDUCHGV , %MDUCHGLB. They are located in the following directory: [SAS Home]\SASFoundation\9.3\core\sasmacro.
This SGF proceeding will give you a practical description of the process:
http://support.sas.com/resources/papers/proceedings12/377-2012.pdf
As for the question you did not ask, "how to present the BI Dashboard webapplication to the external users". You need to set up a reverse proxy web server in a secure zone (DMZ). See this document for details: http://support.sas.com/resources/thirdpartysupport/v92m3/appservers/ApacheProxyJBoss.pdf
Hope this helps!
Stig