I have created an instance on Google BigTable and created an external table on Google BigQuery and created an external table which can able to fetch records from BigTable. I tried to create a BigQuery datasource in Power BI Desktop but the external tables are not listing but the tables created on BigQuery are been listed! Is it an issue? or any possible solutions?
I replicated your scenario and got the same behavior as the one you described. It seems that at the moment Power BI does not support BigTable external tables. A workaround is to create one or more views based on the BigTable table and then connect to those views.
In addition, if you're interested you can report the Power BI behavior here.
Related
How do I connect various different client's google analytics (GA4 & UA) to one instance of Big Query? I want to store the analytics reports on bigquery & then visualise it on a unified dashboard on Looker
You can set up the exports from Google Analytics to go to the same BigQuery project and transfer historical data to the same project as well.
Even if data is spread across multiple GCP projects, you can still query all from a single project. I would suggest you create a query that connects data from multiple sources together, you can then save it as a view and add it as a source in Looker, you can use it as a custom query in Looker or for best efficiency save the results of your query as a new reporting table that feeds into Looker.
Have created a table in BigQuery and connected to BigQuery in PowerBI Desktop. I authenticate successfully and can see my project and dataset structure. However, the table that should be visible under cpi_quotes is not there. Tables relating to public datasets are so this is specific to my data which I can query within BigQuery itself just fine. Is there a permission I need to enable within BigQuery to make this table visible to PowerBI?
I created a reservation for BI Engine to make the reports on Google datastudio load faster.
I create a query and save it as view. I looked at the job information and it says "unable to support input table due to an internal error". What does it mean?
One of your tables is defined as external table (references a spreadsheet). Unfortunately BI Engine only supports tables that are stored in BigQuery natively today. We will work on improving the error message in this case. Supporting external tables was not on our roadmap so far, unfortunately.
To get full BI Engine acceleration, please load the data from spreadsheet into BigQuery.
See BI Engine overview here: https://cloud.google.com/bi-engine/docs/introduction
I need to understand the below:
1.) How does one BigQuery connect to another BigQuery and apply some logic and create another BigQuery. For e.g if i have a ETL tool like Data Stage and we have some data been uploaded for us to consume in form of a BigQuery. So in DataStage or using any other technology how do i design the job so that the source is one BQ and the Target is another BQ.
2.) I want to achieve like my input will be a VIEW (BigQuery) and then need to run some logic on the BigQuery View and then load into another BigQuery view.
3.) What is the technology used to connected one BigQuery to another BigQuery is it https or any other technology.
Thanks
If you have a large amount of data to process (many GB), you should do the transformation of the data directly in the Big Query database. It would be very slow to extract all the data, run it through something locally, and send it back. You don't need any outside technology to make one view depend on another view, besides access to the relevant data.
The ideal job design will be an SQL query that Big Query can process. If you are trying to link tables/views across different projects then the source BQ table must be listed in fully-specified form projectName.datasetName.tableName in the FROM clauses of the SQL query. Project names are globally unique in Google Cloud.
Permissions to access the data must be set up correctly. BQ provides fine-grained control over who can access, and it is in the BQ documentation. You can also enable public access to all BQ users if that is appropriate.
Once you have that SQL query, you can create a new view by sending your SQL to Google BigQuery either through the command line (the bq tool), the web console, or an API.
1) You can use BigQuery Connector in DataStage to read and write to bigquery.
2) Bigquery use namespaces in the format project.dataset.table to access tables across projects. This allows you to manipulate your data in GCP as it were in the same database.
To manipulate your data you can use DML or standard SQL.
To execute your queries you can use the GCP Web console or client libraries such as python or java.
3) BigQuery is a RESTful web service and use HTTPS
I'm having datasets in bigquery google cloud project,
I want to load this data from powerbi,So that I can create visualizations using this bigquery data sets.
Currently I'm able to see the google bigquery in getdata option in powerbi desktop,I tried to connect to bigquery but I'm not abe to see my bigquery dataset in the list,I'm using the same credentials for powerbi and google bigquery.
is there any thig that I need to enable in google cloud side so that it will be visible in the list when I connect from powerbi?
These are the following steps you can follow to connect GBQ to Power BI.
You can write a query and save the table in Google big Query.
Go in Power BI, get data and click on more and scroll down there you will find "Google Big Query" click on that.
Search your saved table and click on "Direct Query"
I think you will be good to go!!
I would not recommend using Power BI for Big Query analysis. As of Q1 2019 Power BI still do not support project billing unless manually editing the connexion script, it does not support Sheets federated tables, do not support nested and repeated fields and Power BI DML is nearly unusable with BQ. Microsoft do not seems to be willing to develop the compatibility between Power BI and Big Query further.
Learn more in this article (i am the author): https://medium.com/#remy_david/which-bi-tool-for-big-query-d9eb838ff7ad