Im working with Google Analytics 360 with a big client and I want to download all the data to BigQuery.
The objective is to have the data available to create some AI algorithms and make a deep analyse.
I want to connect to bigquery from a desktop SQL client like execute query, is this possible?
Thanks!
I want to connect to bigquery from a desktop SQL client like execute query, is this possible?
Check Simba Drivers for Google BigQuery
The intent of these drivers is to help users connect the power of Google BigQuery with existing tooling and infrastructure that does not have native integration. If you are building new integrations with BigQuery, we recommend you consider the native API. The functionality exposed by an ODBC/JDBC connection is necessarily a subset of the full capabilities of BigQuery.
Meantime, asking just in case - have you tried already Native UI?
- https://bigquery.cloud.google.com
Related
More of a curiosity question really. I load data into Power BI report from Google BigQuery (using native Google BigQuery connector in Power BI). All works fine, but for some reason I don't see this query in BigQuery's query history.
Did anyone experience something similar and knows the reason why this happens or how to change that (if at all possible)?
If I do exactly the same thing but using simba ODBC connector, I see this query in BigQuery's query history as expected.
Never seen that before. I am always able to find the query history no matter what 3rd party connection I used. Could you confirm the GCP service-account or auth-account and the GCP project for BQ job query that you used for your native Google BigQuery connector in Power BI?
Please make sure you have the access to the query history of that GCP account in that BQ job project.
There is an option to connect a Cloud mySQL instance from BigQuery. I just wanted to know how we can connect a Cloud SQL Server instance to BigQuery.
SQL Server:
There are a bunch of third-party extensions/tools that provide this service. One of them is SSIS Data Flow Source & Destination for Google BigQuery, which is Visual Studio extension that connects SQL Server with Google BigQuery data through SSIS Workflows.:
https://www.cdata.com/drivers/bigquery/ssis/
https://marketplace.visualstudio.com/items?itemName=CDATASOFTWARE.SSISDataFlowSourceDestinationforGoogleBigQuery
In regards to using SQL Server Integration Services to load the data from the on-premises SQL Server to BigQuery, you can take a look for this site. You can also perform ETL from a relational database into BigQuery using Cloud Dataflow, the official documentation details how it can be done, you might need to use Cloud Storage as an intermediate data sink.
Cloud SQL:
BigQuery allows to query data from Cloud SQL by using federated query. The connection must be created within the same project where your Cloud SQL instance is located. If you want to query your data stored in your Cloud SQL instance from BigQuery located in another project, please follow the steps listed below:
Enable the BigQuery API and the BigQuery connection API within your project.
Create a connection to your Cloud SQL instance within the project by following this documentation.
Once you have created the connection, please locate and select it within BigQuery.
Click on the SHARE CONNECTION button and grant permissions to the users that will be use that connection. Please note that the BigQuery Connection User role is the only needed to use a shared connection.
Additionally, please notice that the "Cloud SQL federated queries" feature is in a Beta stage and might change or have limited support (is no available for certain regions, in which case, it is required to use one the supported options mentioned here). Please remember, that to use Cloud SQL Federated queries in BigQuery, the intances need to have a public IP.
If you are limited e.g. by region, one good option might be exporting the data from CloudSQL to Storage as a CSV, and then load it into BigQuery. If you need, it is possible to automate this process using Cloud Composer, refer to this article.
Other approach is to extract information from Cloud SQL (with exports) and import it into BigQuery through load jobs, or streaming inserts.
I hope you find the above pieces of information useful.
It is possible, but be warned the feature is currently Beta
https://cloud.google.com/bigquery/docs/cloud-sql-federated-queries
I need help for something, I'm new in Bigquery..
I want to see my SQL tables over Bigquery connection at the Data Studio. However I couldn't figure it out how to connect GCP mySQL instance from Bigquery.
I tried to change region/location of my SQL instance.(I think it helped a little) And using this query at the new BigQuery web UI.. but I receive below error.
SELECT *
FROM
myinstanceid.my_database_01.TABLE_YS ys
Error: ((Not found: Project myinstanceid))
You cannot directly connect MySQL databases the only external data sources supported are: GCS files, Cloud Bigtable, Google Drive (even directly google spreadsheets).
To run analytics on data inside your MySQL DB you will need to export data into a supported external data source or even better into a BigQuery native table.
I'm having datasets in bigquery google cloud project,
I want to load this data from powerbi,So that I can create visualizations using this bigquery data sets.
Currently I'm able to see the google bigquery in getdata option in powerbi desktop,I tried to connect to bigquery but I'm not abe to see my bigquery dataset in the list,I'm using the same credentials for powerbi and google bigquery.
is there any thig that I need to enable in google cloud side so that it will be visible in the list when I connect from powerbi?
These are the following steps you can follow to connect GBQ to Power BI.
You can write a query and save the table in Google big Query.
Go in Power BI, get data and click on more and scroll down there you will find "Google Big Query" click on that.
Search your saved table and click on "Direct Query"
I think you will be good to go!!
I would not recommend using Power BI for Big Query analysis. As of Q1 2019 Power BI still do not support project billing unless manually editing the connexion script, it does not support Sheets federated tables, do not support nested and repeated fields and Power BI DML is nearly unusable with BQ. Microsoft do not seems to be willing to develop the compatibility between Power BI and Big Query further.
Learn more in this article (i am the author): https://medium.com/#remy_david/which-bi-tool-for-big-query-d9eb838ff7ad
Not great at programming so not all that confident of being able to pull data from the Google API.
Basically what I'm trying to do is pull everything from Google Analytics, all data and dump it into a table on a SQL Server (2008 R2). What would be the best way of achieving this?
Would like to automate the process so that it could be scheduled say once a month.
Cheers
There an existing add-on specifically for accessing Google Analytics.
Haven't used it but from what I read seems like it will exactly what you are looking for
http://www.google.com/analytics/apps/about?app_id=1325007
There is a specific SSIS DataSource Component for that – https://analyticssource.codeplex.com
It is open source and free.