I'm trying to set up a dashboard in Google Data Studio with Apple News analytics data as one of the sources.
I can see you can download this analytics data manually as a CSV - does anyone know a way of automating this extract? Automatically appending the data weekly to a BigQuery table would be ideal, or Google Sheets or directly into Data Studio if not.
Thanks.
You can load your CSV into BigQuery [1], or schedule a load job, and then use it in datastudio through a BigQuery reader package. Otherwise, if you do not need to append the data you can simply import it with other packages as "Custom JSON/CSV/XML" By Supermetrics.
[1] https://cloud.google.com/bigquery/docs/loading-data#supported_data_formats
Related
I'm using web api to import data into powerbi. After every refresh, old data is replaced by new data of web api so my question is how can I store that old data in power bi ?
Power BI will not store data, unless you have a query source that will support incremental refresh.
https://learn.microsoft.com/en-us/power-bi/admin/service-premium-incremental-refresh
It would be best to use a tool like Azure Function, Azure Logic Apps or Power Automate to get the data and save it as file to a folder then import the data from the folder. Other option would be to move the data to a database table to preserve the history.
I'm only seeing how to load data to my bigquery database but i don't see any wiki or post to help me on my question : Is it possible to load data to an external source like google sheet from a bigquery database (that can go thought the cloud storage plateforme)
you can export data from BigQuery, yes. One way to bring BigQuery data into Google Sheets is with the new Google Sheets Data Connector, see the screenshot below for where you'll find this on the GS interface:
Once connected, you can then write an SQL query and pull this data directly into a Google Sheet.
Here's a link to some official documentation on this data connector: https://support.google.com/docs/answer/9077536
I'm importing datasets in Google Cloud Dataprep (by Trifacta) to perform transformations on my data sources. But I can't see Google Drive Sheets in the list after connecting them to Big Query Console. I'm about to use them as rules for my transformations.
I've already created another dataset and the problem persists.
Is it possible to import them or not supported yet?
Thanks,
You are right. According to the documentation Dataprep only supports native BigQuery tables and views as BigQuery sources.
You could try downloading your Drive sheets as csv and then creating a BigQuery table from it, or maybe you could create a load job from your external table into a new native table using:
SELECT * FROM my_dataset.my_external_table
I have a question regarding refreshing google big query table where the data source is google drive.
Imagine, you have CSV file on google drive and every day someone updates for you.
1. The filename is not changing
2. location URI is same
How can I refresh my big query table by using this google drive file?
Could you please guide me or send me related links?
Thanks
From the BigQuery docs:
Loading data into BigQuery from Google Drive is not currently
supported, but you can query data in Google Drive using an external
table.
The link above provides instructions on how to create an external table that references your stored-in-Drive data source. Considering that you want to be querying data from a Google Drive file which you will be updating in Drive, this is the solution you are looking for (in contrast to downloading your csv locally and then loading it into BQ, in which case you would then have to be updating directly in BQ).
I'm a backend developer who has no experience with Google Analytics, but I've a requirement to find a way to collect the Marketing Medium/Source for each user from Google Analytics and save it in my database, I've been searching and looking how to get it from an API request but I didn't find a way yet, could you guys help?
You can use the Google Python API to fetch the Google Analytics data. You can read more here.
Medium and Source information can be found out by using the dimension ga:sourceMedium
You can find more info about dimensions and metrics here
Following which you can setup a daily script and fetches the data from your Google Analytics account and dumps data into csv which you can successively load into your database using libraries such as psycopg2.