I have a Google Ads account which has a single Smart Campaign and multiple usual campaigns. Also I've set up a data transfer to Google BigQuery. When I try to compare BigQuery data using the query
SELECT sum(Cost) FROM `project.dataset.AccountBasicStats_XXXXXX` where Date between '2021-12-01' and '2021-12-31'
the query result shows a less cost than I see in the Google Ads interface for the same time period. The difference is equal to the spend of my smart campaign. To check this, I've tried the queries:
SELECT * FROM `project.dataset.CampaignBasicStats_XXXXXX` where Date between '2021-12-01' and '2021-12-31' AND CampaignId = {ID of my smart campaign}
SELECT * FROM `project.dataset.CampaignStats_XXXXXX` where Date between '2021-12-01' and '2021-12-31' AND CampaignId = {ID of my smart campaign}
The both give me no results.
Is it true that BigQuery data transfer discards the data of smart campaigns? What are other ways to get statistics for them?
BigQuery Data Transfer service for Google Ads is based on an old API, AdWords API v201809. This version of the API does not support Google Ads features introduced in 2019 or later, and Smart Campaigns is among that.
To get around this limitation, you could consider a different Data Transfer - for instance, SuperMetrics and FiveTran have Data Transfers for Google Ads too. Note they are paid, and you should make sure that they do support Smart Campaigns.
Google is said to be working on a Data Transfer using the newer Google Ads API.
Related
I have an issue with the GEO and Google Storage Transfer service. I use it to pull Google Ads data in to BigQuery. It's not empty but the numbers don't match. E.g. a query like this
SELECT _DATA_DATE, sum(clicks) FROM project.dataset.GeoStats_XXX where LocationType='LOCATION_OF_PRESENCE' and IsTargetingLocation IN (true, false) group by 1
returns a smaller number than the real one, the one that I get if I just create the same report via Google Ads UI. I've tried different manipulations with the LocationType and IsTargetingLocation filters but nothing seems to work.
I see no errors in Storage Transfer service logs
The CampaignStat returns almost correct data but there is no GEO info there and I need the country breakdown. Not Target Location but GEO (location of presence)
I expect the numbers returned by my query to match the numbers that I see via Google Ads UI
We are using the Google Ads transfer in BigQuery to ingest our Google Ads data. One thing I have noticed when querying the results is that all of the metrics are exactly 156x of the values we would expect in the Google Ads UI (cost, clicks, etc.)
We have tested multiple transfers and each time we have this same issue. The transfer process seems pretty straight forward, but am I missing something? Has anyone else noticed a similar issue or have any ideas of what to look at to adjust in the data transfer?
For which tables do you notice this behavior?
The dimension tables such as Customer, Campaign, AdGroup are exported every day and so are partitioned by day.
This could cause your duplication?!
You only need the latest partition/day.
So this is for example how I get the latest account / customer data:
SELECT
-- main reason I cast all the id's to string is because BI reporting tool will not see it as a metric but as a dimension field
CAST(customer_id AS STRING) AS account_id, --globally unique, see also: https://developers.google.com/google-ads/api/docs/concepts/api-structure
customer_descriptive_name,
customer_auto_tagging_enabled,
customer_currency_code,
customer_manager,
customer_test_account,
customer_time_zone,
_DATA_DATE AS date, --source table is paritioned on date
_LATEST_DATE,
CASE WHEN _DATA_DATE = _LATEST_DATE THEN TRUE ELSE FALSE END is_most_recent_record
FROM
`YOURPROJECTID.google_ads.ads_Customer_YOURID`
WHERE
_DATA_DATE = _LATEST_DATE
I am aware that Google Analytics can be linked to Bigquery using BigQuery Linking features in the GA.
But I experienced the drawback that it's scheduled at a random time. So, it's messed up my table with dependencies to these GA data, which I set up at 9 AM using DBT -- so if the GA data is updated above 9 AM, my table won't have today's GA data.
My questions are:
Is there a way to schedule the updated GA data to have constant time, as the cronjob did?
Or if there is not any. Is there a way for DBT to run the job after the GA data is updated on bigquery?
Unfortunately Google provide no SLA on the BigQuery export from Google Analytics 3, if you have the option the best solution would be to migrate to Google Analytics 4, which was an almost realtime export to BigQuery and appears to be much more robust. Find out more on the official Google support page.
I currently get around this by using event based triggers that look at the meta data of a table, or check for the existence of a sharded table for yesterday, then proceed down downstream jobs, I'm sure you could achieve something similar with DBT.
Here is some example SQL code which checks for the existence of yesterday's Google Analytics sharded table by returning the maximum timestamp:
SELECT MAX(cast(PARSE_DATE('%Y%m%d', SUBSTR(table_id,13)) as timestamp)) as max_date
FROM `my_ga_dataset.__TABLES__`
WHERE table_id LIKE'%ga_sessions_%'
AND table_id NOT LIKE '%intraday%'
AND PARSE_DATE('%Y%m%d', SUBSTR(table_id,13)) >= CURRENT_DATE() -9
This works for sharded tables, if you want to use table metadata to get the date/time of the last table update you can use INFORMATION_SCHEMA:
https://cloud.google.com/bigquery/docs/information-schema-tables
I have Google Analytics integrated to Bigquery and I'm trying to write a query to fetch Active Users that should match with the number on GA Portal.
Here's the query I've written;
SELECT
date(date) as date,
EXACT_COUNT_DISTINCT(fullVisitorId) as daily_active_users,
FROM TABLE_DATE_RANGE([<project_id>:<dataset>.ga_sessions_],
TIMESTAMP('2018-01-01'),
TIMESTAMP(CURRENT_DATE()))
group by date
order by date desc
The numbers I get in response are somehow related to the ones Google Analytics shows me, but they aren't a 100% accurate.
The numebers I get in return are slightely higher than the ones on the portal and I assume I need to put a where clause to filter a property GA might be filtering on the portal.
Your query looks fine to me. Assuming that you're looking at the same GA view as the one linked to BigQuery, I think that the problem could be sampling.
Even if the GA UI says that "This report is based on 100% of sessions.", try to export it as an Unsampled Report and check the numbers (in my experience, the users metric sometimes doesn't match between unsampled reports and default reports without sampling).
I started to test Google AdWords transfers for Big Query (https://cloud.google.com/bigquery/docs/adwords-transfer).
I have few questions for which I cannot find answers anywhere.
Is it possible to e.g. edit which columns are downloaded from AdWords to Big Query? E.g. Keyword report has only ad group ID column but not ad group text name.
Or is it possible to decide which tables=reports are downloaded? The transfer creates around 60 tables and I need just 5...
DZ
According to here, AdWords data transfer
store your AdWords data into a Dataset. So, the inputs are in terms of Adwords customer IDs (minimum one customer ID) and the output is a collection of Datasets.
I think, you need a modified version of PubSub to store special columns or tables in BigQuery.