Scheduling Query by using Script - google-bigquery

Question

I'd like to ask if it's possible or not to rum query scheduling by using script?
As for creating table, we could use script
CREATE TABLE dataset.xxx AS
...
Is there any way to do this but to CREATE A SCHEDULER instead of clicking the 'Schedule Query' Button?

Answer 1

As per documentation, in order to schedule a query , you can use one of the following methods:
BigQuery console and click "Schedule Query" as you mentioned in your question.
bq command
Python API
I will share same examples with you, first using bq command. From the Cloud Shell environment you can execute the following command:
bq query \
--use_legacy_sql=false \
--destination_table=mydataset.mytable \
--display_name='My Scheduled Query' \
--replace=true \
'SELECT
1
FROM
mydataset.test'
In addition, using bq command you can also use other flags, described here.
Second, using the Python API, you can configure your schedule query using the DataTransferServiceClient, which allows you to pass all the query configuration through a json dictionary, such as this example in the documentation and below:
from google.cloud import bigquery_datatransfer_v1
import google.protobuf.json_format
client = bigquery_datatransfer_v1.DataTransferServiceClient()
# TODO(developer): Set the project_id to the project that contains the
# destination dataset.
# project_id = "your-project-id"
# TODO(developer): Set the destination dataset. The authorized user must
# have owner permissions on the dataset.
# dataset_id = "your_dataset_id"
# TODO(developer): The first time you run this sample, set the
# authorization code to a value from the URL:
# https://www.gstatic.com/bigquerydatatransfer/oauthz/auth?client_id=433065040935-hav5fqnc9p9cht3rqneus9115ias2kn1.apps.googleusercontent.com&scope=https://www.googleapis.com/auth/bigquery%20https://www.googleapis.com/auth/drive&redirect_uri=urn:ietf:wg:oauth:2.0:oob
#
# authorization_code = "_4/ABCD-EFGHIJKLMNOP-QRSTUVWXYZ"
#
# You can use an empty string for authorization_code in subsequent runs of
# this code sample with the same credentials.
#
# authorization_code = ""
# Use standard SQL syntax for the query.
query_string = """
SELECT
CURRENT_TIMESTAMP() as current_time,
#run_time as intended_run_time,
#run_date as intended_run_date,
17 as some_integer
"""
parent = client.project_path(project_id)
transfer_config = google.protobuf.json_format.ParseDict(
{
"destination_dataset_id": dataset_id,
"display_name": "Your Scheduled Query Name",
"data_source_id": "scheduled_query",
"params": {
"query": query_string,
"destination_table_name_template": "your_table_{run_date}",
"write_disposition": "WRITE_TRUNCATE",
"partitioning_field": "",
},
"schedule": "every 24 hours",
},
bigquery_datatransfer_v1.types.TransferConfig(),
)
response = client.create_transfer_config(
parent, transfer_config, authorization_code=authorization_code
)
print("Created scheduled query '{}'".format(response.name))