I am trying to run one bq script using bigquery.Client
but getting below error,
Syntax error: Expected end of input but got string literal
v_minus_statement_s_t="DECLARE dummy_key DEFAULT to_hex(sha256('*nik'));
DECLARE dummy_value DEFAULT '*nik';
DECLARE currency_value DEFAULT '$';
select
to_hex(sha256(cast(ID as string))) as dw__id,
to_hex(sha256(cast(ID as string))) as dw_sub_id
from voice"
op =client.query(v_minus_statement_s_t).to_dataframe()
Related
I'm following this documentation on how to use the 'bq load' command to load data into BigQuery from a CSV in Google Cloud: https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv
However, I keep getting the same error: "Syntax error: Expected end of input but got identifier "bq" at [12:1]"
This is the very first part of my code, why is it expecting to get an end of input identifier? It's the beginning of my code. I have researched all over the internet and cannot find somebody with a similar issue.
Thank you in advance for your help.
Here's the code:
bq load \
--source_format=CSV \
cpo_metrics.12mo_hc \
gs://unity-ai-people-prd-test/12mo_hc.csv \
--autodetect=TRUE;
Here's what it looks like when I'm trying to run in the BQ console:
[1]: https://i.stack.imgur.com/oIDut.png
While integrating a UDF for use in a query, it's common to get a No matching signature error. These errors are very informative when the function signature is short. However when the function signature is long, such as in a case where it's a JS UDF receiving and returning complex structs, the error message is truncated and becomes useless.
For example:
No matching signature for function f for argument types: STRUCT<docGroup FLOAT64, docNum FLOAT64, docDate DATE, ...>.
Supported signature: f(STRUCT<docGroup FLOAT64, docNum FLOAT64, docDate DATE, ...>) at [202:8]
The actual mismatch is hiding somewhere in the truncated arguments!
Running bq show -j <jobid> also shows the message with a truncated list of arguments.
Is there any way of getting the full error message, or otherwise figuring out what's wrong, without having to take my SQL apart into pieces?
Try running bq show --format=prettyjson -j <jobid>
The documentation says that this is the syntax for seeing the job's complete information:
To see full job details, enter:
bq show --format=prettyjson -j
myproject:US.bquijob_123x456_789y123z456c
I am failing to use a parameter in function declaration.
a SQL script like :
CREATE OR REPLACE FUNCTION test_functon() RETURNS trigger AS
$BODY$
DECLARE
test int:=:SRID;
BEGIN
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql;
in a file.
And running psql -v SRID=2056 -f my_file.sql
leads to the error
ERROR: syntax error at or near ":"
This looks like the SQL is run without the var being properly replaced by its value.
What is the solution for this?
It seems like psql does not interpolate variables in the body of functions.
The following SQL verifies that.
SELECT :SRID;
CREATE OR REPLACE FUNCTION test_functon() RETURNS :TYPE AS
$BODY$
BEGIN
RETURN :SRID;
END;
$BODY$
LANGUAGE plpgsql;
Running that file results in this:
$ psql -v SRID=2056 -v TYPE=int -f query.sql
Expanded display is used automatically.
Null display is "ยค".
?column?
----------
2056
(1 row)
psql:query.sql:9: ERROR: syntax error at or near ":"
LINE 4: RETURN :SRID;
^
Notice how the return type defined by the variable TYPE is still interpolated, but everything inside the body is off limits.
You'll have to resort to a different mechanism to get your variable in there. You could leverage the fact that psql accepts a query through STDIN:
$ sed 's/:SRID/2056/' query.sql | psql
Initially took me a while to figure out that ">" causes an issue in cmd and learned from one of the answers that escaping with ^ works. So after changing my query to below format it works fine.
bq query --use_legacy_sql=false --destination_table= (SELECT * FROM `dataset_ID.table_nm` GROUP BY AAA HAVING SUM(BBB)^>0 )
But now I am trying to run the same query through a file.
bq query --use_legacy_sql=false --destination_table= --flagfile="Y:query.sql"
However above it gives me an error if I have "^>".
Error in query string: Error processing job Syntax error: Unexpected ">" at [1:227]
If I don't have "^" in front of the ">" the query doesn't return an error but the result is empty..
Hoping someone could help out with the issue above.
Thanks!
I've got a param which is like "This is a param", and I'm going to pass it to below hiveQL:
hive -hivevar sys_nm="This is a param" -e 'select * from rd_sys where rd_sys_nm=${hivevar:sys_nm}'
But Hive returned below error message:
Logging initialized using configuration in jar:file:/opt/mapr/hive/hive-0.13/lib/hive-common-0.13.0-mapr-1409.jar!/hive-log4j.properties
FAILED: ParseException line 1:49 missing EOF at 'is' near 'This'
g4t7491_[mgr#g4t7491 ~]$
Does anyone know how to pass it normally?
Hive var don't work like hiveconf where you need to apply "hiveconf:somthing" in the code
when declaring hivevar just add the var name like this -> ${var_name}
for example:
through command line:
hive -hivevar MONTH_VAR='11' -e "select * from table where month=${MONTH_VAR};"
you can also declair through the script:
set hivevar:MONTH_VAR=11;
-- so query would look like this (no hiveconf):
set hivevar:MONTH_VAR=11;
SELECT * from table where month=${MONTH_VAR};
You need to put the string in single quotes for it to parse correctly as a string inside the sql after interpolation.
hive -hivevar sys_nm="'This is a param'" -e 'select * from rd_sys where rd_sys_nm=${hivevar:sys_nm}'