All of a sudden I cannot get synonyms for tables working in BigQuery, so a query like the following works fine:
select id as id, value as value
from pos_dw_api.test
But a query like the following fails:
select a.id as id, a.value as value
from pos_dw_api.test a
The error returned is the following. I have run this from the web console:
Query Failed
Error: Unknown field: a.id
Synonyms were working just fine last week ... The example table I'm using for this select is 387047224813.pos_dw_api.test.
Has the syntax for synonyms changed? Is this a bug?
Table synonyms generally only work when you're doing a JOIN. I don't know of anything that would have caused this to change. I realize that this is kind of strange, and I've filed an internal bug to fix it.
Related
I'm writing a Java application to automatically build and run SQL queries. For many tables my code works fine but on a certain table it gets stuck by throwing the following exception:
Exception in thread "main" org.postgresql.util.PSQLException: ERROR: column "continent" does not exist
Hint: Perhaps you meant to reference the column "countries.Continent".
Position: 8
The query that has been run is the following:
SELECT Continent
FROM network.countries
WHERE Continent IS NOT NULL
AND Continent <> ''
LIMIT 5
This essentially returns 5 non-empty values from the column.
I don't understand why I'm getting the "column does not exist" error when it clearly does in pgAdmin 4. I can see that there is a schema with the name Network which contains the table countries and that table has a column called Continent just as expected.
Since all column, schema and table names are retrieved by the application itself I don't think there has been a spelling or semantical error so why does PostgreSQL cause problems regardless? Running the query in pgAdmin4 nor using the suggested countries.Continent is working.
My PostgreSQL version is the newest as of now:
$ psql --version
psql (PostgreSQL) 9.6.1
How can I successfully run the query?
Try to take it into double quotes - like "Continent" in the query:
SELECT "Continent"
FROM network.countries
...
In working with SQLAlchemy environment, i have got this error with the SQL like this,
db.session.execute(
text('SELECT name,type,ST_Area(geom) FROM buildings WHERE type == "plaza" '))
ERROR: column "plaza" does not exist
Well, i changed == by = , Error still persists, then i interchanged the quotes, like follows. It worked. Weird!
....
text("SELECT name,type,ST_Area(geom) FROM buildings WHERE type = 'plaza' "))
This problem occurs in postgres because the table name is not tablename instead it is "tablename".
for eg.
If it shows user as table name,
than table name is "user".
See this:
Such an error can appear when you add a space in the name of a column by mistake (for example "users ").
QUICK FIX (TRICK)
If you have recently added a field which you have already deleted before and now trying to add the same field back then let me share you this simple trick! i did this and the problem was gone!!
so, now just delete the migration folder entirely on the app,then instead of adding that field you need to now add a field but with the name of which you have never declared on this app before, example if you are trying to add title field then create it by the name of heading and now do the migration process separately on the app and runserver, now go to admin page and look for that model and delete all the objects and come to models back and rename the field that you recently made and name it to which you were wishing it with earlier and do the migrations again and now your problem must have been gone!!
this occurs when the objects are there in the db but you added a field which wasn't there when the earlier objs were made, so by this we can delete those objs and make fresh ones again!
I got the same error when I do PIVOT in RedShift.
My code is similar to
SELECT *
INTO output_table
FROM (
SELECT name, year_month, sales
FROM input_table
)
PIVOT
(
SUM(sales)
FOR year_month IN ('nov_2020', 'dec_2020', 'jan_2021', 'feb_2021', 'mar_2021', 'apr_2021', 'may_2021', 'jun_2021', 'jul_2021', 'aug_2021',
'sep_2021', 'oct_2021', 'nov_2021', 'dec_2021', 'jan_2022', 'feb_2022', 'mar_2022', 'apr_2022', 'may_2022', 'jun_2022',
'jul_2022', 'aug_2022', 'sep_2022', 'oct_2022', 'nov_2022')
)
I tried year_month without any quote (got the error), year_month with double quote (got the error), and finally year_month with single quote (it works this time). This may help if someone in the same situation like my example.
I am trying to run a very simple query that just select all the rows based upon some multiple criteria and all are inclusive i.e. I am using AND in select HiveQL statement. The table is an external table in Hive and the storage handler is phoenix, so I checked in phoenix also about that query and it is working fine, but in Hive, it is showing some java IO exception error which I am not able to get where I am wrong. The query I am using is:
SELECT * FROM msme_api.msme_jk WHERE gender='Male' AND socialcategory='General';
The complete error message is:
Error: java.io.IOException: org.apache.hadoop.hive.ql.exec.UDFArgumentLengthException: The operator 'AND' accepts at least 2 argument. (state=,code=0)
I am trying for external and for internal Hive tables, In both, the issues are still the same but when I give order by, OR statement then it's surprising that it works.
SELECT * FROM msme_api.msme_jk WHERE gender='Male' AND socialcategory='General' order by aid;
SELECT * FROM msme_api.msme_jk WHERE gender='Male' OR socialcategory='General';
Both works fine but with AND, I am getting the error.
I still confused about how hive is taking and processing the above queries and why I am not able to execute simple select statement. Any help will be appreciated.
I am running hive action in oozie, where hive.sql script has the below mentioned hive query.
Agenda:
Goal is to insert the outcome of this query into hive integrated hbase table, where main goal is to retrieve latest timestamp via transaction_dt column.
However when I run the same query via hive action in OOZIE, it fails with
error message:
org.apache.hadoop.hive.ql.parse.SemanticException: Line 0:-1 Argument
type mismatch 'rows': The 1st argument of EQUAL is expected to a
primitive type, but list is found
Seems like UDF defined variable is not expected here, however primitive type is expected.
When I run this query manually on hive shell, it works perfectly.
insert overwrite table hive_lookup_hbase
Select lookup.card_id, lookup.UCL, lookup.postcode, lookup.transaction_dt,lookup.score
from
(
SELECT ct.card_id, row_number() over ( partition by ct.card_id order by ct.transaction_dt desc ) rows,
ct.postcode , ct.member_id, ct.transaction_dt, ms.score, c_u_look.UCL
FROM hive_cardtrans_hbase ct
join cid_ucl_lookup c_u_look
join member_score ms on c_u_look.card_id=ct.card_id and ms.member_id=ct.member_id
)lookup
where rows=1;
As it works with hive shell, needs to know what changes I can make to make it work with oozie.
I'm not sure is this responsible for exception but obviously the join without ON condition is bad thing because it is CROSS JOIN. It seems the ON condition for the first join is not where it should be. It should be like this, isn't it?
FROM hive_cardtrans_hbase ct
join cid_ucl_lookup c_u_look on c_u_look.card_id=ct.card_id
join member_score ms on ms.member_id=ct.member_id
not this:
FROM hive_cardtrans_hbase ct
join cid_ucl_lookup c_u_look
join member_score ms on c_u_look.card_id=ct.card_id and ms.member_id=ct.member_id
Issue got fixed by setting the set hive.auto.convert.join=false in hive script.
This basically disables the optimization done by OOZIE, i thought, this option was disabled by default.
I'm writing a Java application to automatically build and run SQL queries. For many tables my code works fine but on a certain table it gets stuck by throwing the following exception:
Exception in thread "main" org.postgresql.util.PSQLException: ERROR: column "continent" does not exist
Hint: Perhaps you meant to reference the column "countries.Continent".
Position: 8
The query that has been run is the following:
SELECT Continent
FROM network.countries
WHERE Continent IS NOT NULL
AND Continent <> ''
LIMIT 5
This essentially returns 5 non-empty values from the column.
I don't understand why I'm getting the "column does not exist" error when it clearly does in pgAdmin 4. I can see that there is a schema with the name Network which contains the table countries and that table has a column called Continent just as expected.
Since all column, schema and table names are retrieved by the application itself I don't think there has been a spelling or semantical error so why does PostgreSQL cause problems regardless? Running the query in pgAdmin4 nor using the suggested countries.Continent is working.
My PostgreSQL version is the newest as of now:
$ psql --version
psql (PostgreSQL) 9.6.1
How can I successfully run the query?
Try to take it into double quotes - like "Continent" in the query:
SELECT "Continent"
FROM network.countries
...
In working with SQLAlchemy environment, i have got this error with the SQL like this,
db.session.execute(
text('SELECT name,type,ST_Area(geom) FROM buildings WHERE type == "plaza" '))
ERROR: column "plaza" does not exist
Well, i changed == by = , Error still persists, then i interchanged the quotes, like follows. It worked. Weird!
....
text("SELECT name,type,ST_Area(geom) FROM buildings WHERE type = 'plaza' "))
This problem occurs in postgres because the table name is not tablename instead it is "tablename".
for eg.
If it shows user as table name,
than table name is "user".
See this:
Such an error can appear when you add a space in the name of a column by mistake (for example "users ").
QUICK FIX (TRICK)
If you have recently added a field which you have already deleted before and now trying to add the same field back then let me share you this simple trick! i did this and the problem was gone!!
so, now just delete the migration folder entirely on the app,then instead of adding that field you need to now add a field but with the name of which you have never declared on this app before, example if you are trying to add title field then create it by the name of heading and now do the migration process separately on the app and runserver, now go to admin page and look for that model and delete all the objects and come to models back and rename the field that you recently made and name it to which you were wishing it with earlier and do the migrations again and now your problem must have been gone!!
this occurs when the objects are there in the db but you added a field which wasn't there when the earlier objs were made, so by this we can delete those objs and make fresh ones again!
I got the same error when I do PIVOT in RedShift.
My code is similar to
SELECT *
INTO output_table
FROM (
SELECT name, year_month, sales
FROM input_table
)
PIVOT
(
SUM(sales)
FOR year_month IN ('nov_2020', 'dec_2020', 'jan_2021', 'feb_2021', 'mar_2021', 'apr_2021', 'may_2021', 'jun_2021', 'jul_2021', 'aug_2021',
'sep_2021', 'oct_2021', 'nov_2021', 'dec_2021', 'jan_2022', 'feb_2022', 'mar_2022', 'apr_2022', 'may_2022', 'jun_2022',
'jul_2022', 'aug_2022', 'sep_2022', 'oct_2022', 'nov_2022')
)
I tried year_month without any quote (got the error), year_month with double quote (got the error), and finally year_month with single quote (it works this time). This may help if someone in the same situation like my example.
I installed apache hive-0.9.0 and start executing some basic commands and i found one abnormal behavior in select* command. In select statement after * any random characters are allowed in hive but in RDBMS its not allowed. I am not sure its expected behavior or bug in hive. Could some please confirm?
In the below query "abcdef" is random characters.
In RDBMS(oracle):
select *abcdef from mytable;
Output:
ERROR prepare() failed with: ORA-00923: FROM keyword not found where expected
In Hive:
select *abcdef from mytable;
Output:
Query worked fine and display all the records of mytable.
In RDBMS,select is followed by column name/* , but in HIVE,it may be column name/*/any regular expression(partitions) .
Go through this link
http://www.qubole.com/5-tips-for-efficient-hive-queries/
it provides whole description
I got response from apache hive committe. Its a indeed bug in hive. They filed a improvement ticket for this issue
Jira Ticket: https://issues.apache.org/jira/browse/HIVE-8155