Beginner
Im trying to use sqoop import from Oracle to HDFS
I get error message -
SQL command not properly ended , Import Failed : java:IOexception : No >column to generate for ClassWriter
my query is
--query "select a,b,c from db.table where date = to_date('9999-12-31','yyyy-MM-dd')" where \$conditions"
Your query contains two WHERE clause, this is syntax error. Use AND to add $conditions to the same WHERE, whole query should be double-quoted, including $CONDITIONS, it is the part of the same WHERE, and in your query double-quotes are broken.
Like this:
--query "select a,b,c from db.table where date = to_date('9999-12-31','yyyy-MM-dd') and \$CONDITIONS"
Check what exactly sqoop executes, does it resolve correctly or not (conditions passed correctly), use --verbose to get more information and fix query accordingly.
Related
I am running simple select query in BigQuery GCP Console and it works perfectly fine. But, when I run the same query using BQ CLI, it fails.
When I run the same query without the "WHERE" clause, it works.
SELECT field_path FROM `GCP_PROJECT_ID.MY_DATASET.INFORMATION_SCHEMA.COLUMN_FIELD_PATH`
WHERE table_name="MY_TABLE_NAME"
Below is the error message
Error in query string: Error processing job 'GCP_project_ID:jobidxxxxx': Unrecognized name:
MY_DATASET at [1:1xx]
I have tried the following "WHERE" clauses as well. None of these work as well.
... WHERE table_name IN ("MY_TABLE_NAME")
... WHERE table_name like "%MY_TABLE_NAME%"
I have reproduced your query on my own datasets using the Command Line tool and it worked fine. This is the command I ran:
bq query --use_legacy_sql=false 'SELECT field_path FROM `<projectName>.<DatasetName>.INFORMATION_SCHEMA.COLUMN_FIELD_PATHS`where table_name="<TableName>"'
I am trying to run a very simple query that just select all the rows based upon some multiple criteria and all are inclusive i.e. I am using AND in select HiveQL statement. The table is an external table in Hive and the storage handler is phoenix, so I checked in phoenix also about that query and it is working fine, but in Hive, it is showing some java IO exception error which I am not able to get where I am wrong. The query I am using is:
SELECT * FROM msme_api.msme_jk WHERE gender='Male' AND socialcategory='General';
The complete error message is:
Error: java.io.IOException: org.apache.hadoop.hive.ql.exec.UDFArgumentLengthException: The operator 'AND' accepts at least 2 argument. (state=,code=0)
I am trying for external and for internal Hive tables, In both, the issues are still the same but when I give order by, OR statement then it's surprising that it works.
SELECT * FROM msme_api.msme_jk WHERE gender='Male' AND socialcategory='General' order by aid;
SELECT * FROM msme_api.msme_jk WHERE gender='Male' OR socialcategory='General';
Both works fine but with AND, I am getting the error.
I still confused about how hive is taking and processing the above queries and why I am not able to execute simple select statement. Any help will be appreciated.
I'm running the BigQuery command line shell and I'm not able to successfully run multi-line queries (aka queries with line breaks) because whenever I paste the query into the shell each line gets run individually instead of the whole thing together.
For example,
select * from table
works fine because it is in one line but if I try to run
select
*
from
table
it does not work because each line gets run separately.
Is there any way to get this to work?
The query command creates a query job that runs the supplied SQL query. in the documentation Using the bq command-line tool you can find some examples like:
bq query --nouse_legacy_sql \
'SELECT
COUNT(*)
FROM
`bigquery-public-data`.samples.shakespeare'
Honestly I too couldn't find decent help on this, so I created a bit of a workaround using the CONCAT function. Notice that I have a space before the FROM clause. Hope his helps.
DECLARE table_name STRING DEFAULT '20220214';
DECLARE query STRING;
SET query = CONCAT("""select user_pseudo_id, user_id""",
""" FROM `app.analytics.events_""" || table_name || """` where user_id IS NOT NULL group by user_pseudo_id, user_id""");
EXECUTE IMMEDIATE query;
I need to run below Sql query through a python code and get the output of it.
How can I make it work using "os" module. I can't install sql module on the system where I have to execute this query in python.
Program should run this sql query, check for the account id in database and execute the output from sql query. I havent worked with sql from long time, getting issue to understand it.
It should be something like this but something is wrong here.
import os
id = raw_input("Enter account_id:")
os.system('sql2 -q abc.dev.domain.com "select account_name, account_id, is_enrolled from (select * from my_db where account_id = '$id' and is_enrolled = check by account_name')
Why can't we import data into hive CLI as following, The hive_test table has user, comments columns.
insert into table hive_test (user, comments)
value ("hello", "this is a test query");
Hive throws following exception in hive CLI
FAILED: ParseException line 1:28 cannot recognize input near '(' 'user' ',' in select clause
I don't want to import the data through csv file like following for a testing perpose.
load data local inpath '/home/hduser/test_data.csv' into table hive_test;
It's worth noting that Hive advertises "SQL-like" syntax, rather than actual SQL syntax. There's no particular reason to think that pure SQL queries will actually run on Hive. HiveQL's DML is documented here on the Wiki, and does not support the column specification syntax or the VALUES clause. However, it does support this syntax:
INSERT INTO TABLE tablename1 SELECT ... FROM ...
Extrapolating from these test queries, you might be able to get something like the following to work:
INSERT INTO TABLE hive_test SELECT 'hello', 'this is a test query' FROM src LIMIT 1
However, it does seem that Hive is not really optimized for this small-scale data manipulation. I don't have a Hive instance to test any of this on.
I think, it is because user is a built-in (Reserved) keyword.
Try this:
insert into table hive_test ("user", comments)
value ('hello', 'this is a test query');