According to documentation here, I should be able to reference a table wildcard function as an alias:
...
FROM
[project_name:]datasetId.tableId |
(subselect_clause) |
table wildcard function
[[AS] alias]
...
But, when I try to do this, things fail:
bq query "SELECT * FROM TABLE_QUERY(my_data, \"TIMESTAMP(table_id) BETWEEN TIMESTAMP('2014-05-21') AND TIMESTAMP('2014-06-10')\") AS blah WHERE blah.foo = 5 LIMIT 30"
Waiting on bqjob_some_id ... (0s) Current status: DONE
BigQuery error in query operation: Error processing job 'some_id': Field 'blah.foo' not found in table 'mydata.20140521'; did you mean 'foo2'?
I want to do a join on the data returned from the wildcard function treating it as a single table, so aliasing it is very important to me. Ideas?
The page you linked to does mention "Do not use alias with a table wildcard function."
You might want to try
SELECT *
FROM (SELECT * FROM TABLE_QUERY(--insert query here--)) AS blah
WHERE blah.foo = 5 LIMIT 30"
Related
I have simple elastic SQL query like this:
GET /_sql?format=txt
{
"query" :"""
DESCRIBE "index_name"
"""
}
and it works, and the output is like this:
column | type | mapping
-----------------------------------------------------------
column_name1 | STRUCT | object
column_name1.Id | VARCHAR | text
column_name1.Id.keyword | VARCHAR | keyword
Is there a possibility to the prepare above query using filter or where, for example something like this:
GET /_sql?format=txt
{
"query":"""
DESCRIBE "index_name"
""",
"filter": {"terms": {"type.keyword": ["STRUCT"]}}
}
or
GET /_sql?format=txt
{
"query":"""
DESCRIBE "index_name"
WHERE "type" = 'STRUCT'
"""
}
That is not possible, no.
While the DESCRIBE sql command seems to return tabular data, it is not a query and it does not support WHERE clauses or can be used within a SELECT statement. That is actually not specific to Elasticsearch, but the same in RDBMs.
The same apparently is true for the Elasticsearch filter clause. This again will work with SELECT SQL statements, but with DESCRIBE or SHOW COLUMNS - while not producing an error - it simply will have no effect on the results.
In "real" SQL, you could work around this by querying information_schema.COLUMNS, but that is not an option in Elasticsearch.
after searching through the whole www I finally ended up here with a question and hopefully more capable responders than I am.
I am trying to implement a full text search on my webpage.
The Sql query works with one WHERE condition but not with several. Unfortunately I have no clue how to solve my problem with the help of EXIST / IN / INNER JOIN or other Operators.
Code which works:
$sql = "SELECT * FROM artikel WHERE author LIKE '$suchtext' LIMIT $offset, $no_of_abs";
Code which doesnt work:
$sql = "SELECT * FROM artikel WHERE (author OR shorty OR vollartikel) LIKE '%$suchtext%' LIMIT $offset, $no_of_abs";
The variable $suchtext is correctly passed and received through jquery $ajax POST and my php script.
Example goal:
author | volltext | shorty
--------+-------------+---------
tulum | tul | asf
rae | zutotu | vizetu
$suchtext is "tu"
Results: 2
Row1 & Row2
I appreciate any answers.
Solution:
$sql = "SELECT * FROM artikel WHERE author LIKE '%$suchtext%' OR shorty LIKE '%$suchtext%' OR vollartikel LIKE '%$suchtext%' LIMIT $offset, $no_of_abs";
LIKE is a binary operator that takes two strings. OR is a boolean operator that operates on boolean values.
You need to explicitly list this out:
(author LIKE ? OR
shorty LIKE ? OR
vollartikel LIKE ?
)
Note that this uses ? as a parameter placeholder. Use parameters when constructing SQL queries in applications.
I'd like to create a table name in Hive using variable substitution.
E.g.
SET market = "AUS";
create table ${hiveconf:market_cd}_active as ... ;
But it fails. Any idea how it can be achieved?
You should use backtrics (``) for name for that, like:
SET market=AUS;
CREATE TABLE `${hiveconf:market}_active` AS SELECT 1;
DESCRIBE `${hiveconf:market}_active`;
Example run script.sql from beeline:
$ beeline -u jdbc:hive2://localhost:10000/ -n hadoop -f script.sql
Connecting to jdbc:hive2://localhost:10000/
...
0: jdbc:hive2://localhost:10000/> SET market=AUS;
No rows affected (0.057 seconds)
0: jdbc:hive2://localhost:10000/> CREATE TABLE `${hiveconf:market}_active` AS SELECT 1;
...
INFO : Dag name: CREATE TABLE `AUS_active` AS SELECT 1(Stage-1)
...
INFO : OK
No rows affected (12.402 seconds)
0: jdbc:hive2://localhost:10000/> DESCRIBE `${hiveconf:market}_active`;
...
INFO : Executing command(queryId=hive_20190801194250_1a57e6ec-25e7-474d-b31d-24026f171089): DESCRIBE `AUS_active`
...
INFO : OK
+-----------+------------+----------+
| col_name | data_type | comment |
+-----------+------------+----------+
| _c0 | int | |
+-----------+------------+----------+
1 row selected (0.132 seconds)
0: jdbc:hive2://localhost:10000/> Closing: 0: jdbc:hive2://localhost:10000/
Markovitz's criticisms are correct, but do not produce a correct solution. In summary, you can use variable substitution for things like string comparisons, but NOT for things like naming variables and tables. If you know much about language compilers and parsers, you get a sense of why this would be true. You could construct such behavior in a language like Java, but SQL is just too crude.
Running that code produces an error, "cannot recognize input near '$' '{' 'hiveconf' in table name".(I am running Hortonworks, Hive 1.2.1000.2.5.3.0-37).
I spent a couple hours Googling and experimenting with different combinations of punctuation, different tools ranging from command line, Ambari, and DB Visualizer, etc., and I never found any way to construct a table name or a field name with a variable value. I think you're stuck with using variables in places where you need a string literal, like comparisons, but you cannot use them in place of reserved words or existing data structures, if that makes sense. By example:
--works
drop table if exists user_rgksp0.foo;
-- Does NOT work:
set MY_FILE_NAME=user_rgksp0.foo;
--drop table if exists ${hiveconf:MY_FILE_NAME};
-- Works
set REPORT_YEAR=2018;
select count(1) as stationary_event_count, day, zip_code, route_id from aaetl_dms_pub.dms_stationary_events_pub
where part_year = '${hiveconf:REPORT_YEAR}'
-- Does NOT Work:
set MY_VAR_NAME='zip_code'
select count(1) as stationary_event_count, day, '${hiveconf:MY_VAR_NAME}', route_id from aaetl_dms_pub.dms_stationary_events_pub
where part_year = 2018
The qualifies should be removed
You're using the wrong variable name
SET market=AUS; create table ${hiveconf:market}_active as select 1;
I'm unable to reference a SELECT alias in BigQuery (standard mode).
Trying to do this query:
SELECT
REGEXP_EXTRACT_ALL(text,
r"(<div \w+>)") AS matches
FROM
regex.test
WHERE
matches IS NOT NULL
Here are steps to reproduce.
bq mk regex
bq mk -t regex.test id:integer,text:string
echo '{"id":1, "text":"<div a>"}' | bq insert regex.test
echo '{"id":2, "text":"<div b>"}' | bq insert regex.test
echo '{"id":3, "text":"<div>"}' | bq insert regex.test
bq query --use_legacy_sql=false "select REGEXP_EXTRACT_ALL(text, r\"(<div \w+>)\") AS matches FROM regex.test WHERE id IS NOT NULL"
+--------------+
| matches |
+--------------+
| [u'<div b>'] |
| [] |
| [u'<div a>'] |
+--------------+
When I try to reference the matches alias, I see an error:
bq query --use_legacy_sql=false "select REGEXP_EXTRACT_ALL(text, r\"(<div \w+>)\") AS matches FROM regex.test WHERE matches IS NOT NULL"
Error in query string: Error processing job 'myname': Unrecognized name:
matches
I am unable to reference the alias matches, and am unable to filter those results WHERE matches IS NOT NULL.
Does anyone know what I'm doing incorrectly here?
Thanks!
Even in BQ, you can't use a column alias in the where clause. Just use a subquery:
SELECT t.*
FROM (SELECT REGEXP_EXTRACT_ALL(text, r"(<div \w+>)") AS matches
FROM regex.test
) t
WHERE ARRAY_LENGTH(matches) > 0
Check out SELECT list aliases visibility
The reason why comparing with NULL does't work for REGEXP_EXTRACT_ALL is because
it returns array so checking with length is the way. Comparing with NULL still will work for REGEXP_EXTRACT
In addition, ideally you should be able use REGEX_MATCH to filter out records w/o matches, but looks like there is an issue with this function in standard mode
Here is my simplified code (little more than some basic elements to cause the SQL not to execute):
select *
from (
select replace(mytxtfield, "llama", "") as badones
from XYZ
)
where badones is not null;
The outer query runs fine when the WHERE cause is:
badones like "ZZZ-[0-9][0-9][0-9]"
but it breaks when the WHERE cause includes more than one LIKE (of any digit matching pattern) such as:
badones like "ZZZ-[0-9][0-9]" OR
badones like "ZZZ-[0-9][0-9][0-9]"
More info:
mytxtfield is of type Text
It doesn't matter if there is a WHERE cause in the inner query to check eliminate null / empty strings.
64-bit office
Since your sub-query returns the alias "badnews", you must use it in place of "badones" in your
outer query. The following produces no errors:
SELECT *
FROM (select replace(mytxtfield, "llama", "") as badnews
from XYZ
)
WHERE (((badnews) Like "ZZZ-[0-9][0-9]" Or (badnews) Like "ZZZ-[0-9][0-9][0-9]"));