Spark sql case clause - mismatched input - apache-spark-sql

the query
select
CASE WHEN b.colA = 'ZZZ'
THEN 'xxx'
ELSE 'yyy'
END
from mytable b
;
executed in a spark context as SQL returns the following error message:
ParseException:
mismatched input ' ' expecting {<EOF>, ';'}(line 3, pos 13)
== SQL ==
select b.colA,
CASE WHEN b.colA = 'ZZZ'
-------------^^^
THEN 'xxx'
ELSE 'yyy'
END
from mytable b
;
Any idea what is wrong here?
Thanks!

Try
select b.colA,
CASE WHEN b.colA = 'ZZZ'
THEN 'xxx'
ELSE 'yyy'
END
from mytable b
;
val df4 = df.select(col("*"),
expr("case when gender = 'M' then 'Male' " +
"when gender = 'F' then 'Female' " +
"else 'Unknown' end").alias("new_gender"))
search ...

Related

SQL CONCAT statement with multiple conditions

I have a table with output as of following:
I am trying to concat the outputs into a single output, given the conditions that:
When column fields (Type 1 to Type 5) IS NOT NULL OR '', take the value and combine it with another field with the same condition met
Expected output(based on screenshot above): RAM/TOTAL/N.A.
When column field (Type 6) IS NOT NULL OR '', display the result as OUT OF SERVICE, ignoring other values
Expected output: OUT OF SERVICE
Sample query:
SELECT CONCAT(Type1, '/' , Type2, '/' , Type3, '/' , Type4, '/' , Type5, '/' , Type6) AS OUTPUT FROM #myTable
You example screen shots do not agree with your expected output.
You say if Type6 is Null or blank, then it should say 'OUT OF SERVICE'
However, in your screen shot, your expected output concats the previous columns though Type6 is blank! Should it not say OUT OF SERVICE ??
Assuming that this is simply a typo.. what you could do is the following:
Set the values of the column to null if blank and then concat them in the subsequent select. Just one one many ways to do it..
;with mycte as
(select
'' as Type1
, 'RAM' as Type2
,'' as Type3
,'TOTAL' as Type4
,'N/A' as Type5
,'' as Type6
)
,set_null as (
select
Type1 = case when isnull(Type1,'') = '' then NULL else Type1+'/' end
,Type2 = case when isnull(Type2,'') = '' then NULL else Type2+'/' end
,Type3 = case when isnull(Type3,'') = '' then NULL else Type3 +'/'end
,Type4 = case when isnull(Type4,'') = '' then NULL else Type4+'/' end
,Type5 = case when isnull(Type5,'') = '' then NULL else Type5+'/' end
,Type6 = case when isnull(Type6,'') = '' then NULL else Type6 end
from mycte
)
select
case
when isnull(Type6,'') = '' then 'OUT OF SERVICE'
else concat(Type1+Type2,Type3,Type4,Type5,Type6)
end as concat_column
from set_null
You may try this. This is an approach use to remove unnecessary character added due to no null blank value from a string.
First create your string and remove the duplicate unnecessary / from the string.
At the end only need to check whether / is added in suffix or prefix which need to be removed, by using simple case and substring you can achieve the same.
;with cte as
(select 'sd' as Type1, 'RAM' as Type2,'' as Type3,'TOTAL' as Type4,'N.A' as Type5,null as Type6)
,ct as (
SELECT case when Type6 is null ---- you may chnage your condition of checking Type6 is over there
then 'OUT OF SERVICE'
else REPLACE( REPLACE( CONCAT('][', Type1, '][' , Type2, '][' , Type3, '][' , Type4, '][' , Type5, '][' , Type6), '[]', ''), '][', '/')
End AS OUTP FROM cte
)
select
case when OUTP like '/%'
then case when OUTP like '%/'
then SUBSTRING( OUTP, 2, len(OUTP)-2 )
else SUBSTRING( OUTP, 2, len(OUTP)-1 ) end
else case when OUTP like '%/'
then SUBSTRING( OUTP, 1, len(OUTP)-1 )
else OUTP end
End as OUTPU
from ct

pyspark sql jdbc error - Incorrect syntax near the keyword &apos;SET&apos;

Following is the SQL Sample which Im trying to run, but its is giving me error - ***.jdbc.SQLServerException: Incorrect syntax near the keyword &apos;SET&apos;
tp_temp = """
SET NOCOUNT ON
declare #lp_dt dt = (select max(pt_dt) from [DB_TABLE].[db].[DATA] where p_type='WSD')
declare #L_dte tb (pt, pt_type char(1))
insert into #L_dte SELECT pt_dt, pt_type
FROM from [DB_TABLE].[db].[DATA]
where (PT_Type = 'Z' and Month(pt_dt) in (1,4,7,10) and pt_dt > '2017 Sep 1') or (PT_Type = 'K' and pt_dt = #lp_dt )
group by pt_dt, PT_Type
order by pt_dt
SELECT z_id_alt, z_id, z_Status_Alt, c_Gp_CCD, Gender, Age, ctk as CTK_Red, CAT_Title, temp_Group, Comp_price, terminator_action_alt AS Terminator_Action, v_pt.PT_Dt,CASE
WHEN COALESCE([Acb_Indicator], '') = ''
THEN ('No')
ELSE ('Yes')
END AS [Is Acb],
COALESCE(nullif([Region_Desc_NEWCO_Original], ''), Region_Desc_NEWCO) AS Region_Desc_NEWCO,
COALESCE(nullif([POG3_Desc_Original], ''), POG3_Desc) AS POG3_Desc,
COALESCE(nullif([POG9_Desc_Original], ''), POG9_Desc) AS POG9_Desc, COALESCE(nullif(BSD_Function, ''), OrgUnit_Function_L2_NEWCO) as OrgUnit_Function_L2_NEWCO, CASE
WHEN COALESCE(nullif([POG3_Desc_Original], ''), POG3_Desc) IN (
SELECT [POG_Desc]
FROM [DB_FREE].[dbz].[Free_dom]
WHERE POG_Type = 'POG3'
)
OR COALESCE(nullif([POG9_Desc_Original], ''), POG9_Desc) IN (
SELECT [POG_Desc]
FROM [DB_FREE].[dbz].[Free_dom]
WHERE POG_Type = 'POG9'
)
THEN 'Yes'
ELSE 'No'
END AS [Restricted_Home], Zero) AS Booring
FROM (SELECT * FROM [DB_FREE].[dbz].[Free_dom] WHERE noshow = 'N' AND E_Type <> 'SubConsole') as v_pt
INNER JOIN #l_pt_dt as pt_dt on v_pt.pt_dt = pt_dt.pt_dt and v_pt.pt_type = pt_dt.pt_type
LEFT OUTER JOIN [DB_TABLE].[db].[DATA] AS o_p ON o_p.[u_number] = v_pt.ete_idg """
df_temp= spark.read.jdbc(url=jdbcUrl, table=tp_temp, properties=connectionProperties)
For simple SQL starting with Select command it works but for code starting with SET or WITH it throws errors, please let me know how I can create a table from this SQL code in pyspark dataframe

CONCAT with IF condition in SQL Server

I have a table with four columns presenting {YES, NO, N/A} values. What I'd like to obtain is a column with concatenated names of those columns which present a 'YES' value separate by a double underscore.
\, A, B, C, D
1, YES, NO, YES, N/A
2, NO, YES, N/A, N/A
3, YES, NO, NO, YES
Expected result:
A__C
B
A__D
Something like:
select CONCAT(
IF(A = 'YES', 'A'),
IF(B = 'YES', 'B'),
IF(C = 'YES', 'C'),
IF(D = 'YES', 'D'))
from my_table
Hope I understand you right, that you want a double underscore separator.
This solution works without any subquery or cte processing.
select substring(
iif(a='YES','__A','') + iif(b='YES','__B','') +
iif(c='YES','__C','') + iif(d='YES','__D','')
,3,100)
from table1
One should know that this: substring('', 3, 100) will work using SqlServer.
Assuming T1 is your table:
SELECT CASE WHEN LEN(X)>0 THEN LEFT(X, LEN(X)-2) ELSE '' END AS Y
FROM (
SELECT
CASE WHEN A='YES' THEN 'A__' ELSE '' END + CASE WHEN B='YES' THEN 'B__' ELSE '' END + CASE WHEN C='YES' THEN 'C__' ELSE '' END + CASE WHEN D='YES' THEN 'D__' ELSE '' END AS X
FROM T1
) A
WITH ABC
as
(
Select
(
CASE WHEN A = 'YES' THEN 'A_' ELSE '' END as A +
CASE WHEN B = 'YES' THEN 'B_' ELSE '' END as B +
CASE WHEN C = 'YES' THEN 'C_' ELSE '' END as C +
CASE WHEN D = 'YES' THEN 'D_' ELSE '' END as D
) as output
)
Select case when len(output) = 2 then left (output,1)
else output
end as output
From ABC
Select case A then 'YES' then 'A' else '_'end + case B then 'YES' then 'B' else '_'end +case C then 'YES' then 'C' else '_'end +case D then 'YES' then 'D' else '_'end as result from my_table

CONCATENATE a CASE in Oracle SQL

I need to run a CASE expression on a number of columns, the columns are Boolean, so if it's 0 I need to populate the column with the column name and if it's 1, I ignore the column/value. I then need to concatenate all these columns into one. Is it possible to do this in Oracle SQL?
I've tried this:
Select
||CASE
WHEN COL_A = 0 THEN 'COL_A'
ELSE ''
END||','
||CASE
WHEN COL_B = 0 THEN 'COL_B'
ELSE ''
END||
from ABC.123
Can this even been done? If not this way are there any other ways?
Yes, it will work (if you clean up the syntax). Here's a simple example:
with q as (
select 0 col_a, 1 col_b, 'Rec 1' id from dual
union all
select 1, 0, 'Rec 2' from dual
union all
select 0, 0, 'Rec 3' from dual
)
Select id,
CASE
WHEN COL_A = 0 THEN 'COL_A'
ELSE ''
END||','
||CASE
WHEN COL_B = 0 THEN 'COL_B'
ELSE ''
END "TheString"
from q
Result:
ID TheString
------- -------------------
Rec 1 COL_A,
Rec 2 ,COL_B
Rec 3 COL_A,COL_B

Compatible SQL to test for not null and not empty strings

I want to have compatible SQL for both Oracle database and Microsoft SQL server.
I want a compatible SQL expression that will return true for not null and not empty strings.
If I use:
column <> ''
it will work on Microsoft SQL server but not on Oracle database (as '' is null for Oracle)
If I use:
len(column) > 0
it will work on Microsoft SQL server but not on Oracle database (since it uses length() )
NULLIF is available on both Oracle (doc) and SQL Server (doc). This expression should work:
NULLIF(column, '') IS NOT NULL
In both servers, if column is NULL, then the output of NULLIF will just pass the NULL value through. On SQL Server, '' = '', so the output of NULLIF will be NULL. On Oracle, '' is already NULL, so it gets passed through.
This is my test on SQL Server 2008 R2 Express:
WITH SampleData AS
(SELECT 1 AS col1, NULL AS col2
UNION ALL
SELECT 2, ''
UNION ALL
SELECT 3, 'hello')
SELECT *
FROM SampleData
WHERE NULLIF(col2, '') IS NOT NULL;
And this is my test case on Oracle 10g XE:
WITH SampleData AS
(SELECT 1 AS col1, NULL AS col2 FROM DUAL
UNION ALL
SELECT 2, '' FROM DUAL
UNION ALL
SELECT 3, 'hello' FROM DUAL)
SELECT *
FROM SampleData
WHERE NULLIF(col2, '') IS NOT NULL;
Both return 3 as expected.
How about
CASE WHEN column = '' THEN NULL ELSE column END IS NOT NULL
I think the key here is to differentiate between the case when the empty string is equivalent to NULL and when it isn't:
WHERE CASE WHEN '' = '' THEN -- e.g., SQL Server this is true
CASE WHEN col <> '' AND col IS NOT NULL THEN 'Y'
ELSE 'N'
END
WHEN COALESCE(col,NULL) IS NOT NULL THEN 'Y' -- Not SS, e.g., Oracle
ELSE 'N'
END = 'Y';
If the first case is true then empty string is not the same as null, and we have to test for both string being not null and string not being the empty string. Otherwise, our task is easier because empty string and null evaluate the same.
A try to shorten #DCookie's answer. I like his ( '' = '' ) test.
CASE WHEN ( '' = '' ) THEN ( column <> '' )
ELSE ( column = column )
END
Sadly, the above will not work. The next works in SQL-Server. I can't test in Oracle now:
CASE WHEN '' = '' THEN CASE WHEN column <> '' THEN 1 ELSE NULL END
ELSE CASE WHEN column = column THEN 1 ELSE NULL END
END
which can be written also as:
( '' = '' AND column <> '' )
OR ( '' IS NULL AND column = column )