How to use variable value in select query in hive - hive

I am finding number of days in a month as below.
set ndays=datediff(CONCAT(y, '-', (m + 1), '-', '01'), CONCAT(y, '-', m, '-', '01')) FROM (SELECT month(current_date) as m, year(current_date) as y, day(current_date)) tabl1;
I am checking the value.
select ${hiveconf:ndays}; --O/P 31
I am using this variable in a query and getting error.
select
sum(price)/ ${hiveconf:ndays}
from sales_aly
GROUP BY sales_month;
FAILED: ParseException line 3:0 missing EOF at 'from' near 'tabl1'
Please help me.
Thanks in Advance.

set command will not calculate value of this expression: datediff(CONCAT(y, '-', (m + 1), '-', '01'), CONCAT(y, '-', m, '-', '01')) FROM (SELECT month(current_date) as m, year(current_date) as y, day(current_date)) tabl1 but assigns expression itself as a value. Then when you use variable in the code, expression is substituted instead of variable.
You can calculate your expression in shell and pass it to hive like this:
var=$(hive -e "set hive.cli.print.header=false; select datediff(CONCAT(y, '-', (m + 1), '-', '01'), CONCAT(y, '-', m, '-', '01')) FROM (SELECT month(current_date) as m, year(current_date) as y, from tab1 limit 1) tabl1;")
# Then call your script:
hive -hiveconf ndays=$var -f your_script_file_name
Address variable inside your script: ${hiveconf:ndays}
Alternatively you can use macro. See this: http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/macro.q

Related

Extracting the times when the data looks like this - 'Monday|7:00-17:00' in SQL?

Is there a way to extract the starting time and the ending time of this type of text data in SQL? 'Monday|7:00-17:00'
I need to calculate the difference and get the number of hours.
I tried SUBSTR but since the days have different lengths MONDAY, TUESDAY, etc., it doesn't work.
You can use a mix of SUBSTR and INSTR functions to retrieve the hours like this:
SELECT SUBSTR('Monday|7:00-17:00', INSTR('Monday|7:00-17:00','|')+1, 100);
Take a peek at this example: https://dbfiddle.uk/HyMsdfoJ
select dt
, instr(dt, '|')
, substr(dt, instr(dt, '|') + 1, 100)
, instr(substr(dt, instr(dt, '|') + 1, 100), '-')
, substr(
substr(dt, instr(dt, '|') + 1, 100),
1,
instr(substr(dt, instr(dt, '|') + 1, 100), '-') - 1
) as starttime
, substr(
substr(dt, instr(dt, '|') + 1, 100),
instr(substr(dt, instr(dt, '|') + 1, 100), '-') + 1,
100
) as endtime
from (select 'Saturday|9:00-11:00' as dt) t
You can change select 'Saturday|9:00-11:00' as dt to select 'Mon|17:00-18:00' as dt and try it again.
In the query above, I break down the problem by:
finding the position where | exists. If the text was Mon|17:00-18:00, that'd be position 4
extracting the text after | using substr, which would be 17:00-18:00
finding the position of - in 17:00-18:00, which would be 6
putting learnings from the above 3 items to get the start time
I take the text after | (17:00-18:00) and extract from position 1 to position 5
That gives me 17:00
repeat a similar technique to get end time
I take the text after - in 17:00-18:00 by extracting 100 characters after position 7
100 is an arbitrary number I chose. You can get more sophisticated with a bit of math and extract the exact length
Here's another example with a table: http://sqlfiddle.com/#!5/4f347/2/0
EDIT - if you don't have INSTR
You could use this example http://sqlfiddle.com/#!5/9fcb6/1 if you don't have INSTR function.
You can try this at your own convenience.
select
replace(
replace(
ltrim(
'Monday|7:00-17:00',
replace('Monday|7:00-17:00', '|', '')
), '|', ''
),
ltrim(
'Monday|7:00-17:00',
replace('Monday|7:00-17:00', '-', '')
),
'') as starttime,
replace(
ltrim(
'Monday|7:00-17:00',
replace('Monday|7:00-17:00', '-', '')
), '-', ''
) as endtime
Here's a stepwise showcase of how I got to the above method: https://dbfiddle.uk/RBffgB9q

How do I covert a number to a date in SQL?

I have a field named SECURED that has dates listed as 123120, 040320 which is actually 12/31/20 or 4/03/20.
I was able to convert the date in the SELECT area with:
SELECT LEFT(SECURED, 2) + '/' + replace(replace(SECURED, LEFT(SECURED, 2), ''),
RIGHT(SECURED, 2), '') + '/' + RIGHT(SECURED, 2) as 'Modified SECURED as Date'
WHERE CONVERT(date, SECURED, 101)
BETWEEN CONVERT(date, getdate() - 30, 101)
AND CONVERT(date, getdate(), 101)
How can I pull in information from the SECURED column with a date of 30 days ago?
It doesn't work and still sees a date as a number.
You can use datefromparts() to convert your string to a date:
datefromparts(
concat('20', substring(secured, 5, 2)),
substring(secured, 1, 2),
substring(secured, 3, 2)
)
Then you can check it against a given interval:
datefromparts(
concat('20', substring(secured, 5, 2)),
substring(secured, 1, 2),
substring(secured, 3, 2)
) between dateadd(day -30, cast(getdate() as date)) and cast(getdate() as date)
If secured is a string, you can do a little manipulation and a simple cast():
select convert(date, concat('20', right(secured, 2), left(secured, 4))
You might find it convenient to actually create a computed column so this is always available:
alter table t add secured_date as (try_convert(date, concat('20', right(secured, 2), left(secured, 4))
You can even persist the column and create an index, so your queries are more efficient.
There're many ways to convert a 6 digits INT to DATE:
You can do a convertion using LTRIM and CAST like this:
SELECT CAST(RIGHT(LTRIM(123120),2) + LEFT(LTRIM(123120),4) AS DATE)
Using LTRIM with SUBSTRING:
SELECT CAST(SUBSTRING(LTRIM(040320), 5, 2) + SUBSTRING(LTRIM(040320), 1, 4) AS DATE)
Using CONVERT:
SELECT CAST((RIGHT(CONVERT(VARCHAR(6), 123120),2) +
LEFT(CONVERT(VARCHAR(6), 123120),4)) AS DATE)
Or using CONVERT with SUBSTRING:
SELECT CAST(SUBSTRING(CONVERT(VARCHAR(6), 040320), 5, 2) +
SUBSTRING(CONVERT(VARCHAR(6), 040320), 1, 4) AS DATE)
Using STR:
SELECT CAST(RIGHT(STR(123120, 6) ,2) + LEFT(STR(123120, 6) ,4) AS DATE)
Or using STR with SUBSTRING:
SELECT CAST(SUBSTRING(STR(040320, 6), 5, 2) + SUBSTRING(STR(040320, 6), 1, 4) AS DATE)
This casting will format the date as expected but your dates need be
in format mmddyy
Standard EUA (style 10) DATE format (mm-dd-yy) without century (yy)
Edit after comments:
What's the table name of this query?
What's the SECURED field datatype? INT? VARCHAR?
You need convert a date and compare between last 30 days and today?
The query below resolve your problem? (change TABLE_NAME to your table name)
SELECT LEFT(SECURED, 2) + '/' + replace(replace(SECURED, LEFT(SECURED, 2), ''),
RIGHT(SECURED, 2), '') + '/' + RIGHT(SECURED, 2) as 'Modified SECURED as Date'
FROM TABLE_NAME
WHERE CAST(RIGHT(LTRIM(SECURED), 2) + LEFT(LTRIM(SECURED), 4) AS DATE)
BETWEEN GETDATE() - 30 AND GETDATE()

Date Conversion of yymmddhhmm.yymmddhhmm in SQL

I'm getting this type of data in DateandTime Column in SQl.
1803301611.1803301611
Format is yymmddhhmm.yymmddhhmm and I have to convert this data in a date format.
The DateandTime Column date type is Varchar(5)
I don't think the format of your time string can be directly converted by SQL Server. But, we can try to piece together into a format which can be. This is the closest I could come, and it required prepending a 20 in front of the year to make a full 4 digit century:
SELECT
col AS input,
CONVERT(datetime, '20' + SUBSTRING(col, 1, 2) + '-' + SUBSTRING(col, 3, 2) + '-' +
SUBSTRING(col, 5, 2) + ' ' + SUBSTRING(col, 7, 2) + ':' +
SUBSTRING(col, 9, 2), 120) AS output
FROM yourTable;
Demo
We ideally should have been able to use mask 20 without the century, but I could not get it to work.
You can do :
select *, cast(concat(cast(left(col, 6) as date), ' ',stuff(right(col, 4), 3, 0, ':')
) as datetime
) as newdatetime
from table t;

Update the first and last 2 charactors with year and month

I have a value that I need to change the first characters with the year and the last 2 characters with the month. The AttributeValue field value is 17WEBD01. I need to update the 17 with the current years last two digits and I need to update the 01 with the current month of 02.
I tried this statement and I get an error on the Left statement
UPDATE [EC_StoreAttributes]
SET Left(AttributeValue, 2) = SELECT RIGHT(CONVERT(VARCHAR(10),GETDATE(),101),2),
Right(AttributeValue, 2) = SELECT LEFT(CONVERT(VARCHAR(10),GETDATE(),101),2)
WHERE AttributeType = 'ECSRCCODE' and StoreRecordId = '1' 
update one column in one go by Casting and combining strings. They are integer outputs, so you have to cast them to varchar (2). First part of string replacing year (first 2 values) and replace used again to replace right part of string
UPDATE [EC_StoreAttributes]
SET AttributeValue = REPLACE(CAST((
SELECT RIGHT(CONVERT(VARCHAR(10), GETDATE(), 101), 2)
) AS VARCHAR(2)) + REPLACE(AttributeValue, left(AttributeValue, 2), ''), RIGHT(CAST((
SELECT RIGHT(CONVERT(VARCHAR(10), GETDATE(), 101), 2)
) AS VARCHAR(2)) + REPLACE(AttributeValue, left(AttributeValue, 2), ''), 2), '') + REPLACE(RIGHT(CAST((
SELECT RIGHT(CONVERT(VARCHAR(10), GETDATE(), 101), 2)
) AS VARCHAR(2)) + REPLACE(AttributeValue, left(AttributeValue, 2), ''), 2), RIGHT(attributevalue, 2), CAST((
SELECT LEFT(CONVERT(VARCHAR(10), GETDATE(), 101), 2)
) AS VARCHAR(2)))
WHERE AttributeType = 'ECSRCCODE'
AND StoreRecordId = '1'

SQL Server : format 700 as time

I've run an import which has updated many records in my tblRota.StartTime and tblRota.EndTime in the format 900 and 1700.
How can I reformat these to 09:00 and 17:00?
The datatypes of both columns is varchar.
Thank you.
You could use this query:
select stuff(right('0' + replace([StartTime], ':', ''), 4), 3, 0, ':'),
stuff(right('0' + replace([EndTime], ':', ''), 4), 3, 0, ':')
from [tblRota]
The steps are:
Remove the :: replace([StartTime], ':', '')
Get the time on 4 digits: right('0' + <3Or4DigitTime>, 4)
Insert the :: stuff(<4DigitTime>, 3, 0, ':')
Use some string manipulation:
UPDATE tblRota
SET StartTime = LEFT(RIGHT('0'+StartTime , 4),2)+':'+RIGHT(StartTime ,2),
EndTime = LEFT(RIGHT('0'+EndTime , 4),2)+':'+RIGHT(EndTime ,2)