I am trying to partition the data from a Snowflake table and storing it as a parquet file in a S3 bucket. This partition allows me to store the data organised by date (Bucket:///).
I am performing this, using the below Snowflake procedure:
However, this creates the folders structure in the following way:
And I wanted it to be: year=2022/month=05/day=01/
To accomplish this I changed month and day variables to:
However, it produced the same result that I was having before the change.
I even tried to assign some values to both 'day' and 'month' vars like:
var day = '01'
var month = '05'
But it is still removing the leading zeros from the month and day.
Do you know how can I solve this?
Thanks for your help.
You just need to add some additional lines of code to prefix the values with zero when required e.g. (pseudo-code)
day = if day < 10 then "0"||day else day;
month = if month < 10 then "0"||month else month;
You can use a Snowflake SQL to get the correct date format in MM and DD. Here is the SQL
SELECT TO_CHAR(CURRENT_DATE(), 'MM') MON, TO_CHAR(CURRENT_DATE(), 'DD');
and the stored proc to get the values
create or replace procedure date_test()
returns string
language javascript
as
$$
var day;
var mon;
// Dynamically compose the SQL statement to execute.
var sql_command = "SELECT TO_CHAR(CURRENT_DATE(), 'MM') MON, TO_CHAR(CURRENT_DATE(), 'DD');"
// Run the statement.
var stmt = snowflake.createStatement(
{
sqlText: sql_command
}
);
var res = stmt.execute();
res.next();
mon = res.getColumnValue(1);
day = res.getColumnValue(2);
return "day:" + day + " month:" + mon;
$$
;
call date_test();
output:
DATE_TEST
day:13 month:05
Related
This is actually something I want to type into query builder in SAS EG. I am trying to filter down a data set that has dates like this:
SEP2021
AUG2021
JUL2021
etc…
I’m trying to use query builder to filter it with a WHERE statement. I want the table to have only the results dated for last month. So running it now should give SEP2021, and running it next month would give OCT2021, etc…
How can I do this with a WHERE statement?
Use intnx(). Assuming that all the dates start on the first of the month:
where date = intnx('month', today(), -1, 'B');
If they don't:
where intnx('month', date, 0, 'B') = intnx('month', today(), -1, 'B');
You can use the INTCK function to compute the number calendaring intervals between two dates.
Example:
data have;
input datestring $; datalines;
SEP2021
AUG2021
JUL2021
run;
proc sql;
create table want as
select * from have
where intck('month',input(datestring,monyy7.),today()) = 1
;
Here is the answer from KurtBremser in SAS communities, this worked for me! I did not realize what format I had converted the character value into. PROC CONTENTS revealed my values were Numeric, format DTDATE9.
Full thread here: https://communities.sas.com/t5/SAS-Enterprise-Guide/How-can-PROC-SQL-return-only-results-from-the-previous-month/m-p/774523
Solution below:
“
So it IS a datetime value, not a date. You need to use DATEPART to extract the date from it:
data have;
input date_column datetime19.;
format date_column dtdate9.;
datalines;
01sep2021:01:02:03
01oct2021:04:05:06
;
proc contents data=have;
run;
proc sql;
select *
from have
where datepart(date_column) = intnx('month',today(),-1,'b');
quit;
Partial result:
# Variable Typ Länge Ausg.Format
1 date_column Num 8 DTDATE9.
date_column
01SEP2021
“
I have written below query to get the records:
Select *
from Scf_Invoice i
where cast(i.inv_Acceptance_Date as date) = TO_DATE('2018-12-18', 'YYYY-MM-DD');
But, I get no resultseven if there are 2 records for the given date. I think it's considering time also. When I do
Select *
from Scf_Invoice i
where cast(i.inv_Acceptance_Date as date) > TO_DATE('2018-12-18', 'YYYY-MM-DD');
I get the results, But I want those results using ' = ' parameter. What change should I do in my query?
Use trunc to get day
trunc(i.inv_Acceptance_Date) =
The TRUNC (date) function returns date with the time portion of the day truncated to the unit specified by the format model fmt. The value returned is always of datatype DATE, even if you specify a different datetime datatype for date. If you omit fmt, then date is truncated to the nearest day.
Select *
from Scf_Invoice i
where trunc(i.inv_Acceptance_Date) = TO_DATE('2018-12-18', 'YYYY-MM-DD');
Select *
from Scf_Invoice i
where trunc(i.inv_Acceptance_Date) = TO_DATE('2018-12-18', 'YYYY-MM-DD');
or
Select *
from Scf_Invoice i
where trunc(i.inv_Acceptance_Date) = trunc(sysdate);
for get data equal curent date
I am trying to create a datetime variable for the past 3 hours... by concatenating DATE variable (in DATE format) and time variable (string hh:mm:ss) within PROC SQL;
Would highly appreciate any help with this!
Example:
APPLCTN_DT = 05NOV2018:00:00:00.000
APPLCTN_TM = 20:04:57
I would like to create a numeric DATETIME field based on the above
Since it looks like your "date" variable is really a DATETIME variable with zero time you perhaps can just add the time part to it?
new_datetime = APPLCTN_DT + input(APPLCTN_TM,time8.);
Or just to be safe you could force the time part of your datetime value to be zero before adding the time part. Here are a couple of ways.
new_datetime = dhms(datepart(APPLCTN_DT),0,0,input(APPLCTN_TM,time8.));
new_datetime = intnx('dtdate',APPLCTN_DT,0) + input(APPLCTN_TM,time8.);
Presuming the _DT variable is actually a datetime value with no exact time portion (thus just the date)).
Use DATEPART to extract SAS date value, INPUT to convert time string to time value, DHMS to construct a target date time value and INTNX to compute a new date time value offset from the target.
data _null_;
APPLCTN_DT = '05NOV2018:00:00:00.000'dt ;
APPLCTN_TM = "20:04:57";
date_part = datepart(applctn_dt);
time_part = input(applctn_tm,time8.);
target_dt = dhms(date_part,0,0,0) + time_part;
target_minus_3hr_dt = intnx ('dthour'
, dhms(date_part,0,0,0) + time_part
, -3
);
target_minus_3hr_exact_dt = intnx ('dtsecond'
, dhms(date_part,0,0,0) + time_part
, -3*60*60
);
format target: datetime20.;
put target_dt ' combined';
put target_minus_3hr_dt ' combined, 3 hours ago';
put target_minus_3hr_exact_dt ' combine, exactly three hours ago (to the second)';
run;
Will show in log
05NOV2018:20:04:57 combined
05NOV2018:17:00:00 combined, 3 hours ago
05NOV2018:17:04:57 combine, exactly three hours ago (to the second)
Actually, your code is close to the result, i think you forgot to transfer SAS date format to display format.
using put function in variable target_minus_3hr_dt & target_minus_3hr_exact_dt,
since the SAS date is shown as numeric so we need to use put function to transfer.
data _null_;
APPLCTN_DT = '05NOV2018:00:00:00.000'dt ;
APPLCTN_TM = "20:04:57";
date_part = datepart(applctn_dt);
time_part = input(applctn_tm,time8.);
target_dt = put(date_part,date9.)||applctn_tm;
target_minus_3hr_dt = put((intnx ('dthour'
, dhms(date_part,0,0,0) + time_part
, -3
)),datetime20.);
target_minus_3hr_exact_dt = put((intnx ('dtsecond'
, dhms(date_part,0,0,0) + time_part
, -3*60*60
)),datetime20.);
put target_dt ' combined';
put target_minus_3hr_dt ' combined, 3 hours ago';
put target_minus_3hr_exact_dt ' combine, exactly three hours ago (to the second)';
run;
I am trying to find a data with specific where clause of date and month but I am receiving an error can anyone help me with this?
select *
from my_data
where date BETWEEN '11-20' AND '12-15'
MS SQL Server Management Studio
I am receving an error
Conversion failed when converting date and/or time from character string
Most databases support functions to extract components of dates. So, one way of doing what you want is to convert the values to numbers and make a comparison like this:
where month(date) * 100 + day(date) between 1120 and 1215
The functions for extracting date parts differ by database, so your database might have somewhat different methods for doing this.
The conversion is failing because you are not specifying a year. If you were to specify '11-20-2015' your query would work just insert whatever year you need.
SELECT *
FROM my_data
WHERE date BETWEEN '11-20-2015' AND '12-15-2015'
Alternatively if you wanted data from that range of dates for multiple years I would use a while loop to insert information in a # table then read from that table, depending on the amount of data this could be quick or sloooowww here is an example.
DECLARE #mindatestart date, #mindateend date, #maxdatestart date
SET #mindatestart = '11-20-2010'
SET #mindateend = '12-15-2010'
SET #maxdatestart = '11-20-2015'
SELECT top 0 *, year = ' '
INTO #mydata
FROM my_data
WHILE #mindatestart < #maxdatestart
BEGIN
INSERT INTO #mydata
SELECT *, YEAR(#mindatestart)
FROM my_data
where date between #mindatestart and #mindateend
SET #mindatestart = DATEADD(Year, 1, #mindatestart)
SET #mindateend = DATEADD(Year, 1, #mindateend)
END
This will loop and insert the data from 2010-2015 for those date ranges and add a extra column on the end so you can call the data and order by year if you want like this
SELECT * FROM #mydata order by YEAR
Hopefully some part of this helps!
FROM THE COMMENT BELOW
SELECT *
FROM my_data
WHERE DAY(RIGHT(date, 5)) between DAY(11-20) and DAY(12-15)
The reason '11-20' doesn't work is because its a character string which is why you have to input it between ' ' What the Month() function does is take whatever you put between the () and convert it to an integer. Which is why you're not getting anything back using the method in the first answer, the '-Year' from the table date field is being added into the numeric value where your value is just being converted from 11-20 you can see by using these queries
SELECT MONTH(11-20) --Returns 12
SELECT MONTH(11-20-2015) -- Returns 6
SELECT MONTH(11-20-2014) -- Returns 6
Using RIGHT(Date, 5) you only get Month-day, then you date the day value of that so DAY(RIGHT(DATE, 5) and you should get something that in theory should fall within those date ranges despite the year. However I'm not sure how accurate the data will be, and its a lot of work just to not add an additional 8 characters in your original query.
Since you only care about month and day, but not year, you need to use DATEPART to split up the date. Try this:
select *
from my_data
WHERE 1=1
AND (DATEPART(m, date) >= 11 AND DATEPART(d,date) >= 20)
AND (DATEPART(m, date) <= 12 AND DATEPART(d,date) <= 15)
I have a table invoices with this fields:
invDate -> a date field
invTime -> a time field
I need to do querys like
SELECT top 10 * from invoices WHERE DATETIME(invDate+invTime)
BETWEEN DATETIME('2013-12-17 17:58') AND DATETIME()
or something like that. I don't know how to concatenate the invDate and invTime to create a datetime field. The only thing that i could do is this horribly thing:
DATETIME( YEAR(invDate), MONTH(invDate), DAY(invDate), 17, 52 ) AS MyDatetime
Couldn't even get hour and time with hour(invTime) and minute(invTime):
DATETIME( YEAR(invDate), MONTH(invDate), DAY(invDate),
HOUR(invTime), MINUTE(invTime) ) AS MyDatetime
I'm doing the querys throught the VFP Odbc Driver via PHP.
You were pretty close. If the value coming from PHP is not of a date/time, how could VFP interpret it properly. VFP also has a function CTOT() (character to time), and expects it in the format of 'yyyy-mm-ddThh:MM:ss??'
yyyy = 4 digit year
mm = 1 OR 2 digit month
dd = 1 OR 2 digit day
T -- literally the letter "T"
hh = 1 OR 2 digit hour (but typical is 2 anyhow)
MM = 1 or 2 digit minute (but typical is 2)
ss = 1 or 2 digit for seconds -- not required
?? = "AM" or "PM" if you wanted to explicitly provide that vs 24 hour clock
The MM and ss are optional, so if you finished with "T1" would be 1:00:00am
Now, to finish your query.
WHERE DATETIME(invDate+invTime)
BETWEEN DATETIME('2013-12-17 17:58') AND DATETIME()
Since this appears to be querying all invoices between a given date/time and NOW (via DateTime()), you don't even need between, you can do
WHERE YourTable.Column > CTOT( '2013-12-17T17:58')
If you specifically DID have a date/time range to consider, THEN you could do something like
WHERE YourTable.Column BETWEEN CTOT( '2013-12-05T10:00') AND CTOT( '2013-12-14T11:58')
PROBLEMS WITH your DATE() and TIME() implementations
The problem is Date() is a function to either return current date, or create based on y/m/d provided such as date( 2013, 12, 7 ). If you are passing a string, use CTOD( 'mm/dd/yyyy' ) such as CTOD( 12, 7, 2013 ).
As for the TIME() function that just expects a number and is of no use for you. From the OleDbProvider, your best bet is to just create a php function that builds a single string in the CTOT() format I've described and pass to the php function the date and time fields. Then use that as your "CTOT( functionReturnResult )"
To add a Date and a Time Field together you will need to convert them both to a same datatype 1st and than just simply add them together something like this....
DECLARE #D DATE = '2013-12-17'
DECLARE #T TIME = '17:58:00'
SELECT CAST(#D AS DATETIME) + CAST(#T AS DATETIME)
Result
2013-12-17 17:58:00.000
Your Query
SELECT top 10 *
from invoices
WHERE CAST(invDate AS DATETIME) + CAST(invTime AS DATETIME)
BETWEEN '20131217 17:58:00.000' AND GETDATE()