I have a SAS program I want to automate to run. I have to pull the first date & the last date of the month with the time stamps.
This is what it looks like hard coded - that does work! But I don't want to change it every month.
%let st_dt= '2020-10-01-00.00.00';
%let ed_dt= '2020-10-31-23.59.59';
PROC SQL;
SELECT t1.ENTRYTIME
FROM MYTABLE t1
WHERE t1.ENTRYTIME BETWEEN &st_dt. AND &ed_dt.
I have tried so many different things but cannot seem to get it to work. And BONUS, I'm pulling from transactional tables that have over 17 BILLION - yes BILLION WITH A B records. I've tried using datepart on the ENTRYTIME & casting as a date - but with that many records it just runs forever and eventually gets killed - or just flat out fails! :(
This is what I've tried so far - neither PROC SQL work.
%let start=%sysfunc(intnx(month, %sysfunc(today()),-1), date9.);
%let end=%sysfunc(intnx(month, %sysfunc(today()),-1,e), date9.);
PROC SQL;
SELECT t1.ENTRYTIME
FROM MYTABLE t1
WHERE cast(t1.ENTRYTIME as date) BETWEEN &start. AND &end.
PROC SQL;
SELECT t1.ENTRYTIME
FROM MYTABLE t1
WHERE cast(t1.ENTRYTIME as date) BETWEEN &start. AND &end.
So if you can get me a macro or a let statement or anything that is formatted like this to automatically update without human intervention: '2020-10-01-00.00.00' & '2020-10-31-23.59.59' - BRING IT ON! Thank you for your help!!
Just tack on the time suffix as constant text.
%let st_dt= "%sysfunc(intnx(month,%sysfunc(today()),-1,b),yymmddd10.)-00.00.00";
%let ed_dt= "%sysfunc(intnx(month,%sysfunc(today()),-1,e),yymmddd10.)-23.59.59";
You probably want to pass through to ensure best performance. The core syntax in the pass through is
entrytime >= 'yyyymmdd' and entrytime < 'yyyymmdd'
^^^^^^^^ ^^^^^^^^
1st of month 2nd of month
or
entrytime >= 'yyyymmdd' and entrytime < 'yyyymmdd'
^^^^^^^^ ^^^^^^^^
last of month 1st of next month
Example:
%let today = %sysfunc(date());
%let month_first_dt = %sysfunc(intnx(MONTH,&today,0));
%let month_last_dt = %sysfunc(intnx(MONTH,&today,0,E));
%let month_first = %sysfunc(intnx(DAY,&month_first_dt,0),yymmddn8.);
%let month_first_next = %sysfunc(intnx(DAY,&month_first_dt,1),yymmddn8.);
%let month_last = %sysfunc(intnx(DAY,&month_last_dt,0),yymmddn8.);
%let month_last_next = %sysfunc(intnx(DAY,&month_last_dt,1),yymmddn8.);
%let ss_MF_literal = %sysfunc(quote(&month_first,%str(%')));
%let ss_MFN_literal = %sysfunc(quote(&month_first_next,%str(%')));
%let ss_ML_literal = %sysfunc(quote(&month_last,%str(%')));
%let ss_MLN_literal = %sysfunc(quote(&month_last_next,%str(%')));
libname remote sqlsrv 'connection string';
proc sql;
create table sas_table as select * from connection to remote
(
select entrytime from remote-table-name
where
entrytime >= &ss_MF_literal and entrytime < &ss_MFN_literal
or
entrytime >= &ss_ML_literal and entrytime < &ss_MLN_literal
)
;
Related
This is actually something I want to type into query builder in SAS EG. I am trying to filter down a data set that has dates like this:
SEP2021
AUG2021
JUL2021
etc…
I’m trying to use query builder to filter it with a WHERE statement. I want the table to have only the results dated for last month. So running it now should give SEP2021, and running it next month would give OCT2021, etc…
How can I do this with a WHERE statement?
Use intnx(). Assuming that all the dates start on the first of the month:
where date = intnx('month', today(), -1, 'B');
If they don't:
where intnx('month', date, 0, 'B') = intnx('month', today(), -1, 'B');
You can use the INTCK function to compute the number calendaring intervals between two dates.
Example:
data have;
input datestring $; datalines;
SEP2021
AUG2021
JUL2021
run;
proc sql;
create table want as
select * from have
where intck('month',input(datestring,monyy7.),today()) = 1
;
Here is the answer from KurtBremser in SAS communities, this worked for me! I did not realize what format I had converted the character value into. PROC CONTENTS revealed my values were Numeric, format DTDATE9.
Full thread here: https://communities.sas.com/t5/SAS-Enterprise-Guide/How-can-PROC-SQL-return-only-results-from-the-previous-month/m-p/774523
Solution below:
“
So it IS a datetime value, not a date. You need to use DATEPART to extract the date from it:
data have;
input date_column datetime19.;
format date_column dtdate9.;
datalines;
01sep2021:01:02:03
01oct2021:04:05:06
;
proc contents data=have;
run;
proc sql;
select *
from have
where datepart(date_column) = intnx('month',today(),-1,'b');
quit;
Partial result:
# Variable Typ Länge Ausg.Format
1 date_column Num 8 DTDATE9.
date_column
01SEP2021
“
I am trying to get the current month records from a table. Since this is going to be an automated monthly job, I can't give explicit values in where clause. So I am using the intnx function to get the month's start and end date. Then comparing them with the table value (date) in where clause. I tried different date formats but every time I am getting the same error: "ERROR: Expression using equals (=) has components that are of different data types."
Below is my script. What am I doing wrong?
DATA _NULL_;
CALL
SYMPUT('MonthStart',TRIM(PUT(INTNX('MONTH',TODAY(),0,'B'),ddmmyys10.)));
CALL
SYMPUT('MonthEnd',TRIM(PUT(INTNX('MONTH',TODAY(),0,'E'),ddmmyys10.)));
RUN;
%PUT &MonthStart.;
%PUT &MonthEnd.;
PROC SQL;
CREATE TABLE CURRENT_MONTH_ACTIVITY AS
SELECT * FROM MONTH_END_BASE
WHERE ACTIVITY_DATE >= '&MonthStart.' AND ACTIVITY_DATE <= '&MonthEnd.' ;
QUIT;
Thanks in advance!
You can use the INTCK function to check for when the month range from the current date to the activity date is zero.
where INTCK('MONTH', today(), activity_date) = 0;
An alternative would be to use macro to 'code-gen' directly the between points. You don't need a data step to populate a macro variable that is later used.
Example:
data have;
do activity_date = today() - 365 to today() + 365;
output;
end;
format activity_date yymmdd10.;
run;
data want;
set have;
where intck('MONTH', today(), activity_date) = 0;
run;
data want;
set have;
where activity_date
between
%sysfunc(INTNX(MONTH, %sysfunc(today()), 0, B))
and
%sysfunc(INTNX(MONTH, %sysfunc(today()), 0, E))
;
run;
I agree with Tom. Probably the variable activate_date is a numeric type with date format, then the solution is assign to MonthStart and MonthEnd a numeric value.
DATA _NULL_;
CALL SYMPUT('MonthStart',INTNX('MONTH',TODAY(),0,'B'));
CALL SYMPUT('MonthEnd',INTNX('MONTH',TODAY(),0,'E'));
RUN;
%PUT &MonthStart.;
%PUT &MonthEnd.;
PROC SQL;
CREATE TABLE CURRENT_MONTH_ACTIVITY AS
SELECT * FROM MONTH_END_BASE
WHERE ACTIVITY_DATE >= &MonthStart. AND ACTIVITY_DATE <= &MonthEnd. ;
QUIT;
I have a table named case_DataTable_d in which column name value_dt have different date values. I want get number of day difference between that date and date of today.
This is my code
proc sql noprint;
create table daystoOverdue_list as
select distinct business_object_rk , DateDiff(DAY, value_dt, Today()) as value_dt
from case_DataTable_d as tbl
where tbl.cust_field_nm eq "x_case_dte_dd"
and datepart(tbl.value_dt) < today();
quit;
I'm having errors that
Day is not any column name
function DateDiff could not be located.
DateDiff is not a valid SAS function. Try intck:
%let today=%sysfunc(date());
proc sql noprint;
create table daystoOverdue_list as
select distinct business_object_rk
, intck('DAY', datepart(tbl.value_dt), &today) as value_dt
from case_DataTable_d as tbl
where tbl.cust_field_nm eq "x_case_dte_dd"
and datepart(tbl.value_dt) < &today;
This function returns the number of interval boundaries of a given kind that lie between two dates, times, or datetime values (see documentation).
i connected using
proc sql;
connect to odbc as odbc("......");
create table work.market as select distinct * from connection to odbc(
select distinct C.Product#, A.county, B.DT, profit2, Rev2)
From Mtable.duv A, Ttable.duv B, otable C
Where B.Product# = C.Product#
and B.Product# = A.Product#
and B.Dt = C.Dt
and B.dt between A.dt_start and dt_end
and B.dt between 20140331 and 20170630
);
disconnect from odbc;
quit;
data work.smallmarket;
set work.market;
where country=Nigeria;
NetMargin=profit2/Rev2;
keep Product# NetMargin DT;
run;
1) if DT is my date, how do I change the date format from YYYYMMDD to a SAS date format like 01Jan1960? When I run the above I get my data but the dates come out as 20170630 for example. How can i convert by date column to show in the format 30Jun2017. I posted how I got my initial data set "work.market" just in case that is part of the issue. Sorry can't post the log. Can you please help?
If you want to convert to a SAS date format, you have two places to do that.
First, you can do it in the create table statement; that is a SAS statement, not a SQL Server/whatever statement.
proc sql;
connect to odbc as odbc("......");
create table work.market as select Product#, county, dt format=date9.,profit2, rev2
from connection to odbc(
select distinct Product#, county, DT, profit2, Rev2);
The second place you can do it is in the data step following.
data work.smallmarket;
set work.market;
where country=Nigeria;
format dt date9.;
NetMargin=profit2/Rev2;
keep Product# NetMargin DT;
run;
I am pulling a large amount of data from an SQL Server through SAS. I would like to pull one minute (or hour) of data at a time, using a loop.
The format of the timestamp is 'yyyymmdd hh:mm:ss.000'.
Usually, I would do something like:
%macro Loop(num_days, tmstmp_begin):
%do i = 0 to &num_days.;
proc sql;
...
where tmstmp between &tmstmp_begin + &i minutes and &tmstmp_begin (&i+1) minutes;
quit;
%end
%mend;
But the minute addition is not supported on the server. In Teradata, I can use:
DATEADD(minute, 1, tmsmtmp)
to add a minute to the timestamp, but this will not execute in SAS (it doesn't pass through to the server?):
ERROR: Function DATEADD could not be located.
Anyway I was wondering if there is a neat %sysfunc solution that would help me avoid generating a SAS table of timestamps from which I would read into macro variables, or something else just as silly.
Basically I need to go from:
%let i = 1;
%let tmstmp = '20150801 00:00:00.000'
to:
%put ...something involving tmstmp and i...;
'20150801 00:01:00.000'
Thanks!
The best way I've found to handle these scenarios is to use a custom datetime format. You can find a link on building them here. I recommend saving the format to a common library so it's always available to your SAS sessions. The format will be:
proc format ;
picture mssqldt low-high = '''%Y-%0m-%0d %0H:%0M:%0S.000''' (datatype = datetime) ;
run ;
This will take a regular SAS datetime stamp and format it like so (including quotes):
'2015-09-21 15:04:16.000'
The best way to incorporate this into your SAS code is to always keep your dates and date times in their SAS representation, and have separate variables for your SQL server formatted variables. E.g.
Calculate the datetime we want to work with:
%let my_datetime = %sysfunc(datetime());
Create two new variables with the SQL server formatted datetime stamps. I always call mine &sql_start and &sql_end so that they read nicely and I never have to think about it...
%let sql_start = %sysfunc(sum(&my_datetime),mssqldt.);
%let sql_end = %sysfunc(intnx(minute,&my_datetime,1),mssqldt.);
You can see that to calculate sql_start I used the sum() function within %sysfunc() and passed in the SAS datetime variable. I do it this way because it doesn't change the value of the datetime, and allows me to make use of the second parameter of %sysfunc() which applies the specified format to the value being returned.
For sql_end I used the intnx() function as per usual, and again made use of the second %sysfunc() parameter to format it.
Let's print out the values to see how they look:
%put &sql_start &sql_end;
Gives:
'2015-09-21 15:04:16.000' '2015-09-21 15:05:00'
Then it's simply a case of using it in your code like so:
proc sql;
...
where tmstmp between &sql_start and &sql_end;
quit;
Here's all the code in one spot (assuming you have already defined the format):
%let my_datetime = %sysfunc(datetime());
%let sql_start = %sysfunc(sum(&my_datetime),mssqldt.);
%let sql_end = %sysfunc(intnx(minute,&my_datetime,1),mysqldt.);
%put &sql_start &sql_end;
proc sql;
...
where tmstmp between &sql_start and &sql_end;
quit;
Now if you want to pull the data one piece at a time, you can compile it all into a loop like this:
%macro get_data(iStart=,iEnd=);
%local tmp_start tmp_end sql_start sql_end;
%let tmp_start = &iStart;
%do %while(&tmp_start le &iEnd);
%let tmp_end = %sysfunc(intnx(hour,&tmp_start,0,end));
/* MAKE SURE END OF LOOP ISNT GREATER THAN END DATETIME */
%if &tmp_end > &iEnd %then %do;
%let tmp_end = &iEnd;
%end;
%let sql_start = %sysfunc(sum(&tmp_start),mssqldt.);
%let sql_end = %sysfunc(sum(&tmp_end ),mssqldt.);
/* DO SQL HERE */
%put &sql_start &sql_end;
/* INCREMENT THE LOOP */
%let tmp_start = %sysfunc(intnx(hour,&tmp_start,1,beginning));
%end;
%mend;
Call it for today through to some time tomorrow:
%get_data(iStart=%sysfunc(datetime()),
iEnd =%sysfunc(dhms(%sysfunc(date())+1,2,30,13))
);
The resulting runs are for the following periods:
'2015-09-21 15:25:33.000' '2015-09-21 15:59:59.000'
'2015-09-21 16:00:00.000' '2015-09-21 16:59:59.000'
'2015-09-21 17:00:00.000' '2015-09-21 17:59:59.000'
'2015-09-21 18:00:00.000' '2015-09-21 18:59:59.000'
'2015-09-21 19:00:00.000' '2015-09-21 19:59:59.000'
'2015-09-21 20:00:00.000' '2015-09-21 20:59:59.000'
'2015-09-21 21:00:00.000' '2015-09-21 21:59:59.000'
'2015-09-21 22:00:00.000' '2015-09-21 22:59:59.000'
'2015-09-21 23:00:00.000' '2015-09-21 23:59:59.000'
'2015-09-22 00:00:00.000' '2015-09-22 00:59:59.000'
'2015-09-22 01:00:00.000' '2015-09-22 01:59:59.000'
'2015-09-22 02:00:00.000' '2015-09-22 02:30:13.000'
SAS stores datetimes as number of seconds, so rather than add one minute you can try adding &i minutes*60seconds/minute to get a valid interval.
where tmstmp between "&tmstmp_begin"dt + &i*60 and "&tmstmp_begin"dt + (&i+1)*60;
EDIT: This won't work if you have a character variable which it appears you do, but only if you're using an actual SAS datetime value.