I have a macro where I am currently passing in 6 table names and 6 columns. However, the number of columns and tables will not always be constant.
Is there a way to have a variable number of parameters? I am familiar with the concept in python with **kwargs.
Also, is there a way to parameterize the proc sql statement to only take as many col and table inputs as provided? Or do a try catch of some sort in SAS to check if the variables exist before running the sql statement?
Here is my macro I'm trying to parameterize.
%macro Generate_TP_tbl(new_tbl_name, trans_col, tbl_1, tbl_2, tbl_3, tbl_4,
tbl_5, tbl_6, col_1, col_2, col_3, col_4, col_5, col_6);
proc sql;
CREATE TABLE &new_tbl_name AS
SELECT a1._NAME_, a1.&trans_col as &col_1, a2.&trans_col as &col_2,
a3.&trans_col as &col_3, a4.&trans_col as &col_4, a5.&trans_col as &col_5,
a6.&trans_col as &col_6
FROM &tbl_1 as a1, &tbl_2 as a2, &tbl_3 as a3, &tbl_4 as a4, &tbl_5 as a5,
&tbl_6 as a6
WHERE a1._NAME_ = a2._NAME_ = a3._NAME_ = a4._NAME_ = a5._NAME_ = a6._NAME_;
run;
%mend Generate_TP_table;
An even more generic way of doing this is as follows:
%macro mymacro /parmbuff;
%put &SYSPBUFF;
%mend;
You can then call %mymacro with any parameters you like and parse them all out from the &SYSPBUFF automatic macro variable.
This would probably need more work than Reeza's solution would, but I thought I'd post this anyway for completeness, as it's occasionally useful.
Pass them in as a single parameter and have the macro parse them out later.
%macro (parameters = , table_list = tb1 tb2 tb3 ... tb6, col_list=col1 col2 ... col6, other_parms= ... );
I would recommend building the rest of your code using a do loop with the number of parameters. The documentation here has a somewhat bad example of how to extract each element of a list:
http://support.sas.com/documentation/cdl/en/mcrolref/67912/HTML/default/viewer.htm#p1n2i0ewaj1zian1ria5579z1zjh.htm
The SQL is ugly...I wonder if a data step would be easier since you're merging on a single variable? Then it really becomes a rename from each table as in the example above in many respects.
Related
How to write SAS dates to Microsoft SQL Server 2016 Date data type in database?
I got SAS data with a sas date DataEndDay and I want to write that into a database. The following bit is in use (buffer is just to speed up the testing-failing) :
libname valu oledb provider=sqloledb schema="dbo" INSERTBUFF=100
properties=("User ID"="&username." Password="&pw."
"data source" = &database.
"initial catalog"=&catalog.);
proc sql noprint;
insert into valu.Data_upload_from_me
( <some_columns...>,
<more-columns...>
,DataEndDay
)
select
<some_columns_source...>,
<more-columns_source...>
,DataEndDay
from work.SAS_data_to_publish
;quit;
Of course because SAS dates are numbers, direct writing is going to fail. What works is if I hard-code this as:
select
<some_columns_source...>,
<more-columns_source...>
,'2018-12-12'
from work.SAS_data_to_publish
;quit;
But If I convert the SAS date to string in SAS datasteps:
data SAS_data_to_publish ;
set SAS_data_to_publish ;
dataEndday0 = put(DataEndDay, yymmddd10.);
DataEndDay1 = quote(dataEndday0, "'") ;
run;
and try to write either of these, I get conversion error:
ERROR: ICommand::Execute failed. : Conversion failed when converting date and/or time from character string.
When I select the string it looks pretty ok:
proc sql; select DataEndDay1 from SAS_data_to_publish; quit;
'2018-12-12'
previously I've managed to write dateTimes with similar trick, which works:
proc format;
picture sjm
. = .
other='%Y-%0m-%0d %0H:%0M:%0S:000' (datatype=datetime)
;run;
data to_be_written;
set save.raw_data_to_be_written;
DataEndDay0 = put(dhms(DataEndDay,0,0,0), sjm. -L);
run;
Anyone ran into similar issues? How could I write the dates?
I could ask them to change the column to dateTime, maybe....
Thank you in advance.
Edit:
I managed to develop a work-around, which works but is ugly and -frankly- I don't like it. It so happens that my date is same for all rows, so I can assing it to macro variable and then use it in database writing.
data _NULL_;
set SAS_data_to_publish;
call symput('foobar', quote( put (DataEndDay , yymmddd10. -L), "'") ) ;
run;
....
select
<some_columns_source...>,
<more-columns_source...>
,&foobar.
from work.SAS_data_to_publish
;quit;
Of course this would fail immediately should DataEndDay vary, but maybe demonstrates that something is off in Proc SQLs select clause....
Edit Edit Pasted the question to SAS forums
I finally managed to crack the issue. The issue was for the missing values. As I am passing the values as strings into the database the parser interpreted missing values as real dots instead of empty strings. The following works:
data upload;
set upload;
CreatedReportdate2 = PUT(CreatedReportdate , yymmddn8.);
run;
libname uplad_db odbc noprompt =
"DRIVER=SQL Server; server=&server.; Uid=&user.;Pwd=&pw.; DATABASE=&db.;"
INSERTBUFF=32767;
proc sql;
insert into uplad_db.upload_table
(.... )
select
case when CreatedReportdate2 ='.' then '' else CreatedReportdate2 end,
...
from upload;
quit;
SAS does not really properly support the SQL server DATE data type. I imagine this is due to the fact that it's newer, but for whatever reason you have to pass the data as strings.
For missing values, it's important to have a blank string, not a . character. The easiest workaround here is to set:
options missing=' ';
That will allow you to insert data properly. You can then return it to . if you wish. In a production application that might be used by others, I'd consider storing aside the option value temporarily then resetting to that, in order to do no harm.
Normally I just use PROC APPEND to insert observations into a remote database.
proc append base=valu.Data_upload_from_me force
data=work.SAS_data_to_publish
;
run;
Make sure your date variable in your SAS dataset use the same data type as the corresponding variable names in your target database table. So if your MS SQL database uses TIMESTAMP fields for date values then make sure your SAS dataset uses DATETIME values.
If you want to use constants then make sure to use SAS syntax in your SAS code and MS SQL syntax in any pass through code.
data test;
date = '01JAN2017'd ;
datetime = '01JAN2017:00:00'dt ;
run;
proc sql ;
connect to oledb .... ;
execute ( ... date = '2017-01-01' .... datetime='2017-01-01 00:00' ...)
by oledb;
quit;
I have a code such as below in sql(lot more and and not ins but just wanted to list few) i am new to sas and know proc sql a bit etc, learning and exploring everyday,
Select * from table
Where date=‘20180112’
and type=‘apple’ and location=‘dc’ and not
(columnName)in(‘a’,’b’) And lat=‘ten’
I am not able to understand sas equivalent of above sql as below. Can someone please explain sas code of if part and then do
Data sample;
Set sourcetble;
If date=‘20180112’ and type=‘apple’
And location=‘dc’ then do;
Blah1=‘rain’
Blah2=‘something else’
If columnName in(‘a’, ‘b’) and lat=‘ten’ Then do;
This just subsets based the values and variables in the WHERE statement.
Data sample;
set table;
WHERE date='20180112' and type='apple' And location='dc'
and columnName in (‘a’, ‘b’) and lat=‘ten’;
<other optional code>;
run;
Not like SQL query, a SAS data step will result in creating a new dataset. If you don't need to have a new dataset, you can use "data _null_;". Alternatively there are SAS procedures that will simply display dataset such as SQL "select" would do.
The "set" in SAS is equivalent to the "from" in SQL: it specifies the base dataset(s) from which you build the new dataset.
By default, SAS data step keeps all variables of the "set" datasets. It is equivalent to "select *" in SQL. If you need only some variables, you can use "keep" and "drop" statements in SAS.
The "where" clause and "and"/"or" operators work similarly in SAS and SQL, but with slightly different syntax.
The if … then in the data step has no correspondce to the SQL shown in the question. A conditional assignment in SQL is done using a case statement.
So a DATA step statement such as
data want;
set have;
…
if date="20180112" and type="apple" and location="dc" then do;
Blah1="rain";
Blah2="something else";
end;
would be concordant with SQL
Proc SQL;
create table want as
select …
, case when date="20180112" and type="apple" and location="dc"
then "rain"
else ""
end as Blah1
, case when date="20180112" and type="apple" and location="dc"
then "something else"
else ""
end as Blah2
from
have
…
;
For the case of some algorithm needing to assign several variables at once when some criteria (if logic) is met:
DATA Step has do; … end; syntax which can have several assignments statements within.
SQL select statement can only assign one variable per logic evaluation (case statement), thus the logic code has to be repeated for each variable being assigned based on criteria.
Is it possible to make a macro of this form work?
%macro tableMath(input1,input2);
%local result;
proc sql; ---some code here using inputs--- quit;
proc sql; ---more code here--- quit;
proc sql;
select something into: result
quit;
&result
%mend;
I want to run some fairly complicated logic on each observation of a dataset, and in any other language I've used before the way to do this would be to encapsulate it in a function that returns a result each time it's called--I'm not sure how to do this logic in SAS however.
EDIT: input1 and input2 would be columns of a dataset and result would be used to create a new column in some other macro in another part of the program. I don't need a specific code solution I just literally don't get how you're supposed to do traditional function logic where you need a return value in SAS...
As Richard wrote, function-style macros emit SAS code. The general rule of developing function-style macros is that they contain only macro language statements. Any SAS code they contain will be emitted. Historically, this made it difficult/annoying to write a function-style macro that would process data like you would with a DATA step. Luckily, SAS has added a function, DOSUBL, which makes it easier to write function-style macros that execute SAS code in a "side session" and emit a result. See Rick Langston's paper.
Here is an example of a function-style macro which used DOSUBL to count the number of records in a table, and emits the count. (This is a very inefficient way to get a record count, just an example of doing something in SQL).
%macro SQLcount(table);
%local rc emit;
%let rc=%sysfunc(dosubl(%nrstr(
proc sql noprint;
select count(*) into :emit trimmed
from &table
quit;
)));
&emit
%mend ;
It can be used like:
proc sql ;
select name
,%SQLcount(sashelp.shoes) as ShoeCount /*emits 395*/
from sashelp.class
;
quit ;
When the above step runs, it will return 19 rows of names from sashelp.class, and the value of ShoeCount will be 395 on every row. Note that the macro SQLcount only executed once. While the PROC SQL step is being compiled/interpreted the call to SQLcount is seen and the macro is executed and emits 395. The step becomes:
proc sql ;
select name
,395 as ShoeCount /*emits 395*/
from sashelp.class
;
quit ;
DOSUBL uses a "side session" to execute code, which allows you to execute a PROC SQL step in the side session while the main session is interpreting a PROC SQL step.
I can't tell from your question if that sort of use case is what you want. It's possible you want a function-style macro where you could pass values to it from a table, and have the macro execute on each value and return something. Suppose you had a table which was a list of table names, and wanted to use SQL to get the count of records in each table:
data mytables ;
input table $20. ;
cards ;
sashelp.shoes
sashelp.class
sashelp.prdsale
;
quit ;
You can do that by using the resolve() function to build macro calls from data, delaying the execution of the macro until the SELECT statement executes:
proc sql ;
select table
,resolve('%SQLcount('||table||')') as count
from mytables
;
quit ;
With that, SQLcount will be called three times, and will return the number of records in each dataset.
table count
---------------------------
sashelp.shoes 395
sashelp.class 19
sashelp.prdsale 1440
The macro call is not seen when the PROC SQL step is interpreted, because it is hidden by the single quotes. The resolve function then calls the macro when the SELECT statement executes, passing the value of table as a parameter value, and the macro emits the record count. This is similar to a CALL EXECUTE approach for using data to drive macro calls.
You state you want to:
run some fairly complicated logic on each observation of a dataset
To do that you should use the SAS language instead of the macro processor or PROC SQL. You can use a data step. Or for even more complicated logic you should look at PROC DS2.
Sounds like you may want to create an FCMP function using proc fcmp. This is basically a way to create your own SAS functions that can be used within proc sql and data steps. For example:
/******************************************************************************
** PROGRAM: COMMON.FCMP_DIV.SAS
**
** DESCRIPTION: PERFORMS A MATHEMATICAL DIVISION BUT WILL RETURN NULL IF THE
** NUMERATOR OR DENOMINATOR IS MISSING (OR IF THE DIVISOR IS 0).
**
******************************************************************************/
proc fcmp outlib=common.funcs.funcs;
function div(numerator, denominator);
if numerator eq . or denominator in (0,.) then do;
return(.);
end;
else do;
return(numerator / denominator);
end;
endsub;
run;
Example Usage (example is data step but works equally well within SQL):
data x;
x1 = div(1,0);
x2 = div(1,.);
x3 = div(1,1);
x4 = div(0,0);
x5 = div(0,.);
x6 = div(0,1);
x7 = div(.,0);
x8 = div(.,.);
x9 = div(.,1);
put _all_;
run;
Macro functions do not return values. A macro function can 'emit' source code that
that are one or more steps,
that is a snippet that code be incorporated in a statement,
that is one or more statements that are part of a step,
etc
For your case of wanting to 'do' things in SQL, you could write SQL views that are then
opened with %sysfunc(open()) and
processed with
%sysfunc(set()) and
%sysfunc(getvarn()) and
%sysfunc(getvarc()).
Not all SQL functionality can utilized by this technique -- the select something into :result, would have to be a view with the select something and the macro would getvarc to read the result.
Access done in the open/set/get manner does not cause a step boundary to occur, so the macro processing can proceed with it's logic and eventually emit source code for snippet level consumption. (The consumer is the SAS executor that processes macro code, and implicitly compiles and runs SAS steps)
is there a solution for putting the select statement in brackets or something like this?
I need to perform this:
select t1.plz, t1.help, t1.me AS...
The Problem is, that my columns getting from a variable and my code perform this
select t1.plz, help, me
It works to the point i join it with an other table and the key gets mixed up. This works - t2.key, car,...
but
t2.car,key not, because i need to rename key as key2 and without the t2.key in front it doenst work...long story.
I need to get that t1./t2. in front of every column.
Is there a solution for this problem?
My Code(SAS)
create table work.test as
select t1.&string1 t2.&string2
I can´t put the t1. in front of every string, because i perform a loop, so this would end in a t1.plz, t1.t1.help, t1.t1.t1,me.
Use the TRANWRD() function to replace all the ", " with ", t1." then use SYMPUTX() to create the macro variables.
The Code below will fix this for you by creating the macros with the correct prefix:
data _null_;
%let str1= "plz, help, me";
%let str2= "plz, help, me";
t1= cats('t1.',tranwrd(&str1,", ",", t1."));
t2= cats('t2.',tranwrd(&str2,", ",", t2."));
call symputx('string1',t1,'L');
call symputx('string2',t2,'L');
put _all_;
run;
Output: The two macros &string1 and &string2 will have the values below.
t1.plz, t1.help, t1.me
t2.plz, t2.help, t2.me
Are there any statements\functions capable of get the name of variables?
Preferrably putting them into a column of another data set, a text field or a macro variable.
E.g.
- Data set 1
Name age sex
Jk 14 F
FH 34 M
Expected data set
Var_name_of_dataset1
Name
age
sex
PS: I know a statement: select into, which does sth relevantly
It can read the value of a column into a field with customized separetors, and therefore wish there are similar ways of reading column names into a field or a column.
Thanks
PROC CONTENTS would be the quickest way to get that information in a dataset. Column names can be found in the column NAME.
proc contents data=sashelp.class out=contents noprint;
run;
You can also use a datastep and array functions, e.g.
data colnames ;
set sashelp.class (obs=1) ;
array n{*} _NUMERIC_ ;
array c{*} _CHARACTER_ ;
do i = 1 to dim(n) ;
vname = vname(n{i}) ;
output ;
end ;
do i = 1 to dim(c) ;
vname = vname(c{i}) ;
output ;
end ;
run ;
%macro getvars(dsn);
%global vlist;
proc sql;
select name into :vlist separated by ' '
from dictionary.columns
where memname=upcase("&dsn");
quit;
%mend;
This creates a macro variable called &vlist that will contain the names of all the variables in your dataset, separated by a space. If you want commas between the variable names, all you have to do is change the 'separated by' value from ' ' to ', '. The use of the upcase function in the where statement avoids problems with someone passing the dataset name in the wrong case. The global statement is needed since the macro variable created will not necessarily be available outside the macro without defining it as global
Slightly changed from SAS help and documentation.
%macro names(dsid);
%let dsid=%sysfunc(open(&dsid, i));
%let num=%sysfunc(attrn(&dsid,nvars));
%let varlist=;
%do i=1 %to &num ;
%let varlist=&varlist %sysfunc(varname(&dsid, &i));
%end;
%let rc = %sysfunc(close(&dsid)); /*edit by Moody_Mudskipper: omitting this line will lock the dataset */
%put varlist=&varlist;
%mend names;
%names(sasuser.class) ;
Then we preserve case and the order off data, even if numeric and character is mixed.
I'm not sure Rawfocus assertion that reading dictionary tables queries all libraries is true, had the example used sashelp.vcolumn instead then it would be true, that approach is very slow and does access all the libraries allocated. (You can prove this with the SAS RTRACE system option.)
I am of the opinion that a sql query to dictionary.columns is the fastest of the methods outlined here. Obviously the macrotised code would work without the macro but the point of the macro here is I think as a utility; put the code into your favourite macro library and you never need to think about it again.