using macro variable to read folder content in SAS 9.4 [duplicate] - file-io

This question already has answers here:
Why won't my macro variable resolve?
(2 answers)
Closed 5 years ago.
I'm trying to get the list of files in a directory with a SAS macro that uses a macro variable to specify dynamically the folder name. The code I run is the following:
%macro veicolo(codice_veicolo);
filename pipedir pipe ' dir "some_path\&codice_veicolo" /S' lrecl=5000;
data &codice_veicolo;
infile pipedir truncover;
input line $char1000.;
length directory $1000;
retain directory;
if line =' ' or
index(upcase(line),'<DIR>') or
left(upcase(line))=:'VOLUME' then
delete;
if left(upcase(line))=:'DIRECTORY OF' then
directory=left(substr(line,index(upcase(line),'DIRECTORY OF')+12));
if left(upcase(line))=:'DIRECTORY OF' then
delete;
if input(substr(line,1,10),?? mmddyy10.) = . then
substr(line,1,10)='12/31/2999';
date=input(substr(line,1,10),?? mmddyy10.);
format date mmddyy10.;
run;
proc sort data=&codice_veicolo;
by directory descending date;
run;
data folder_&codice_veicolo(drop=i line);
set &codice_veicolo;
by directory;
length filename $75;
retain number_of_files_in_directory directory_size;
if first.directory then
do;
number_of_files_in_directory=input(scan(line,2,' '),32.);
call symput(nfiles,number_of_files_in_directory);
directory_size=input(scan(line,4,' '),comma32.);
end;
file_size=input(scan(line,3,' '),comma32.);
filename=' ';
do i=4 to 100;
filename=trim(left(filename))||' '||scan(line,i,' ');
if scan(line,i,' ')=' ' then
leave;
end;
if index(upcase(line),'FILE(S)') then
delete;
if date ge '30DEC2999'd then
delete;
run;
%mend;
When I then execute the macro with the argument codice_veicolo being the name of the folder I want to search in, I get the following error:
Output Err std:
The system cannot find the path specified.
NOTE: 20 records were read from the infile PIPEDIR.
The minimum record length was 0.
The maximum record length was 90.
NOTE: The data set WORK.JL6AME1A6FK000442 has 2 observations and 3 variables.
NOTE: DATA statement used (Total process time):
real time 0.05 seconds
cpu time 0.01 seconds
I supposed that for some reason it could not resolve the macro variable, but if I run:
%let pgmpath = %sysfunc(pathname(pipedir));
%put &pgmpath;
I get the proper path and the proper directory, therefore I assume the problem is in the infile statement. The code runs fine without using macro variables.
I am using SAS 9.4 on Windows 8. Any ideas??
Thank you in advance :)
Luca

Macro variable references are not expanded inside of single quotes.
Try this instead.
filename pipedir pipe %sysfunc(quote(dir /s "some_path\&codice_veicolo")) lrecl=5000;

Related

To grep contents from a CSV/Text File using Autohotkey(AHK) Script

Can anyone please help me in writing a script in AHK based on below requirement.
Requirement:
I have a CSV/TXT file in my windows environment which contains 20,000+ records in below format.
So, when I run the script it should prompt a InputBox to enter an instance name.
Example : If i enter Instance4 , it should display result in MsgBox as ServerName4
Sample Format:
ServerName1,ServerIP,Instance1,Type
ServerName2,ServerIP,Instance2,Type
ServerName3,ServerIP,Instance3,Type
ServerName4,ServerIP,Instance4,Type
ServerName5,ServerIP,Instance5,Type
.
.
.
Also as the CSV/TXT file contains large no of records , pls also consider the best way to avoid delay in fetching the results.
Please post your code, or at least show what you've already done.
You can use a Parsing Loop with CSV as the delimiter, and make a variable for each 'Instance' who's value is that of the current row's 'ServerName'.
The steps are to first FileRead the data from the file, then Loop, Parse like so:
Loop, Parse, data, CSV
{
; Parses row by row, then column by column in each row.
; A_LoopField // Current value
; A_Index // Current loop's index
; Write a script that makes a variable named with the current value of column 3, and give it the value of column 1
}
After that, you can make a Goto loop that spams InputBox and following a command that prints out the needed variable using the MsgBox command, like so:
MsgBox % %input%

SAS: How to run a loop on a set of input variables for a function

I'm basically trying to make a program that can extract the relevant variables from a CSV file and perform stationarity tests on these timeseries variables without having to type out the variable names manually. I'm fairly new and maybe there's redundancies, but I've basically embeded the variable names into variable1, variable2, etc...
So when I've tested the ADF macro and input the variable as:
proc arima data=&y;
identify var= &variable1 stationarity =(adf = 3);
run;
it works, but I'm struggling to automate this process and would appreciate any help you could offer. Thanks.
Here is what I have to far:
proc import datafile=".....csv"
out=data
dbms=csv
replace;
getnames=yes;
run;
proc contents data=data out=contents noprint;
run;
data contents;
set contents(keep = name);
if Name = "Quarter" then delete;
run;
data _NULL_;
set contents;
call symputx(cats('variable',strip(_n_)),name);
run;
data want;
do i=1 to 4;
expert=symget(cats('variable',i));
output;
end;
run;
%macro ADF(y = ,x = );
proc arima data=&y;
identify var= &x stationarity =(adf = 3);
run;
%mend ADF;
You're very close... You just need to look into call execute. It will allow you to build commands and have them execute after the datastep finishes.
In the below code we're building the calls to the macro that we want to run. They will be exected as instructions after the datastep has finished processing. This will allow us to use values within the datastep to build the commands:
data want;
do i=1 to 4;
expert=symget(cats('variable',i));
call execute (cats('%ADF(y=data,x=',expert,');'));
end;
run;
There are other ways to do it but this is the simplest based on what you already have in place.
Just make sure your macro is defined prior to running the datastep.

How do we print characters line by line and save it to csv or text file in PLSQL

DECLARE
V_NUMBER NUMBER :=23;
BEGIN
LOOP
V_NUMBER:=V_NUMBER+1;
EXIT WHEN V_NUMBER:=25;
--Some kind of function to be applied for printing and nesting lines into CSV or TEXT file.
END LOOP;
COMMIT;
END;
Scripting an Oracle SQL Query for Creating a CSV or Text Typed File Output
Consider running this from a SQL Plus session and use the SPOOL command. All output of the SQL command that follows will be written to the file name you specify.
If you need to append your results each successive time the SQL commands are run, then an OS level command would work appropriately when invoking this sqlplus executable block of PL/SQL:
Where the file name of this script is: "sample_csv_out.sql"
DECLARE
v_total_columns constant number:= 3; -- Number of columns queried
v_column_counter number;
v_csv_record varchar2(1000);
c_csv_column_format constant varchar2(15):=
'<<COLUMN1_VAL>>,<<COLUMN2_VAL>>,<<COLUMN3_VAL>>';
cursor result_cur is
SELECT column1, column2, column3
FROM tablea
WHERE column1 = ... ;
BEGIN
v_csv_record:= 'COLUMN1,COLUMN2,COLUMN3';
dbms_output.put_line (v_csv_record);
FOR i in result_cur LOOP
v_csv_record:= replace(c_csv_column_format, '<<COLUMN1_VAL>>', i.column1);
v_csv_record:= replace(v_csv_record, '<<COLUMN2_VAL>>', i.column2);
v_csv_record:= replace(c_csv_record, '<<COLUMN3_VAL>>', i.column3);
dbms_output.put_line(v_csv_record);
END LOOP;
END;
So, for example in a WINDOWS O/S environment, the call to append the output to a specific file name would be:
C:\> sqlplus sample_csv_out.sql >> mycsv_out.csv
The >> notation instructs the operating system to pipe the output of running sample_csv_out.sql via a sqlplus session.
The command DBMS_OUTPUT does the rest. If you need more details, see more Oracle documentation on DBMS_OUTPUT.
COMMENTS: I chose the RECORD STRING TEMPLATE approach to make this script a little more flexible and reusable. I recommend to keep any data manipulation logic within the CURSOR statement. Often when the two are mixed, it gets harder to debug any typos in syntax within a long string of values.
The construction of an output record was also designed to reduce typos, mistakes and frustration... if there are more than 3 columns in your own scripts, adding another element to the output string is mostly a cut-and-paste operation. Likewise with the "header" row (column titles).
You can read and write files in PL/SQL using the UTIL_FILE package
http://docs.oracle.com/cd/B19306_01/appdev.102/b14258/u_file.htm

SAS: Using the Put statement to create dynamic code

I'd like to create dynamic code using the PUT statement. According to this document from SUGI 29 (http://www2.sas.com/proceedings/sugi29/175-29.pdf),
put
"data XXXXX; "
/ 'infile "&datadir/&compid&filetype" missover ls=' tbla_fle
';' / 'input'
;
is equivalent to running
data onecomp ;
infile
"&datadir/&compid&filetype"
missover ls = 268 ;
input
However, when I try something similar to their example, the code enclosed in the PUT statement doesn't run and is instead written to the SAS Output Log:
data _NULL_;
put // "data put_test;" / "b=2;" / "run;";
run;
In Output Log:
data put_test;
b=2;
run;
I've checked the SAS documentation, and it seems that PUT is only used to "Write lines to the SAS log, to the SAS output window, or to an external location that is specified in the most recent FILE statement." Nowhere does it say that it can be used to create dynamically generated code.
I know that I must be missing something, but I'm not sure what. I'm using SAS Enterprise Guide 4.1.
Thank you!
The idea is to use put to write your generated code to a file. You then %include the file into your SAS session to run it. What you're missing is a file statement and the %include directive.
data _null_;
file 'temp.sas'; /* redirects put to a file instead of the SAS log */
put
"data XXXXX; "
/ 'infile "&datadir/&compid&filetype" missover ls=' tbla_fle
';' / 'input'
;
run;
%include 'temp.sas';

SAS - Reading a File Backwards?

I need SAS to read many large log files, which are set up to have the most recent activities at the bottom. All I need is the most recent time a particular activity occurred, and I was wondering if it's possible for SAS to skip parsing the (long) beginning parts of the file.
I looked online and found how to read a dataset backwards, but that would require SAS to first parse everything in the .log file into the dataset first. Is it possible to directly read the file starting from the very end so that I can stop the data step as soon as I find the most recent activity of a particular type?
I read up on infile as well, and the firstobs option, but I have no idea how long these log files are until they are parsed, right? Sounds like a catch-22 to me. So is what I'm describing doable?
I'd probably set up a filename pipe statement to use an operating system command like tail -r or tac to present the file in reverse order to SAS. That way SAS can read the file normally and you don't have to worry about how long the file is.
If you mean parsing a sas log file, I am not sure if reading the log file backward is worth the trouble in practice. For instance, the following code executes less than a tenth of a second on my PC and it is writing and reading a 10,000 line log file. How big is your log files and how many are there? Also as shown below, you don't have to "parse" everything on every line. You can selectively read some parts of the line and if it is not what you are looking for, then you can just go to the next line.
%let pwd = %sysfunc(pathname(WORK));
%put pwd=&pwd;
x cd &pwd;
/* test file. more than 10,000 line log file */
data _null_;
file "test.log";
do i = 1 to 1e4;
r = ranuni(0);
put r binary64.;
if r < 0.001 then put "NOTE: not me!";
end;
put "NOTE: find me!";
do until (r<0.1);
r = ranuni(0);
put r binary64.;
end;
stop;
run;
/* find the last line that starts with
NOTE: and get the rest of the line. */
data _null_;
length msg $80;
retain msg;
infile "test.log" lrecl=80 eof=eof truncover;
input head $char5. #;
if head = "NOTE:" then input #6 msg $char80.;
else input;
return;
eof:
put "last note was on line: " _n_ ;
put "and msg was: " msg $80.;
run;
/* on log
last note was on line: 10013
and msg was: find me!
*/