SAS Proc SQL string truncation issue - sql

Have the creation of a simple table from values in another table below:
create table summary3 as
select
substr(&Start_dt.,1,4) as time_range,
NFDPs ,
NFDPExceeds,
NblkExceeds,
NFDPExceedsLT30s as NFDPExceedsLT30,
NReports as Nbr_report ,
prcnt_FDP_ext ,
prcnt_blk_ext ,
prcnt_extLT30 as prcnt_ext_LT30,
prcnt_report,
monotonic() as id
from OAP_exceedances_by_year;
my problem is arising on the very first column i created, time_range. When i try adding values to this table later on, I noticed that this column is capped to char's of length 4 or shorter, and it automatically truncates anything greater. Is there a way I can either change that first statement, or perhaps my future insert / set statements to avoid the truncation? IE i still want the first row to only be 4 characters but I may need future rows to be more.
Thanks!

This depends on how you do your future processing. If your data step later on says
data summary_final;
set summary3;
time_range = "ABCDEF";
run;
Then you could just change it like so:
data summary_final;
length time_Range $6;
set summary3;
time_range = "ABCDEF";
run;
But you certainly could do what you say also in the initial pull. For example...
proc sql;
create table namestr as
select substr(name,1,4) as namestr length=8
from sashelp.class;
quit;
That creates namestr as length=8 even though it has substr(1,4) in it; the names there will be truncated, as the substr asks it to, but future names will be allowed to be 8 long.

Related

Commas using SAS and TD SQL

I am using SAS to pull data in a Teradata environment. I am counting the rows in the Teradata table, but want the output to be in a comma format (i.e. 1,000,000). I was able to use the code below to display the value as a comma, but when I try to add the column in SAS, I can't since the output is in a character format. Does anyone have any suggestions on how to format the number value as comma, so that it can be used for calculation purposes in SAS? Thanks.
CAST(Count(*) as (format 'Z,ZZZ,ZZ9')) as char(10)) as rowCount,
Assuming you're using pass through, pull it in as numeric and format it on the SAS side. You've now converted it to character (char10) and SAS doesn't do math on character variables which makes logical sense.
select rowCount format=comma12. from con
(select
count(*) as rowCount ....
)
If you have a select * you can always format it later in a data step or via PROC DATASETS. SAS separates the display and storage layers so the format controls the appearance but the underlying data still remains numeric.

Access SQL Date Function

So I'm working on editing some SQL code and I've just began learning it. I'm trying to fix an update query so it updates a table's value5 column with a corresponding database value. The value type from the database is a number, which I want to convert to a date and place into my table. The database number is in yyyymmdd format so I've been trying to use datefromparts() which doesn't work. Anyone have any ideas?
UPDATE tbl INNER JOIN dB ON
(dB.value1= tbl.value1 OR
dB.value2 =tbl.value2 ) AND
(LEFT(dB.value3 ,5)=tbl.value3 ) AND
(dB.value4 =tbl.value4 )
SET tbl.value5 = DateFromParts(Left(dB.value5,4),Mid(dB.value5,5,2),Right(dB.value5,2))
WHERE tblInvoice.value5 IS NULL;
The current program uses the code
"SET tbl.value5 = dB.value5"
instead (it runs perfectly fine) and I am having another issue with testing the conversion SQL code (datefromparts()). Because I am converting from numbers to time/date, I have to go into the design view of the target table and change the input data type of the value5 column from numbers to time/date. When I run the query with the conversion SQL code, the query stalls for a bit and no values get updated, leaving me with just a blank value5 column. If I now want to fill in the original number values, I change the SQL code back into its original "SET tbl.value5 = dB.value5", change the input data type from time/date to numbers, and rerun the program. The query stalls and no values are updated, and I am again left with blank columns, even though the same code left me with the corrected update values before the modifications to the SQL and table input Data types. I come from a VBA background and I'm just really confused with how this is working. Any tips would be appreciated, thanks!
Have you tried with substring instead?
SELECT DATEFROMPARTS ( left('20101231',4), substring('20101231',5,2), right('20101231',2) ) AS Result;
MS Access (and MS Jet too) have no DateFromParts function. Using DateSerial instead.
SET tbl.value5 = DateSerial(Left(dB.value5, 4), Mid(dB.value5, 5, 2), Right(dB.value5, 2))
It's not clear if you work with T-SQL or Access SQL. In Access, you can use Format:
SET tbl.value5 = CDate(Format(dB.value5, "####\/##\/##"))
In T-SQL you could use a similar method.

How can I avoid this error in SAS?

When trying to merge datasets in SAS I continuously get the following error for a number of variables:
Column 115 from the first contributor of OUTER UNION is not the same type as its
counterpart from the second
I've been able to get around this error usually by doing the following:
Changing one of the variables to the same "type" of the other. For example, changing variable A to a character type from a numeric type so that it matches the variable in the other dataset thereby allowing the merge to happen.
Importing the datasets that I am trying to merge together as CSV files and then adding the "guessing rows" option in the proc import step. For example:
proc import datafile='xxxxx'
out=fadados
dbms=csv replace;
getnames=yes;
guessingrows=200;
run;
However, sometimes in spite of importing my files as CSVs and using "guessingrows" I still get the above error and sometimes there are so many that it is VERY time consuming and not feasible to actually convert all variables to the same "type" so that they match between datasets.
Can anyone advise me on how I can easily AVOID this error? Is there another way that people get around this? I get this error so often that I am tired of having to convert every single variable. There must be another way!
******UPDATE*****
Here is an example that everyone is asking for:
proc sql;
title 'MED REC COMBINED';
create table combined_bn_hw as
select * from bndados
outer union corr
select * from hwdados;
quit;
And here is the output I get in the log:
21019 proc sql;
21020 title 'MED REC COMBINED';
21021 create table combined_bn_hw as
21022 select * from bndados
21023 outer union corr
21024 select * from hwdados;
ERROR: Column 115 from the first contributor of OUTER UNION is not the same type as its
counterpart from the second.
ERROR: Column 120 from the first contributor of OUTER UNION is not the same type as its
counterpart from the second.
ERROR: Column 173 from the first contributor of OUTER UNION is not the same type as its
counterpart from the second.
ERROR: Numeric expression requires a numeric format.
ERROR: Column 181 from the first contributor of OUTER UNION is not the same type as its
counterpart from the second.
ERROR: Column 185 from the first contributor of OUTER UNION is not the same type as its
counterpart from the second.
ERROR: Column 186 from the first contributor of OUTER UNION is not the same type as its
counterpart from the second.
21025 quit;
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE SQL used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds
Don't use PROC IMPORT to guess what types of variables you have in your data. Its decision is going to depend on what values are in the file. Just write a data step to read your CSV files yourself. Then you can control how the variables are defined.
PROC IMPORT has to guess if your ID variable is numeric or character. And since it is doing based on what is in the file it can make different decisions for different sets of data. A common example is when a character variable is totally empty then PROC IMPORT will think it should be a numeric variable.
You could recall the data step code that PROC IMPORT generates and update that to use consistent data types for your variables. But writing your own is not very hard. you don't have to make as complicated a program as PROC IMPORT generates. Just include an INFILE statement, define your variables, including attaching any required INFORMATS (like for date values) and then use a simple INPUT statement.
data want;
infile 'myfile.csv' dsd firstobs=2 truncover;
length var1 $20 var2 8 ... varlast 8 ;
informat var2 yymmdd10.;
format var2 yymmdd10.;
input var1 -- varlast;
run;
Without an example it is difficult to test. Did you try the FORCE option on PROC APPEND?
Example:
proc append base=base data=one force; run;
proc append base=base data=two force; run;
proc append base=base data=e04 force; run;
Source:
http://www.sascommunity.org/wiki/PROC_APPEND_Alternatives

SQL not finding results

This query currently is returning no results, and it should. Can you see anything wrong with this query
field title are NEED_2_TARGET, ID, and CARD
NEED_2_TARGET = integer
CARD = string
ID = integer
value of name is 'Ash Imp'
{this will check if a second target is needed}
//**************************************************************************
function TFGame.checkIf2ndTargetIsNeeded(name: string):integer;
//**************************************************************************
var
targetType : integer; //1 is TCard , 2 is TMana , 0 is no second target needed.
begin
TargetType := 0;
Result := targetType;
with adoquery2 do
begin
close;
sql.Clear;
sql.Add('SELECT * FROM Spells WHERE CARD = '''+name+''' and NEED_2_TARGET = 1');
open;
end;
if adoquery2.RecordCount < 1 then
Result := 0
else
begin
Adoquery2.First;
TargetType := adoquery2.FieldByName(FIELD_TARGET_TYPE).AsInteger;
result := TargetType;
end;
end;
sql db looks like below
ID CARD TRIGGER_NUMBER CATEGORY_NUMBER QUANTITY TARGET_NUMBER TYPE_NUMBER PLUS_NUMBER PERCENT STAT_TARGET_NUMBER REPLACEMENT_CARD_NUMBER MAX_RANDOM LIFE_TO_ADD REPLACED_DAMAGE NEED_2_TARGET TYPE_OF_TARGET
27 Ash Imp 2 2 15 14 1 1
There are a number of things that could be going wrong.
First and most important in your trouble-shooting is to take your query and run it directly against your database. I.e. first confirm your query is correct by eliminating possibilities of other things going wrong. More things confirmed working, the less "noise" to distract you from solving the problem.
As others having pointed out if you're not clearing your SQL statement, you could be returning zero rows in your first result set.
Yes I know, you've since commented that you are clearing your previous query. The point is: if you're having trouble solving your problem, how can you be sure where the problem lies? So, don't leave out potentially relevant information!
Which bring us neatly to the second possibility. I can't see the rest of your code, so I have to ask: are you refreshing your data after changing your query? If you don't Close and Open your query, you may be looking at a previous execution's result set.
I'm unsure whether you're even allowed to change your query text while the component is Active, or even whether that depends on exactly which data access component you're using. The point is, it's worth checking.
Is your application connecting to the correct database? Since you're using Access, it's very easy to be connected to a different database file without realising it.
You can check this by changing your query to return all rows (i.e. delete the WHERE clause).
You my want to change the quotes used in your SQL query. Instead of: ...CARD = "'+name+'" ORDER... rather use ...CARD = '''+name+''' ORDER...
As far as I'm aware single quotes is the ANSI standard. Even if some databases permit double quotes, using them limits portability, and may produce unexpected results when passed through certain data access drivers.
Check the datatype of your CARD column. If it's a fixed length string, then the data values will be padded. E.g. if CARD is char(10), then you might actually need to look for 'Ash Imp '.
Similarly, the actual value may contain spaces before / after the words. Use select without WHERE and check the actual value of the column. You could also check whether SELECT * FROM Spells WHERE CARD LIKE '%Ash Imp%' works.
Finally, as others have suggested, you're better off using a parameterised query rather dynamically building the query up yourself.
Your code will be more readable and flexible.
You can make your code strongly typed; and so avoid converting things like numbers and dates into strings.
You won't need to worry about the peculiarities of date formatting.
You eliminate some security concerns.
#GordonLinoff all fields in db are all caps
If that is true then that is your problem. SQL usually performs case sensitive comparisons of character/string values unless you tell it not to do so, such as with STRCMP() (MySQL 4+), LOWER() or UPPER() (SQLServer, Firebird), etc. I would also go as far as wrapping the conditions in parenthesis as well:
sql.Text := 'SELECT * FROM Spells WHERE (NEED_2_TARGET = 1) AND (STRCMP(CARD, "'+name+'") = 0) ORDER by ID';
sql.Text := 'SELECT * FROM Spells WHERE (NEED_2_TARGET = 1) AND (LOWER(CARD) = "'+LowerCase(name)+'") ORDER by ID';
sql.Text := 'SELECT * FROM Spells WHERE (NEED_2_TARGET = 1) AND (UPPER(CARD) = "'+UpperCase(name)+'") ORDER by ID';
This is or was an issue with the
With Adoquery2 do
begin
...
end
when using name in the sql, it was really getting adoquery2.name not the var name. I fixed this by changing name to Cname had no more issues after that.

Manipulating a record data

I am looking for a way to take data from one table and manipulate it and bring it to another table using an SQL query.
I have a Column called NumberStuff that has data like this in it:
INC000000315482
I need to cut off the INC portion of the number and convert it into an integer and store it into a Column in another table so that it ends up looking like this:
315482
Any help would be much appreciated!
Another approach is to use the Replace function. Either in TSQL or as a Derived Column Expression in SSIS.
TSQL
SELECT REPLACE(T.MyColumn, 'INC', '') AS ReplacedINC
SSIS
REPLACE([MyColumn], "INC", "")
This removes the character based data. It then becomes an optional exercise in converting to a numeric type before storing it to the target table or letting the implicit conversion happen.
Simplest version of what you need.
select cast(right(column,6) as int) from table
Are you doing this in a SSIS statement, or?...is it always the last 6 or?...
This is a little less dependant on your formatting...removes 0's and can be any length (will trim the first 3 chars and the leading 0's).
select cast(SUBSTRING('INC000000315482',4,LEN('INC000000315482') - 3) as int)