SAS - SQL ODBC connection - converts date type to text [duplicate] - sql

I'm fairly new to SAS and recently we migrated some of our SAS datasets to a SQL Server table but we are still using SAS to do our analysis. I have run into a problem when SAS is trying to bring in the data from the SQL Server table and have SAS check if the srv_edt date is between the SAS dates of dos_beg_dt1 and dos_end_dt1.
When SAS tries to compare the dates I get an error of: ERROR: WHERE clause operator requires compatible variables.
The dos_beg_dt1, dos_end_dt1, and srv_edt (SQL date format) all "appear" in the format of yyyy-mm-dd. When I bring the srv_edt into a SAS table it reads it as a character date. So I've tried changing the format of the dates and then I will get an error like:ERROR: Variable srv_edt has been defined as both character and numeric. I can't seem to find the correct format or function to get SAS to do the comparison to see if the srv_edt (SQL) is between the dos_beg_dt1 and dos_end_dt1 SAS dates.
The code I use is as follows:
libname sql odbc dsn=test schema=dbo;
%let dos_beg_dt1 = %sysfunc(intnx(qtr,&date,-1,beginning),yymmdd10.);
%let dos_end_dt1 = %sysfunc(intnx(qtr,&date,-1,end),yymmdd10.);
data sample;
set sql.table;
where &dos_beg_dt1 <= srv_edt <= &dos_end_dt1;
run;
For reference I am using SAS 9.2 to connect via odbc to SQL Server 2008.
Any help or advice would be greatly appreciated.

SAS stores and uses dats as numeric variables. If you had not identified the column srv_edt as a date column when you migrated the database to SQL server everything would now process correctly.
I assume that currently and into the future you will just store the tables in SQL server and all the processing will be in SAS.
You have a few options.
1/ re-migrate the SAS tables but identify all the date, time and datetime columns as just numeric. They all can be stored as 8 byte floating point. The date variables may also be stored (in SQL Server) is long integers. The code would need a slight change so that the macro variables would be numeric.
%let dos_beg_dt1 = %sysfunc(intnx(qtr,&date,-1,beginning));
%let dos_end_dt1 = %sysfunc(intnx(qtr,&date,-1,end));
2/ keep the date, time, and datetime variables in SQL Server format and change the data type of the column when using the data. (Note the reverse will be necessary on output). SQL Server will present the date variables as strings (character) so that your expression above will need to be -
%let dos_beg_dt1 = %sysfunc(intnx(qtr,&date,-1,beginning));
%let dos_end_dt1 = %sysfunc(intnx(qtr,&date,-1,end));
data sample;
set sql.table;
where &dos_beg_dt1 <= (input(srv_edt, yymmdd10.0)) <= &dos_end_dt1;
That to make sure when using SAS processing the type is numeric which is what the input function will do.
3/ keep the date, time, and datetime variables in SQL Server format and change your working to accommodate that fact. That is comparisions will be using character data and output will need to produce characters. SQL Server will present the date variables as strings (character) so that your expression above will need to be -
%let dos_beg_dt1 = %sysfunc(intnx(qtr,&date,-1,beginning), yymmdd10.);
%let dos_end_dt1 = %sysfunc(intnx(qtr,&date,-1,end), yymmdd10.);
data sample;
set sql.table;
where ("&dos_beg_dt1" <= srv_edt) and
(srv_edt <= "&dos_end_dt1");`
Note double quotes " " required surrounding macro variables as this
comparison is numeric.

If the column srv_edt is showing up in your SAS data set as a character variable, that means it is really a character variable or it's been converted to character by the ODBC driver you are using (possibly because the native data type is not supported by ODBC).
You'd be better off changing this to a PROC SQL pass-thru query if possible. You would need to figure out the native syntax that corresponds to the SAS intnx function (and I cannot help you there). As written, the entire table must be read (because you are using a SAS function). If you use a pass-thru query, SAS will only receive the rows that match the whee clause.
There might be setting in the ODBC driver that control this behavior. I'll add the ODBC and SQL Server tags to your question; you may get more "hits".

SQL Server introduced new date and datetime types in SQL Server 2008 (prior, there was only one type for all date/datetime variables). This usage note suggests that you need to install a new set of SQL Server ODBC drivers for SAS to read the date variables correctly. It suggests this would be installed normally if you have SQL Server 2008 Tools (like SQL Server Management Studio) on the machine that is doing the ODBC connection, but you might have multiple drivers installed and need to ensure you are using the right one.
That said, it is not a bad idea to use pass-through SQL to pull the data across, as that might make it easier to do the pull (as you don't have to worry as much about the ODBC driver). The generalized pass through connection string is
proc sql;
connect to odbc (required="driver=sql server native client 10.0;
Server=server;Trusted_Connection=Yes;DATABASE=database;");
create table X as select * from connection to odbc(... sql server native code here ...);
quit;
From your question it sounds like you're more of a SQL person and can then construct the query yourself; if you are not, either edit the question to include that request (and then either a SQL Server person or myself will answer). You can use SAS macro variables in that query (ie, to pass the current date) as long as you do not enclose them or the query in single quotes.

Related

colon(:) and dot(.) as millisecond separator in datetime2

I have migrated a Sybase database to SQL server 2008.
The main application that using the database trying to set some of dateTime2 column with data like 1986-12-24 16:56:57:81000 which is giving this error:
Conversion failed when converting date and/or time from character string.
Running the same query using dot(.) instead of colon(:) as millisecond separator like 1986-12-24 16:56:57.81000 or limiting the milliseconds to 3 digits like 1986-12-24 16:56:57:810 will solve the problem.
NOTE:
1- I don't have access to the source of application to fix this issue and there are lots of table with the same problem.
2. Application connect to database using ODBC connection.
Is there any fast forwarding solution or should i write lots of triggers on all tables to fix it using the above solutions?
Thanks in advance
AS Gordon Linoff said
A trigger on the current table is not going to help because the type
conversion happens before the trigger is called. Think of how the
trigger works: the data is available in a "protorow".
But There is a simple answer!
Using SQL Server Native Client Connection instead of basic SQL Server ODBC connection handle everything.
Note:
1. As i used SQL Server 2008 version 10 of SQL server native client works fine but not the version 11 (it's for SQL Server 2012).
2. Use Regional Settings make some other conversion problem so don't use it if you don't need it.
Select REPLACE(getdate(), ':', '.')
But it will Give String Formate to datetime Which is not covert into DateTime formate
Why would you need triggers? You can use update to change the last ':' to '.':
update t
set col = stuff(col, 20, 1, '.');
You also mistakenly describe the column as datetime2. That uses an internal date/time format. Your column is clearly a string.
EDIT:
I think I misinterpreted the question (assuming the data is already in a table). Bring the data into staging tables and do the conversion in another step.
A trigger on the current table is not going to help because the type conversion happens before the trigger is called. Think of how the trigger works: the data is available in a "protorow".
You could get a trigger to work by creating views and building a trigger on a view, but that is even worse. Perhaps the simplest solution would be:
Change the name and data type of the column so it contains a string.
Add a computed column that converts the value to datetime2.

Using a Teradata UDF in SAS Implicit Sql Pass Thru

I am trying to use a Teradata UDF (User Defined Function) in a SAS Implicit SQL which establishes the connection to Teradata using LIBNAME Statement.Assume that the function is called PTY_DECRYPT and is defined in a Database called TEST in Teradata. The Purpose of this function is to decrypt values in a Column of a View in Teradata.
What works is using the UDF in an Explicit Sql .Below I am using the function on a column called SSN_NBR in a view called V_TEST_PERS present in the Database called SAMPLE.
Explcit Sql:
Options debug=DBMS_TIMERS sastrace=',,,d'
sastraceloc=saslog no$stsuffix fullstimer;
Proc Sql;
Connect to TERADATA(User=XXXXX pwd=XXXXX server=XXXXX);
Create Table Final as
select * from connection to teradata
(
Select
sub_id,
SSN_NBR,
TEST.PTY_DECRYPT(SSN_NBR,'T_ssn_test',400,0,0 ) as SSN_NBR_Decrypt
from SAMPLE.V_TEST_PERS
);
disconnect from teradata;
Quit;
But I would like to use the same function in an Implicit SQL but it does not work. Any ideas as to how to make it work in an Implicit Sql with minimum changes to the Implicit SQL?
Implicit Sql
Options debug=DBMS_TIMERS sastrace=',,,d'
sastraceloc=saslog no$stsuffix fullstimer;
Libname Td Teradata User=XXXXX pwd=XXXXX server=XXXXX database=SAMPLE ;
Proc sql;
Create table Final as
select
sub_id,
SSN_NBR,
TEST.PTY_DECRYPT(SSN_NBR,'T_ssn_test',400,0,0 ) as SSN_NBR_Decrypt
from Td.V_TEST_PERS;
Quit;
In your implicit SQL you reference the view with the LIBNAME alias TD, however when you reference the UDF you are not aliasing the TEST database containing the UDF with the LIBNAME alias. Syntactically, you may not be able to do that in SAS. (e.g. TD.TEST.PTY_DECRYPT() - in fact I wouldn't expect this to work)
The UDF may need to be placed in SYSLIB or TD_SYSFNLIB so that it is in a default search path for the database optimizer to find the UDF without being fully qualified. (e.g. TD_WEEK_BEGIN()) Alternatively, the UDF could be placed in database SAMPLE but that likely violates how UDFs are maintained in your environment, as it would in my environment.
Otherwise, the UDF call could be embedded in a view on the database, but then you have other issues to consider with the security of that column if your environment is not granting security on a column level basis to views containing encrypted data elements. (e.g. PHI, PII, etc.) Without a row-column level security mechanism in place to dynamically filter a users ability to see the column you are decrypting in the view putting the UDF into the view isn't going to work.
I asked the same question the SAS Communities Forum and I am glad to say that i did find a Solution to this Problem.
Please see the link below :
https://communities.sas.com/t5/Base-SAS-Programming/Using-a-Teradata-UDF-in-SAS-Implicit-Sql-Pass-Thru/m-p/266850/highlight/false#M52685

SQL: to_char alternative for Oracle AND SQL-Server

I have some sql statements, which i am using for Oracle. This sql statements are used by a programm from me.
I want to support Oracle and SQL-Server with my program without having different sql statements for Oracle and SQL-Server.
Which alternative can i use for the specific Oracle SQL-Statements:
to_char(FIELDNAME, 'YYYY')
to_char(FIELDNAME, 'YYYYMMDD')
to_char(FIELDNAME, 'DD.MM.YYYY')
The sql statements have to work for Oracle and SQL-Server.
Even if at a first glance the SQL implementation from two different vendors looks similar, when working with real life enterprise applications you will stumble upon a large number of differences, and I am only talking about SQL, when comparing PL/SQL with T-SQL there is hardly any resemblance left.
When trying to reduce the usage of two databases to only common functionality, you will loose a lot of their power, you could as well use a txt file on the file system.
One elegant solution, as someone already suggested, would be to leave the columns in the database as DATE data type and extract your data in the application code that stands above the database, if any. For example, in Java, you will map your database DATE columns to java.sql.Date no matter if that date comes from Oracle or from SQL Server.
Still, if you want to get your formatted data from the database, you could create separate columns that hold the formatted date, for example :
FIELDNAME | FIELDNAME_YYYY | FIELDNAME_YYYYMMDD | FIELDNAME_DDMMYYYY
I don't think there are common functions to do what you want. Oracle supports the ANSI standard extract() function for extracting date parts. SQL Server has separate functions for YEAR(), MONTH(), and DAY(). Oracle uses TO_CHAR(); SQL Server uses CONVERT().
One option is to define the functions YEAR(), MONTH(), and DAY() in Oracle and then use string concatenation (via the CONCAT()) function to combine the data. Or, write specific functions in each database for what you want to do. Or, perhaps, someone has implemented TO_CHAR() in SQL Server and you can grab the code from the web.
Finally i found a solution. Maybe its useful some other people too.
You can set the input format for a date...
Oracle: ALTER SESSION SET NLS_DATE_FORMAT = 'DD.MM.YYYY'
SQL-Server: SET DATEFORMAT dmy

Lookup transformation between DB2 packed decimal and SQL Server DT_NUMERIC in SSIS

We use DB2 as our main production database, but we use SQL Server for many other things i.e. to do integration between other customers and vendors via EDI etc.
I have a table in SQL with SO numbers and I try to make a lookup in DB2 to get all the invoices for the SO's in my table, so here's what I did.
Created a connection to the DB2 using the Microsoft® OLEDB Provider for DB2
Created a data fllow with a source using a SQL Server connection.
Added a Data Conversion Transformation trying to convert the INT so value to a decimal with a precision of 12, but I couldn't change a precision in a DT_DECIMAL, so the only datatype that I have the option to change the precision is DT_NUMERIC.
Added a lookup transformation to lookup the data withing DB2.
Now when i try to create the join between the source table and DB2 I get an error Cannot map the input column, 'so', to the lookup column, 'orno', because the data types do not match.
According to Microsoft this is not a bug and they suggest to use the DT_NUMERIC where you can change the precision.
If I try to convert the SO to a DT_DECIMAL without changing the precision I'd get the same error mentioned above.
Is there any way to work around the limitations from SSIS and change the precision in a DT_DECIMAL conversion so I could do the match?
Or any other suggestions?
The simple answer is to change the connection property in the DB2 connection to treat DECIMAL as NUMERIC.
See bellow

Teradata and SAS with BigInt's

We have a teradata database that is filled with BIGINT datatypes. SAS is the primary analytic engine for the organization. The SAS access library chokes on BIGINT and therefore forces all code to cast variables into a integer or decimal before returning them to SAS. Example:
proc sql;
connect to database (blah blah);
create table sas_dataset as
select * from connection to database(
select
cast(bigint_var as integer),
varchar_var,
char_var,
from some_table
);
disconnect from database;
quit;
Does anyone know a way to fix this? Possibly a macro for casting variables or SAS access fix? Keep in mind that there are literally thousands of variables in this database that are bigint and making changes to the database is out of the question.
If you can't fix it on the SAS side, you can always create a set of views on the Teradata side to do the casting. Then have SAS access the views:
create view sas_cast_db.some_table as
select col1, col2, cast(bigint_var as decimal(18)), col3
from real_db.some_table;
Since you have lots of tables, it may be possible to write some SQL to auto-generate these from the data dictionary.
The issue is that SAS can only handle a maximum of 15 digits for a BIGINT data type.
SAS have provided a few work-arounds for SAS 9.2 (one mentioned above) for this issue. You can also get your SAS Platform Admin to arrange for the SAS server to be updated so that it truncates the BIGINT fields to 15 digits (obvious caveats apply), or update your LIBNAME statement to set all BIGINTs to be cast automatically.
http://support.sas.com/kb/39/831.html
Sample code
data temp1;
set mylib.bigclass (dbsastype= (id='char(20)'));
run;