SSIS Variable Date Failing between SQL Server and ORACLE - sql

Good Afternoon All,
I have spent about 6 hours trying to get formatting to work through SSIS using a Max Date Variable to identify into a where clause - Just have no luck!
I have created a variable called my_date which fetched the Max(Date) from a local SQL server table to understand the last load point for that table - using the below code:
SELECT CAST(FORMAT(MAX(Business_Date), 'dd-MMM-yyyy') AS varchar) AS my_date FROM Table
This fetches the date correctly as 17-Sep-2018.
I have then mapped my result set as my_date -> User::max_date
I have set my max_date variable to a string data type under the package scope.
I have tested my variable out by using breakpoints to ensure this runs all the way through in the correct format - and this works 100%.
I then have a data flow task running to fetch data from my ORACLE DB to insert into my SQL Server table which contains the following SQL command:
SELECT *
FROM Table2
WHERE (BUSINESS_DATE > to_date('#[User::max_date]', 'DD-MON-YYYY'))
However I get the ORA-01858 - [TABLE] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E07.
An OLE DB record is available. Source: "Microsoft OLE DB Provider for Oracle" Hresult: 0x80040E07 Description: "ORA-01858: a non-numeric character was found where a numeric was expected
".
If I go and replace my variable directly with the contents of the variable shown by the breakpoint in locales it works perfectly!
I have attempted multiple format types from the initial export through to the final where clause and this seems to be the closest I have come to pushing it through but it is still complaining about format.
Please help - Images below to help see setup.
Control Flow - displaying the execute SQL and the data flow task
Locale showing Variable is inserting after breakpoint is reached

Managed to get it working!
By adding an intermediary variable where the value and expression contains the following:
"SELECT *
FROM TABLE
WHERE (BUSINESS_DATE > to_date('"+#[User::max_date]+"' , 'DD-MON-YYYY'))"
I then amended my source OLEDB to SQL Command from variable and selected the above variable created and it worked perfectly!

Try mapping your user parameter in the "OLE DB Data Source Editor", under "Parameters".
1) Change SQL Command Text (change #[User::max_date] to ?), like this:
SELECT *
FROM Table2
WHERE (BUSINESS_DATE > to_date('?', 'DD-MON-YYYY'))
2) Then in the parameter editor, map parameter 1 to #[User::max_date].
https://learn.microsoft.com/en-us/sql/integration-services/data-flow/map-query-parameters-to-variables-in-a-data-flow-component?view=sql-server-2017
Also, the "Oracle Provider for OLE DB" behaves differently than the "Microsoft OLE DB Provider for Oracle", so it depends which you are using.

Related

Getting DataType error in SSIS - Package is failing

I have two packages that I built in SSIS. This package goes out to a file path, pulls in two excel files, and loads them to two tables in SQL Server db. One of the packages runs fine and loads its respective table. However, the other excel/csv file contains two date columns and the package is failing because the SQL Server table that it needs to load to has two DateTime columns. The error I am getting is:
-There was an error with OLE DB Destination.Inputs[OLE DB Destination Input].Columns[CustDate] on OLE DB Destination.Inputs[OLE DB Destination Input].
I have tried to cast the datetime column to Date in the SQL table, and it still doesn't work and throws the same error. I have added a data conversion task in SSIS to try and convert the two date fields to multiple datatypes, and I still get an error. Can someone please tell me what datatype I should convert these two fields in the SSIS package where it would work? The first package runs fine because it doesn't contain any date fields in the excel file or the table it loads to.
So far, I have tried to convert these two date fields to: string[DT_STR], Unicode string[DT_WSTR], date[DT_DATE] and none of them work. Can anyone offer up any suggestions? Thank you!
So far, I have tried to convert these two date fields to: string[DT_STR], Unicode string[DT_WSTR], date[DT_DATE] and none of them work. Can anyone offer up any suggestions? Thank you!
Made the following 2 changes. It will give you answer.
1) Select your excel file connection and add IMEX=1 to the end of the connection under properties. So your connection string will now look like
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\SSIS\Test.xls;Extended Properties="EXCEL 8.0;HDR=YES;IMEX=1";
2) Use the following expression in the derived column transformation
ISNULL([CustDate]) ? NULL(DT_DATE) : (LEN(TRIM([CustDate])) == 0 ? NULL(DT_DATE) : (DT_DATE)((DT_DBDATE)TRIM([CustDate])))

Error in SSIS - Parameters cannot be extracted when parsing query, syntax error

I am building an SSIS package to select some data as XML and export it to a flat file. I've running it via a for each loop, I have a sql execute task to select all records in a table and then loop through each one to create a single XML file.
Here is the SQL statement I am trying to execute as part of the data flow in the OLE DB Source Editor. The select is inside another SELECT (...) AS XML_TEXT to make the output to be text
SELECT
(SELECT SalesCode, ProductName, ManufacturingSite, Country, Language, Status, RevisionDate, ArchiveDate, PFFileName, XMLFileName, Version
FROM ir_cp_connect_data
WHERE (id = ?)
FOR XML RAW ('Data'), ROOT ('Sales'), ELEMENTS XSINIL)
AS XML_TEXT
I've tried to enter it direct as a SQL command and also in a variable. When I enter it directly into as a command, it errors saying it cannot parse the query, giving syntax error, permission violation or other non specific error.
When I switch it out to be a variable in data access mode, it errors with a long error message,
SSIS Error Code DTS_E_OLEDBERROR. An OLD DB Error has occurred, Error Code 0x80004005. An OLE DB record is available.
Again it then goes on to list 'syntax error, permission violation or other non specific error.'
If I run the select statement in SQL Management Studio it works, albeit I substitute the id = ? for id = #id and declare the #id in and assign it a value.
I'm working with VS2008 and SQL Server 2008R2.
I feel I am missing something very obvious but I cannot put my finger on it. If I drop the opening SELECT (..) AS XML_TEXT, the query parses fine, but the output is System.Byte[] which is no good for the flat file.
Any help would be much appreciated.

Can't convert String to Numeric/Decimal in SSIS

I have five or six OLE DB Sources with a String[DT_STR], with a length of 500 and 1252 (Latin) as Code Page.
The format of the column is like 0,08 or 0,10 etc etc. As you can see, it is separated with a comma.
All of them are equal except one of them. In this one source, I have a POINT as separation. On this it is working when I set the Data Type in the advanced editor of the OLE DB Source. On another (with comma separated) it is also working, if I set the Data Type in the advanced editor of the OLE DB Source. BUT the weird thing is, that it isn't working with the other sources although they are the same (sperated with comma).
I tested Numeric(18,2) and decimal(2).
Another try to solve the problem with the conversion task and/or the derived column task, failed.
I'm using SQL Server 2008 R2
Slowly, I think SSIS is fooling me :)
Has anyone an idea?
/// EDIT
Here a two screens:
Is working:
click
Isn't working:
click
I would not set the Data Type in the Advanced Editor of the OLE DB Source. I would convert the data in the SQL Code of the OLE DB Source, or in a Script Transformation e.g. using Decimal.TryParse , which would update a new column.
SSIS is unbeleivably fussy over datatypes and trying to mess with its internals is not productive.
Check that there are any spaces in between the commas, so that the SSIS is throwing an error trying to convert the blank space to a number. A blank space does not equal nothing in between spaces.
Redirect error rows and output the data to a file. Then you can examine the data that is being rejected by the SSIS and determine why it's causing error.
Reason for the error
1) Null’s are not properly handled either in the destination database or during SSIS package creation. It is quite possible that the source contains a null database but the destination is not accepting the NULL data leading to build generate above error.
2) Data types between source and destination does not match. For example, source column has varchar data and destination column have an int data type. This can easily generate above error. There are certain kind of datatypes which will automatically convert to another data type and will not generate the error but there are for incompatible datatypes which will generate The value could not be converted because of a potential loss of data. error.
The Issue arises when there is unhandled space or null. I have worked around using the Conditional (Ternary) Operator which checks the length:
LEN(TRIM([Column Name])) >= 1 ? (DT_NUMERIC,38,8)[Column Name] : 0

Conversion Failed Due To Data Overflow (Numeric)

I am trying to move data from a .dbf file to a table in SQL Server 2008 and am getting the following error on multiple numeric columns:
OLE DB provider "MSDASQL" for linked server "(null)" returned message "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.".
Msg 7341, Level 16, State 2, Line 1
Cannot get the current row value of column "[MSDASQL].apryr" from OLE DB provider "MSDASQL" for linked server "(null)". Conversion failed because the data value overflowed the data type used by the provider.
It only happens on numeric columns and not on every numeric column. Character data is fine and there is no date/time data that could give any issues.
Here is a sample of the code I'm using:
insert into [table] select * from OPENROWSET('MSDASQL',
'DRIVER=Microsoft Visual FoxPro Driver;
SourceDB=[filepath];
SourceType=DBF',
'select *
from [file].dbf)
Since the data in the dbf file is customer data, I've been told I can't manually fix the garbage data in the file (assuming there is any) and everything has to be done through the SQL code. I have searched around the internet and haven't really found a solution to this problem. I'd appreciate any help.
Thank you.
Without knowing more specifics, the situation sounds simple enough: There is data in the dbf file that does not match the data(s) type in your SQL Server table. If that is the case, then you have two options:
Change your SQL Server table to accommodate the data in your dbf file.
Do not import data from the dbf file that is causing the issue.
In option #1, you could modify restrictive numeric or date-type fields to varchar or nvarchar fields. Then, you would want to modify any programs that might be assuming certain data types in the dbf file to accommodate varchar or nvarchar data. For instance you could use some kind of try-catch language that tests the conversion of data before letting a program have access to it.
If you decide to go with option #2, you can change your select query to be filter out data that does not meet the field requirements of your SQL Server table(s).
Good luck!
Check your field types on SQL Server table. Maybe some of them unable to take your DBF's BCD value.

SQL Command text with parameter OLEDB Source

I am working with SQL Server Integration Services 2008. In my Data Flow task I got a OLEDB source. I choose SQL command as the Data Access mode and following is my sql text
Select * from table1 where ID=?
And from parameters tab my Parameter Name is zero "0" and value is coming from a package level variable.
I tried to replace the parameter name with "Parameter0" but I always get the same error that
Error: "No Value given for one or more required parameters"
But there is a value for my variable
You are doing the right thing, but it is the "Preview" that is not working for parameterized source tasks. Try executing the package and you should be ok.