Occasional Script Task error in SSIS - vb.net

I am going through a very peculiar situation while executing a package. The derived columns in all the DFTs in the package have been replaced with Script tasks due to a business requirement.
Executing the package on development resulted in an occasional error sometimes with the error statement as 'Value is too large to be added to the buffer' and sometimes the script task failed stating
System.ArgumentOutOfRangeException: Year, Month, and Day parameters
describe an un-representable DateTime
OR
The milliseconds value is out of bounds (not between 0 & 999).
Some other error messages received during the executions are stated below:-
Description: Unspecified error End Error Error: 2016-11-24
10:51:03.37 Code: 0xC0047062 Source: Dft_x [279]
Description: System.ArgumentOutOfRangeException: Year, Month, and Day
parameters describe an un-representable DateTime. at
Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.HandleUserException(Exception
e) at
Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.ProcessInput(Int32
inputID, PipelineBuffer buffer) at
Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100
wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr
bufferWirePacket) End Error Error:
&
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The
PrimeOutput method on Flat File Source 1 returned error code
0xC02020C4. The component returned a failure code when the pipeline
engine called PrimeOutput(). The meaning of the failure code is
defined by the component, but the error is fatal and the pipeline
stopped executing. There may be error messages posted before this
with more information about the failure.
[OleDst_Pricing [165]] Error: SSIS Error Code DTS_E_OLEDBERROR. An
OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is
available. Source: "Microsoft SQL Server Native Client 11.0"
Hresult: 0x80004005 Description: "Invalid date format".
[OleDst_xyz [165]] Error: There was an error with
OleDst_xyz.Inputs[OLE DB Destination Input].Columns[DateColumn] on
OleDst_xyz.Inputs[OLE DB Destination Input]. The column status
returned was: "Conversion failed because the data value overflowed the
specified type.". [OleDst_x [165]] Error: SSIS Error Code
DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "OleDst_xyz.Inputs[OLE
DB Destination Input]" failed because error code 0xC020907A occurred,
and the error row disposition on "OleDst_xyz.Inputs[OLE DB
Destination Input]" specifies failure on error. An error occurred on
the specified object of the specified component. There may be error
messages posted before this with more information about the failure.
[FltSrc_x 1 [313]] Error: The attempt to add a row to the
Data Flow task buffer failed with error code 0xC0047020.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The
PrimeOutput method on FltSrc_BR_x 1 returned error code
0xC02020C4. The component returned a failure code when the pipeline
engine called PrimeOutput(). The meaning of the failure code is
defined by the component, but the error is fatal and the pipeline
stopped executing. There may be error messages posted before this
with more information about the failure.
However after tweaking the DefaultMaxBufferRows & DefaultMaxBufferSize property, I successfully executed the package individually and through SQL Server about 40 times on Development and Test environment to make sure it doesn't fail again. But the Production execution again failed with similar date specific error.
I am posting the code below contained in the Script Tasks which is similar in each of the DFTs:-
Public Class ScriptMain
Inherits UserComponent
Dim xyzArray() As String
Dim rowValue As String
Dim strDate As String
Dim columnxyz As String '= Me.Variables.MaterialMaster.ToString()
'Dim v1 As IDTSVariables100
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
rowValue = Row.XYZtoABC.ToString() + "~".ToString()
xyzArray = rowValue.Split(New Char() {"~"c})
strDate = Row.DateColumn.ToString()
columnxyz = Row.columnxyz.ToString()
CreateNewOutputRows()
End Sub
Public Sub CreateNewOutputRows()
ResultBuffer.AddRow()
ResultBuffer.xyz1 = xyzArray(1)
ResultBuffer.xyz2 = xyzArray(2)
ResultBuffer.xyz3 = xyzArray(3)
ResultBuffer.xyz4 = xyzArray(4)
ResultBuffer.xyz5 = xyzArray(5)
ResultBuffer.Datecolumn = CDate(strDate)
ResultBuffer.columnxyz = columnxyz
End Sub
End Class

The problem is data related. You are occasionally importing data that cannot be represented as a date, and trying to output it as a date. Since you don't have any code in place to catch this, you are getting an error.
You could put this line in a TRY .. CATCH block, and decide what to do in the cases where the string cannot be converted to a date:
ResultBuffer.Datecolumn = CDate(strDate)

If strDate has a specific format (i.e.: "dd-MM-yyyy HH:mm:ss"). you have to use Date.ParseExact() function like the following:
ResultBuffer.Datecolumn = Date.ParseExact(strDate,"dd-MM-yyyy HH:mm:ss",System.Globalization.CultureInfo.InvariantCulture,System.Globalization.DateTimeStyles.None)
Also if you have many specific formats, you can declare a string array and pass it to this function like the following:
dim strFormats() as string = {"dd-MM-yyyy","yyyy-MM-dd"}
ResultBuffer.Datecolumn = Date.ParseExact(strDate,strFormats,System.Globalization.CultureInfo.InvariantCulture,System.Globalization.DateTimeStyles.None)
CDate function is related to your regional settings so it is best practice to work with Date.ParseExact
Another point is to use DB_TIMESTAMP as column type instead of DT_DATE

Related

(DECIMAL)Conversion failed because the data value overflowed the specified type

Everything runs fine, just error in the OLE DB destination, Error said its conversion failed and i also try to convert the ProjectedROI but convert failed. so i check the source sqlcommand the output is correct
then i try to see check the output or the data passing thro the source and i see that the format of the output is not same with the destination column ProjectedROI which is decimal(2,2)
below are error messages:
[Insert New Record to Fact_Sales [162]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E21.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 11.0" Hresult: 0x80040E21 Description: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.".
[Insert New Record to Fact_Sales [162]] Error: There was an error with Insert New Record to Fact_Sales.Inputs[OLE DB Destination Input].Columns[ProjectedROI] on Insert New Record to Fact_Sales.Inputs[OLE DB Destination Input]. The column status returned was: "Conversion failed because the data value overflowed the specified type.".
[Insert New Record to Fact_Sales [162]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "Insert New Record to Fact_Sales.Inputs[OLE DB Destination Input]" failed because error code 0xC020907A occurred, and the error row disposition on "Insert New Record to Fact_Sales.Inputs[OLE DB Destination Input]" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Insert New Record to Fact_Sales" (162) failed with error code 0xC0209029 while processing input "OLE DB Destination Input" (175). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.
You have a lot of steps between your source and destination and you haven't shown what the data looks like after each one. At some point, your data is going from Decimal(2,2) to something like Decimal(14,11) if I am counting right.
Starting with the source and going to each step after until you find the culprit, right click on each component and open advanced properties, and look at the datatype of the output column ProjectedROI. Verify it has the correct Precision and Scale.

SSIS Excel Source error: [Excel Source [1]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005

I have a very simple SSIS Data Flow that Unions two Excel Sources together, does a small data conversion, and loads into a staging table.
However, my two Excel Sources are failing immediately as the Execute Phase begins.
I am only receiving the following nondescript errors:
[LA Report Source [627]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on component "LA Report Source" (627) returned error code 0xC0202009. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
[NOLA Report Source [640]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on component "NOLA Report Source" (640) returned error code 0xC0202009. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
This package had been working for several days, and suddenly started giving this error message a couple days ago, with no real changes to the data source.
In addition, when I go to preview the Excel Data Source, I get this weird error:
Any ideas? Using Visual Studio 2008.
I want to share the solution I came up with:
I was over-zealous with my SQL Query in the Excel Source. I had a WHERE filter on my select statement, which worked for a little while, but an anomaly in the data must have thrown something off - SSIS didn't like it at all.
I've removed the WHERE portion, just leaving the SELECT portion, and I've handled the filter with a Conditional Split instead. Problem solved!

SSIS Error: "Invalid character value for cast specification"

I'm trying to import an Excel file (.xls) with 18 columns; the first row has column names. The file contains 3718 rows of students personal data. This is the error I receive importing the file into a SQL Server database:
- Copying to [dbo].[masterlist$] (Error)
Messages
* Error 0xc0202009: Data Flow Task 1: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80004005 Description: "Invalid character value for cast specification".
(SQL Server Import and Export Wizard)
* Error 0xc020901c: Data Flow Task 1: There was an error with input column "dbirth" (121) on input "Destination Input" (84). The column status returned was: "Conversion failed because the data value overflowed the specified type.".
(SQL Server Import and Export Wizard)
* Error 0xc0209029: Data Flow Task 1: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "input "Destination Input" (84)" failed because error code 0xC020907A occurred, and the error row disposition on "input "Destination Input" (84)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
(SQL Server Import and Export Wizard)
* Error 0xc0047022: Data Flow Task 1: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Destination - masterlist$" (71) failed with error code 0xC0209029 while processing input "Destination Input" (84). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.
(SQL Server Import and Export Wizard)
* Error 0xc02020c4: Data Flow Task 1: The attempt to add a row to the Data Flow task buffer failed with error code 0xC0047020.
(SQL Server Import and Export Wizard)
* Error 0xc0047038: Data Flow Task 1: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on component "Source - masterlist$" (1) returned error code 0xC02020C4. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
(SQL Server Import and Export Wizard)
These are the two relevant errors:
"Invalid character value for cast specification"
"Conversion failed because the data value overflowed the specified type."
Your input (Excel) data is most likely not formatted correctly or is invalid.
Without seeing your input file and destination table it is difficult to help, but you should try researching those errors yourself.

SSIS Truncation Failure

I am getting this truncation error and am not sure how to fix it. The length is 4000 in input and output.
Error: 0xC02020C5 at Data Flow Task, Data Conversion [16]: Data conversion failed while converting column "DESCRIPTION" (99) to column "Copy of DESCRIPTION" (201). The conversion returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page.".
Error: 0xC020902A at Data Flow Task, Data Conversion [16]: The "output column "Copy of DESCRIPTION" (201)" failed because truncation occurred, and the truncation row disposition on "output column "Copy of DESCRIPTION" (201)" specifies failure on truncation. A truncation error occurred on the specified object of the specified component.
Error: 0xC0047022 at Data Flow Task, SSIS.Pipeline: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Data Conversion" (16) failed with error code 0xC020902A while processing input "Data Conversion Input" (17). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.
I ended up just ignoring the error as it was not altering my data in any way.

SSIS export to CSV file failing

I am trying to export the contents of a SQL Server 2005 table to a csv file using SSIS. In the Data Flow Task I have a OLE DB Source for the table and a Flat File Destination for the file.
When copying the data I started getting a failure on one of the column on a certain row and following some investigation found the problem was with comma's in the data below
Data Issue (nvarchar255)
errors code l075 showing,,,re test.
OLE DB Source for Comment col
Derived Column
Given that this was the issue I created a Derived Column object between the source and destination and destination objects and tried filtering out the comma's using a replace REPLACE(Comment,","," ") but the same column is still failing with the below errors.
Destination Component
Exception
[Inspection Failures Destination [206]] Error: Data conversion failed.
The data conversion for column "Comment" returned status value 4 and
status text "Text was truncated or one or more characters had no
match in the target code page.".
[Inspection Failures Destination [206]] Error: Cannot copy
or convert flat file data for column "Comment".
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED.
The ProcessInput method on component "Inspection Failures
Destination" (206) failed with error code 0xC02020A0 while
processing input "Flat File Destination Input" (207). The
identified component returned an error from the ProcessInput
method. The error is specific to the component, but the error
is fatal and will cause the Data Flow task to stop running.
There may be error messages posted before this with more
information about the failure.
[Inspecton Failures Source [128]] Error: The attempt to
add a row to the Data Flow task buffer failed with error
code 0xC0047020.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.
The PrimeOutput method on component "Inspecton Failures Source"
(128) returned error code 0xC02020C4. The component returned
a failure code when the pipeline engine called PrimeOutput().
The meaning of the failure code is defined by the component,
but the error is fatal and the pipeline stopped executing.
There may be error messages posted before this with more
information about the failure.
Ok, the problem actually appears to be a hidden illegal character in the text
In the image below the top line shows a square before the re test string. The comment column in the database is an nvarchar which apparently uses a different character set so I can not just use the CHAR(13) + CHAR(10) to replace the carriage return.
The fix involved converting the field from an nvarchar to a varchar then performing a replace on the converter ? character resulting in the corrected second ling in the image
SELECT ID,
REPLACE(REPLACE(CAST(Comment AS varchar(255)),'?',' '),',',' ') Comment
FROM tblInspectionFailures WHERE (ID = 216899)
The conversion requirement is detailed here
This does not should like an ideal solution to me but it does work. Does anyone have any other options.
Without replacing comment column can you create another column and map the new derived column to destination column and see.