In my SSIS package, I want to see if a particular Oracle view has data in it.
I have a Execute SQL Task with a select count statement:
Select CAST( count(*) as Integer) As Row_Count from OWNER.VIEW_MV
My RowCount is in the Result Set, pointing to my Variable User:TableCount. If I set TableCount to anything except Object, I get an error such as:
Error: 0xC002F309 at Check for data in
view, Execute SQL Task: An error
occurred while assigning a value to
variable "TableCount": "The type of
the value being assigned to variable
"User::TableCount" differs from the
current variable type. Variables may
not change type during execution.
Variable types are strict, except for
variables of type Object.
However, if I do make the data type to Object, then in my next step, a Script Task, when I read that object, I don't see how to convert it to an integer so that I can use and view it. Then I get the error:
Conversion from type 'Object' to type 'Integer' is not valid.
with the code
Dim nTableCount As Int32
nTableCount = CInt(Dts.Variables("TableCount").Value)
Perhaps I am going about this wrong. What is the best way to determine if an Oracle table is empty, and then act on that knowledge? Essentially, we want an error message about the empty table, rather than continuing on in our process.
Update:
I've tried Select CAST( count(*) as char) As Row_Count from OWNER.VIEW_MV where ROWNUM < 2, and sending it to a SSIS variable of type char. I've cast it to Numeric and Integer, with SSIS variables of Int32, Int64. I've tried varchar2(20) with SSIS type String. Everything gives an error except for SSIS datatype Object.
Right now, I'm saving as datatype Object, but when I try to get the value, setting my nTableCount as String, I can use nTableCount = Dts.Variables("TableCount").Value().ToString(), but it returns 0. How do I extract that string out of the Object variable? (Assuming of course, that it actually put it in there.)
I know nothing about SSIS so can't help you there, but, as a note, if you only want to check for the presence of data in a table, it's more efficient to include a ROWNUM clause in the SQL. e.g.
SELECT COUNT(*)
FROM table
WHERE ROWNUM < 2;
(will return 0 if the table is empty, 1 if any rows are in the table)
In this way Oracle can stop reading the results from the table/view as soon as it finds any rows, thus (potentially) finishing execution of the query much sooner.
SSIS has a lot of trouble talking to Oracle. I have faced similar issues before. Did you try using string data type? You will not have trouble converting string to integer in script.
I found a solution. It may or may not be the best solution, but it will work. Other solutions and improvements are always welcome.
First, rather than having a SQL Task, I have a Data Flow task with the Oracle source and the Data Access Mode is a SQL Command with the original shown in the question (although, I'll probably use cagcowboy's advice once this is working properly).
The output is a destination Script Transformation. I selected ROW_COUNT as my input column, didn't mess with the Inputs and Outputs, and for the Script selected my TableCount as ReadWriteVariables. My script is as below:
Public Class ScriptMain
Inherits UserComponent
Dim ScriptCount As Single
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
' Assign the variable to something outside of the sub
ScriptCount = Row.ROWCOUNT
End Sub
Public Overrides Sub PostExecute()
' Variable is only available to set in PostExecute
Me.Variables.TableCount = ScriptCount
End Sub
End Class
My public variable is of type Double, and in my final script task (outside of the data flow), I simply access it with the lines
Dim nTableCount As String
nTableCount = Dts.Variables("TableCount").Value().ToString()
I suspect I should do a bit more with the data types, because it doesn't seem like it should need to be converted to a string at this point. But that's ok, I can now determine if there is data in the view, and fork accordingly.
this is what worked for me. The idea is to convert count(*) to char and use String type variable in SSIS:
SELECT TO_CHAR(COUNT(*)) FROM yourtable;
Related
I was wondering if there is any way to access the expected data type within a function similar to an event arg. I am doubtful that this is possible, though it would be an excellent feature.
I frequently work with (old and disorganized)Mysql databases creating interfaces through VB.Net. Often I will have an optional field which contains a NULL value in the database. I am frequently dealing with errors due to NULL and dbnull values in passing data to and from the database.
To complicate things, I often am dealing with unexpected datatypes. I might have an integer zero, a double zero, an empty string, or a string zero.
So I spend a fair amount of code checking that each entry is of the expected type and or converting NULLs to zeros or empty strings depending on the case. I have written a function ncc(null catch convert) to speed up this process.
Public Function ncc(obj As Object, tp As Type) As Object 'Null Catch Convert Function...
My function works great, but I have to manually set the type every time I call the function. It would be so much easier if it were possible to access the expected type of the expression. Here is an example of what I mean.
Dim table as datatable
adapter.fill(table)
dim strinfo as string
dim intinfo as long
strinfo = ncc(table.Rows(0).Item(0),gettype(String)) 'here a string is expected
intinfo = ncc(table.Rows(0).Item(0),gettype(Long)) 'here a long is expected
It would be so much more efficient if it were possible to access the expected type directly from the function.
Something like this would be great:
Public Function ncc(obj As Object, optional tp As Type = nothing) As Object
If tp Is Nothing Then tp = gettype(ncc.expectedtype)
That way I do not have to hard code the type on each line.
strinfo = ncc(table.Rows(0).Item(0))
You can make the ncc function generic to simplify calling it:
Public Function ncc(Of T)(obj As T) As T
If DbNull.Value.Equals(obj) Then Return Nothing
Return Obj
End Function
This kind of function will be able to in some cases infer the type, but if there's any possibility of null you'll still want to include a type name (because DBNull will be the inferred type for those values). The advantage is not needing to call gettype() and so gaining a small degree of type safety:
strinfo = ncc(Of String)(table.Rows(0).Item(0))
But I think this has a small chance to blow up at run time if your argument is not implicitly convertible to the desired type. What you should be doing is adding functions to accept a full row and return a composed type. These functions can exist as static/shared members of the target type:
Shared Function FromDataRow(IDataRow row) As MyObject
And you call it for each row like this:
Dim record As MyObject = MyObject.FromDataRow(table.Rows(i))
But, you problem still exists.
What happens if the column in the database row is null?
then you DO NOT get a data type!
Worse yet? Assume the data column is null, do you want to return null into that variable anyway?
Why not specify a value FOR WHEN its null.
You can use "gettype" on the passed value, but if the data base column is null, then you can't determine the type, and you right back to having to type out the type you want as the 2nd parameter.
You could however, adopt a nz() function (like in VBA/Access).
So, this might be better:
Public Function ncc(obj As Object, Optional nullv As Object = Nothing) As Object
If obj Is Nothing OrElse IsDBNull(obj) Then
Return nullv
End If
Return obj
End Function
So, I don't care if the database column is null, or a number, for such numbers, I want 0.
So
dim MyInt as integer
Dim MyDouble As Double
MyInt = ncc(rstData.Rows(0).Item("ContactID"), 0)
MyDouble = ncc(rstData.Rows(0).Item("ContactID"), 0)
dim strAddress as string = ""
strAddress = ncc(rstData.Rows(0).Item("Address"), "")
Since in NEAR ALL cases, you need to deal with the null from the DB, then above not only works for all data types, but also gets you on the fly conversion.
I mean, you CAN declare variables such as integer to allow null values.
eg:
dim myIntValue as integer?
But, I not sure above would create more problems than it solves.
So,
You can't get exactly what you want, because a function never has knowledge of how it's going to be used. It's not guaranteed that it will be on the right-hand side of an assignment statement.
If you want to have knowledge of both sides, you either need to be assigning to a custom type (so that you can overload the assignment operator) or you need to use a Sub instead of an assignment.
You could do something like this (untested):
Public Sub Assign(Of T)(ByVal field As Object, ByRef destination As T,
Optional ByVal nullDefault As T = Nothing)
If TypeOf field Is DBNull Then
destination = nullDefault
Else
destination = CType(field, T)
End If
End Sub
I haven't tested this, so I'm not completely certain that the compiler would allow the conversion, but I think it would because field is type Object. Note that this would yield a runtime error if field is not convertible to T.
You could even consider putting on a constraint requiring T to be a value type, though I don't think that would be likely to work because you probably need to handling String which is a reference type (even though it basically acts like a value type).
Because the destination is an argument, you wouldn't ever need to specify the generic type argument, it would be inferred.
I have added dummy data to my SQL database.
When I try to run the VB it comes back with a null error.
What I have done is delete the dummy data from the database so it not reading the null value anymore.
Even after deleting it the VB file is still throwing the error like the null values are still there.
I have run a sql script to check and I can confirm it not longer there.
Here is the line of code that throwing the error.
If Date.Now.AddDays(LoadsFilter * -1) > Convert.ToDateTime(myReader(2)) Then
ShowLoad = 0
End If
I'm still quite new to vb so I'm not sure what to do here.
I was thinking of passing a method to read null values but I have already deleted the null value. I'm not sure how to go about this, can anyone help please?
There's no need for any conversion. The data reader has its own methods for null tests and for getting data in the appropriate type, e.g.
If Not myDataReader.IsDBNull(2) Then
Dim dt = myDataReader.GetDateTime(2)
'Use dt here.
End If
You can also use a single line to get a Nullable(Of Date):
Dim dt = If(myDataReader.IsDBNull(2), New Date?, myDataReader.GetDateTime(2))
Note that, if you prefer to use column names rather than indexes, which make your code clearer, then you can use the GetOrdinal method, as those other methods only support indexes:
Dim dt = If(myDataReader.IsDBNull(myDataReader.GetOrdinal("ColumnName")),
New Date?,
myDataReader.GetDateTime(myDataReader.GetOrdinal("ColumnName")))
I am running an SQL Query with data readers in vb.net
reader3 = myCommand3.ExecuteReader
reader3.Read()
If reader3.HasRows Then
FromToDates = reader3.GetString(3)
End If
There are no rows being returned in this particular query, however even though the if reader3.HasRows is being used its still showing an error saying:
Additional information: Data is Null. This method or property cannot be called on Null values.
this is when trying to set the FromToDates variable but there are no rows being returned so it should not even reach this part
I doubt that there are no rows, i assume that there is at least one row where the value is NULL. You could check it with the IsDBNull-method:
If reader3.HasRows AndAlso Not reader3.IsDbNull(3) Then
FromToDates = reader3.GetString(3)
End If
However, the variable name suggests that it's a date, but you are storing it as string. Use the correct type which is date or datetime. If it's actually a datetime use:
Dim FromToDates As Date
If reader3.HasRows AndAlso Not reader3.IsDbNull(3) Then
FromToDates = reader3.GetDateTime(3)
End If
if you are using Reader.read() then i don't think so you need reader,hasrow(). you can do somthing like this
reader3 = myCommand3.ExecuteReader
If reader3.Read()Then
FromToDates = reader3.GetString(3)
End If
and related to your error its look like you are getting NULL value from database and in vb you can not read NULL, i will suggest to use ISNULL(Column_NAME,'') function in your SQL Script.
It seems that even though the row is empty (all DBNull, perhaps), there IS a row object returning from the query. There are several different solutions presented in this thread that all take differing approaches to catching the empty row before it errors!
http://www.vbforums.com/showthread.php?555183-datareader-and-empty-records
I am serializing some data to xml, then writing the output to an Oracle DB.
The serialized data can be very long and is writing to a column type of LONG (it never will reach the max value of LONG, which is something like 32,000 chars)
Sometimes when my data is written, it appears to corrupt and just display "?" and lots of other control characters, sometimes the write works perfectly.
The function i am using always returns the XML with no problem, the problem occurs when the data is written.
This is my serializer function:
Private Function Serialize(myObject As object)
//serialize the object to a string...
Dim x = New System.Xml.Serialization.XmlSerializer(myObject .[GetType]())
Dim stringWriter = New StringWriter()
x.Serialize(stringWriter, myObject )
Dim test = stringWriter.ToString()
Return stringWriter.ToString()
stringWriter.Close()
End Function
I am then writing the datawhich is returned to the DB using nHibernate, i will not include this code as it is very long and has worked without fail for a long time. The problem seems to be with how Oracle is interpreting the data.
My unitofwork commits without any error - just the data is corrupt when oracle receives it.
UPDATE
If i copy the stringwriter output, then paste this into the column in the database i can commit without a problem
As advised by wolφi in the comments.
The fix for this was to use CLOB as the column data type. XMLType did not exist in my version of Oracle.
Okay so i have a macro that goes in and checks the table to determine the format of a application that is set by the user. If this format is set to be a European format then it runs a function that will go through a table and swap the date formatting from MM/dd/yyyy to dd/MM/yyyy. everything seems to be set up correctly and when it's in US formatting the macro runs correctly(which it should considering it skips over this line if the formatting is not set to European.) however, whenever the formatting is set to European, i get this run-time error that pops up:
Run-Time error '3052':
File sharing lock count exceeded. Increase MaxLocksPerFile registry entry.
Now i'm always wary whenever i get an error that states the fix is to modify something in the registy. Not to mention this is not something i would want a client to be doing just to get the application to work. However i gave it a shot just to see if that would indeed fix the problem by following these steps. Although when i did modify the default MaxLocksPerFile from it's default(9500) to (30,000) and ran the macro again i still got the same error at a slightly larger count.
Before i would get the error after running through about 12,000 rows, after the change it would come up around 15,0000 rows.
This leads me to believe that i am incorrectly closing out my updates after each row is modified.
This is my code below:
Public Function UKDateFormat() As Variant
Dim varPieces As Variant
Dim strNew As String
Dim varReturn As Variant
Dim Strsql As String
Dim db As dao.Database
Dim rstAlarmdetDateMod As dao.Recordset
Dim i As Long
Set db = CurrentDb()
Strsql = "select AlarmDate From AlarmdetDateMods;"
Set rstAlarmdetDateMod = db.OpenRecordset(Strsql, dbOpenDynaset)
If (rstAlarmdetDateMod.RecordCount > 0) Then
rstAlarmdetDateMod.MoveFirst
i = 0
While (rstAlarmdetDateMod.EOF) = False
i = i + 1
rstAlarmdetDateMod.Edit
rstAlarmdetDateMod![alarmdate] = CDate(Format(rstAlarmdetDateMod![alarmdate], "dd/MM/yyyy"))
rstAlarmdetDateMod.Update
rstAlarmdetDateMod.MoveNext
Wend
End If
rstAlarmdetDateMod.Close
db.Close
End Function
This is my Update query:
UPDATE DISTINCTROW AlarmdetDateMods SET AlarmdetDateMods.AlarmDate = CDate(Format([AlarmDate],"dd/mm/yyyy"));
What i would like to know is: What exactly i am doing wrong with my function that it causes this error to come up and how can i correct it in a way so that i don't have to go into and modify the registry to get this function to work?
Any help or suggestions are greatly appreciated.
Thanks.
In addition to your UKDateFormat function, you showed us this UPDATE statement.
UPDATE DISTINCTROW AlarmdetDateMods
SET AlarmdetDateMods.AlarmDate = CDate(Format([AlarmDate],"dd/mm/yyyy"));
However, it's not clear what's going on with that. Does it work? Does it fail with the same error as the UKDateFormat function? A different error?
Try a revised UPDATE statement. I think DISTINCTROW is not useful in this case; suggest you discard it.
Also your UPDATE seems to rely on implicit data type conversions. AlarmDate is a text value. You pass that text to the Format() function, to treat it as a Date/Time value and transform it to a different formatted string. Then you ask CDate() convert that string value to a Date/Time value. Finally the Date/Time value is stored back to the AlarmDate text field.
At a minimum I would avoid casting the formatted string back to a Date/Time value before storing it in the text field. However, I would also make the data type conversions explicit rather than implicit. And limit the UPDATE attempt to only those rows where AlarmDate holds the text representation of a valid Date/Time value.
UPDATE AlarmdetDateMods
SET AlarmDate = Format(CDate(AlarmDate),"dd/mm/yyyy")
WHERE IsDate(AlarmDate) = True;
Note this suggestion assumes changing all your stored AlarmDate text values is reasonable. I have doubts about that. Seems like maybe AlarmDate should be Date/Time rather than text. And if you need to change the format when displaying those Date/Time values, do it with the format property of a bound control on a form or with the Format() function in a query.