so I'm using SQLite in a VB.net project with a populated database. I'm using the Microsoft.Data.Sqlite.Core and System.Data.SQLite NuGet package libraries. So the problem presents when I'm trying to get the result of a query. At first the SQLiteDataReader gets the response and all the elements of the desired table. I know this cause in the debugger I have a breakpoint after the setting the object and when I check the parameters of the SQLiteDataReader object the Result View shows all the elements of my table, but as soon as I remove the mouse from the object and check it again the Result View turns out empty, without even resuming with the next line of code. Does anyone know if its a known bug or something cause Ive used this exact method of querying a table in another project and it works.
The code:
Public Function RunQuery(com As String)
If CheckConnection() Then
command.CommandText = com
Dim response As SQLiteDataReader
response = command.ExecuteReader
Dim len As Integer = response.StepCount
Dim col As Integer = response.FieldCount
Dim resp(len, col) As Object
For i = 0 To col - 1
Using response
response.Read()
For j = 0 To len - 1
resp(i, j) = response.GetValue(j)
Next
End Using
Next
Debugger with populated result view
Debugger with empty result view
edit: added the for loop to show that its not only on the debugger that the result view is empty. When using response.Read() it throws an exception "System.InvalidOperationException: 'No current row'"
As I have told you in the comment, a DataReader derived class is a forward only retrieval object. This means that when you reach the end of the records returned by the query, that object is not capable to restart from the beginning.
So if you force the debugger to enumerate the view the reader reaches the end of the results and a second attempt to show the content of the reader fails.
The other part of your problem is caused by a misunderstanding on how to work on the reader. You should loop over the Read result not using a StepCount property. This property as far as I know, is not standard and other data providers don't support it. Probably it is present in the SQLite Provider because for them it is relatively easy to count the number of records while other providers don't attempt do calculate that value for performance reasons.
However, there are other ways to read from the reader. One of them is to fill a DataTable object with its Load method that conveniently take a DataReader
Dim data As DataTable = New DataTable()
Using response
data.Load(response)
End Using
' Now you have a datatable filled with your data.
' No need to have a dedicated bidimensional array
A DataTable is like an array where you have Rows and Columns instead of indexes to iterate over.
I currently use a TableAdapter to fill() a typed datatable in a typed dataset. ClearBeforeFill is set to True as the underlying data is different each time. The problem is that in the case the database is unreachable, the old data gets cleared before it knows. I want the old data to stay in the datatable (and subsequently stay displayed) in the case an error occurs retrieving from the database. Ideally, I'd still use a tableAdapter and the GetData() method instead of the Fill() method but I can't seem to figure out how to replace the current datatable with the one returned by GetData(). Tables.Remove() then Tables.Add() doesn't seem to work. Below code gives an error: "'table' argument cannot be null." & vbCrLf & "Parameter name: table"
Dim TempTbl As DS_ERecord.DT_spRefreshAndSelectStepConnectorMonitorDataTable
TempTbl = TAi_spRefreshAndSelectStepConnectorMonitor.GetData(ForceRefresh, CmbMonitorView.SelectedValue)
DSi_ERecord.Tables.Remove(DSi_ERecord.DT_spRefreshAndSelectStepConnectorMonitor)
DSi_ERecord.Tables.Add(TempTbl)
I am serializing some data to xml, then writing the output to an Oracle DB.
The serialized data can be very long and is writing to a column type of LONG (it never will reach the max value of LONG, which is something like 32,000 chars)
Sometimes when my data is written, it appears to corrupt and just display "?" and lots of other control characters, sometimes the write works perfectly.
The function i am using always returns the XML with no problem, the problem occurs when the data is written.
This is my serializer function:
Private Function Serialize(myObject As object)
//serialize the object to a string...
Dim x = New System.Xml.Serialization.XmlSerializer(myObject .[GetType]())
Dim stringWriter = New StringWriter()
x.Serialize(stringWriter, myObject )
Dim test = stringWriter.ToString()
Return stringWriter.ToString()
stringWriter.Close()
End Function
I am then writing the datawhich is returned to the DB using nHibernate, i will not include this code as it is very long and has worked without fail for a long time. The problem seems to be with how Oracle is interpreting the data.
My unitofwork commits without any error - just the data is corrupt when oracle receives it.
UPDATE
If i copy the stringwriter output, then paste this into the column in the database i can commit without a problem
As advised by wolφi in the comments.
The fix for this was to use CLOB as the column data type. XMLType did not exist in my version of Oracle.
I am using following code to get data from database and load to picture box
Dim vrPicFromDB = IO.File.ReadAllBytes(DsPic.tblPicTest.Item("Picture"))
Dim ms As New MemoryStream(vrPicFromDB)
PictureBox1.Image = Image.FromStream(ms)
It gives error on DsPic.tblPicTest.Item("Picture")) portion of the statement.
I also tried
CByte(DsPic.tblPicTest.Item("Picture")))
but it gives the same error.
Please advise how to fix it.
Thanks
Furqan
Assuming that the "Picture" column is an Image column in the database, your line to load the vrPicFromDB byte array will look something like this:
Dim vrPicFromDB As Byte()
vrPicFromDB = CType(DsPic.tblPicTest.Rows(0).Item("Picture"), Byte())
That line assume you have loaded at least 1 row of data.
In my SSIS package, I want to see if a particular Oracle view has data in it.
I have a Execute SQL Task with a select count statement:
Select CAST( count(*) as Integer) As Row_Count from OWNER.VIEW_MV
My RowCount is in the Result Set, pointing to my Variable User:TableCount. If I set TableCount to anything except Object, I get an error such as:
Error: 0xC002F309 at Check for data in
view, Execute SQL Task: An error
occurred while assigning a value to
variable "TableCount": "The type of
the value being assigned to variable
"User::TableCount" differs from the
current variable type. Variables may
not change type during execution.
Variable types are strict, except for
variables of type Object.
However, if I do make the data type to Object, then in my next step, a Script Task, when I read that object, I don't see how to convert it to an integer so that I can use and view it. Then I get the error:
Conversion from type 'Object' to type 'Integer' is not valid.
with the code
Dim nTableCount As Int32
nTableCount = CInt(Dts.Variables("TableCount").Value)
Perhaps I am going about this wrong. What is the best way to determine if an Oracle table is empty, and then act on that knowledge? Essentially, we want an error message about the empty table, rather than continuing on in our process.
Update:
I've tried Select CAST( count(*) as char) As Row_Count from OWNER.VIEW_MV where ROWNUM < 2, and sending it to a SSIS variable of type char. I've cast it to Numeric and Integer, with SSIS variables of Int32, Int64. I've tried varchar2(20) with SSIS type String. Everything gives an error except for SSIS datatype Object.
Right now, I'm saving as datatype Object, but when I try to get the value, setting my nTableCount as String, I can use nTableCount = Dts.Variables("TableCount").Value().ToString(), but it returns 0. How do I extract that string out of the Object variable? (Assuming of course, that it actually put it in there.)
I know nothing about SSIS so can't help you there, but, as a note, if you only want to check for the presence of data in a table, it's more efficient to include a ROWNUM clause in the SQL. e.g.
SELECT COUNT(*)
FROM table
WHERE ROWNUM < 2;
(will return 0 if the table is empty, 1 if any rows are in the table)
In this way Oracle can stop reading the results from the table/view as soon as it finds any rows, thus (potentially) finishing execution of the query much sooner.
SSIS has a lot of trouble talking to Oracle. I have faced similar issues before. Did you try using string data type? You will not have trouble converting string to integer in script.
I found a solution. It may or may not be the best solution, but it will work. Other solutions and improvements are always welcome.
First, rather than having a SQL Task, I have a Data Flow task with the Oracle source and the Data Access Mode is a SQL Command with the original shown in the question (although, I'll probably use cagcowboy's advice once this is working properly).
The output is a destination Script Transformation. I selected ROW_COUNT as my input column, didn't mess with the Inputs and Outputs, and for the Script selected my TableCount as ReadWriteVariables. My script is as below:
Public Class ScriptMain
Inherits UserComponent
Dim ScriptCount As Single
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
' Assign the variable to something outside of the sub
ScriptCount = Row.ROWCOUNT
End Sub
Public Overrides Sub PostExecute()
' Variable is only available to set in PostExecute
Me.Variables.TableCount = ScriptCount
End Sub
End Class
My public variable is of type Double, and in my final script task (outside of the data flow), I simply access it with the lines
Dim nTableCount As String
nTableCount = Dts.Variables("TableCount").Value().ToString()
I suspect I should do a bit more with the data types, because it doesn't seem like it should need to be converted to a string at this point. But that's ok, I can now determine if there is data in the view, and fork accordingly.
this is what worked for me. The idea is to convert count(*) to char and use String type variable in SSIS:
SELECT TO_CHAR(COUNT(*)) FROM yourtable;