I am serializing some data to xml, then writing the output to an Oracle DB.
The serialized data can be very long and is writing to a column type of LONG (it never will reach the max value of LONG, which is something like 32,000 chars)
Sometimes when my data is written, it appears to corrupt and just display "?" and lots of other control characters, sometimes the write works perfectly.
The function i am using always returns the XML with no problem, the problem occurs when the data is written.
This is my serializer function:
Private Function Serialize(myObject As object)
//serialize the object to a string...
Dim x = New System.Xml.Serialization.XmlSerializer(myObject .[GetType]())
Dim stringWriter = New StringWriter()
x.Serialize(stringWriter, myObject )
Dim test = stringWriter.ToString()
Return stringWriter.ToString()
stringWriter.Close()
End Function
I am then writing the datawhich is returned to the DB using nHibernate, i will not include this code as it is very long and has worked without fail for a long time. The problem seems to be with how Oracle is interpreting the data.
My unitofwork commits without any error - just the data is corrupt when oracle receives it.
UPDATE
If i copy the stringwriter output, then paste this into the column in the database i can commit without a problem
As advised by wolφi in the comments.
The fix for this was to use CLOB as the column data type. XMLType did not exist in my version of Oracle.
Related
so I'm using SQLite in a VB.net project with a populated database. I'm using the Microsoft.Data.Sqlite.Core and System.Data.SQLite NuGet package libraries. So the problem presents when I'm trying to get the result of a query. At first the SQLiteDataReader gets the response and all the elements of the desired table. I know this cause in the debugger I have a breakpoint after the setting the object and when I check the parameters of the SQLiteDataReader object the Result View shows all the elements of my table, but as soon as I remove the mouse from the object and check it again the Result View turns out empty, without even resuming with the next line of code. Does anyone know if its a known bug or something cause Ive used this exact method of querying a table in another project and it works.
The code:
Public Function RunQuery(com As String)
If CheckConnection() Then
command.CommandText = com
Dim response As SQLiteDataReader
response = command.ExecuteReader
Dim len As Integer = response.StepCount
Dim col As Integer = response.FieldCount
Dim resp(len, col) As Object
For i = 0 To col - 1
Using response
response.Read()
For j = 0 To len - 1
resp(i, j) = response.GetValue(j)
Next
End Using
Next
Debugger with populated result view
Debugger with empty result view
edit: added the for loop to show that its not only on the debugger that the result view is empty. When using response.Read() it throws an exception "System.InvalidOperationException: 'No current row'"
As I have told you in the comment, a DataReader derived class is a forward only retrieval object. This means that when you reach the end of the records returned by the query, that object is not capable to restart from the beginning.
So if you force the debugger to enumerate the view the reader reaches the end of the results and a second attempt to show the content of the reader fails.
The other part of your problem is caused by a misunderstanding on how to work on the reader. You should loop over the Read result not using a StepCount property. This property as far as I know, is not standard and other data providers don't support it. Probably it is present in the SQLite Provider because for them it is relatively easy to count the number of records while other providers don't attempt do calculate that value for performance reasons.
However, there are other ways to read from the reader. One of them is to fill a DataTable object with its Load method that conveniently take a DataReader
Dim data As DataTable = New DataTable()
Using response
data.Load(response)
End Using
' Now you have a datatable filled with your data.
' No need to have a dedicated bidimensional array
A DataTable is like an array where you have Rows and Columns instead of indexes to iterate over.
I have added dummy data to my SQL database.
When I try to run the VB it comes back with a null error.
What I have done is delete the dummy data from the database so it not reading the null value anymore.
Even after deleting it the VB file is still throwing the error like the null values are still there.
I have run a sql script to check and I can confirm it not longer there.
Here is the line of code that throwing the error.
If Date.Now.AddDays(LoadsFilter * -1) > Convert.ToDateTime(myReader(2)) Then
ShowLoad = 0
End If
I'm still quite new to vb so I'm not sure what to do here.
I was thinking of passing a method to read null values but I have already deleted the null value. I'm not sure how to go about this, can anyone help please?
There's no need for any conversion. The data reader has its own methods for null tests and for getting data in the appropriate type, e.g.
If Not myDataReader.IsDBNull(2) Then
Dim dt = myDataReader.GetDateTime(2)
'Use dt here.
End If
You can also use a single line to get a Nullable(Of Date):
Dim dt = If(myDataReader.IsDBNull(2), New Date?, myDataReader.GetDateTime(2))
Note that, if you prefer to use column names rather than indexes, which make your code clearer, then you can use the GetOrdinal method, as those other methods only support indexes:
Dim dt = If(myDataReader.IsDBNull(myDataReader.GetOrdinal("ColumnName")),
New Date?,
myDataReader.GetDateTime(myDataReader.GetOrdinal("ColumnName")))
I'm trying to save a local document stored in SQL db using UiPath.
From the execute query activity it returns a DataTable value with one column(“Data”).
If I'm not wrong somehow I need to transform the DataTable.rows(0)(“Data”) (studio is interpreting it as object) to an array of bytes.
I used invoke code but the file is broken:
Dim _MemoryStream As New System.IO.MemoryStream()
Dim _BinaryFormatter As New System.Runtime.Serialization.Formatters.Binary.BinaryFormatter()
_BinaryFormatter.Serialize(_MemoryStream, inputObj)
_MemoryStream.ToArray()
File.WriteAllBytes(“C:\Users\Desktop\test.tiff”, _MemoryStream.ToArray())
Unless you serialised an object when saving, which I doubt, there's no reason to try to deserialise an object when retrieving. Every field from a DataRow is returned as an Object reference, because it must support any type of object, but that object still has its own type. If you saved binary data then that type will be Byte(), i.e. a Byte array. Kinda looks like File.WriteAllBytes takes a Byte array:
File.WriteAllBytes(“C:\Users\Desktop\test.tiff”, DirectCast(inputObj, Byte()))
That is what I would expect that you need to do. It does depend on how you saved the data in the first place but if you used File.ReadAllBytes or Image.Save then this should do the job. If not, show us how you sabed the data in the first place.
I've Googled the underlying question of how to store an image in the database and found many examples. However, all of the answers I'm seeing (for example this), seem to use a SQLCommand as the solution whether they are using the older "image" type, text, or suggesting VARCHAR(MAX). I am using SQL 2008 and SQL 2012
I'm having trouble incorporating this into my app since I have a legacy dll which will do some other work, and then store my image. I want to wrap this into a SQL transaction. The challenge is that the DLL is essentially a black-box to me since I cannot change the source code. The Dll exposes a parameter for an optional SQL statement it can run as part of its transaction.
The structure of the DLL is something like this:
Public Class MyClass
Public Property ExtraSQL as string
Public Function DoWork
Dim sMainSQL as String
'Start A transaction
Execute(sMainSQL)
If ExtraSQL <> "" Then
Execute(ExtraSQL)
End If
If OK Then Commit Transaction
End Function
End Class
My goal is to feed a SQL string into property ExtraSQL, but I'm not sure how to do this as all the examples are indicating something SqlCommand.Parameters(#Image).Value = byteArray(). I'm getting the image data from a web service. Over its life cycle, it's formatted as an Image, a base64 Encoded string, a memorystream and a byte array, so converting between the types is not the problem. I'm just unclear on the cleanest syntax for doing this. Is it best to just take my original base64 string and do something like:
INSERT INTO tblImage (ImageData) Values ('Base64 blah blah')
and then whenever I read the image back out do this:
Public Function Base64ToImage(ByVal base64String As String) As System.Drawing.Image
Dim img As System.Drawing.Image = Nothing
Try
'Convert Base64 string to byte array
Dim btImage As Byte() = Convert.FromBase64String(base64String)
Dim ms As New IO.MemoryStream(btImage, 0, btImage.Length)
ms.Write(btImage, 0, btImage.Length)
img = System.Drawing.Image.FromStream(ms, True)
Catch ex As Exception
End Try
Return img
End Function
I would appreciate any supporting links, examples, and advice. Thank you in advance.
Well, considering you clearly can't send in parameterized SQL, yes that's the best approach. I sure don't like it because it's wide open to SQL injection. I would consider decompiling the assembly you're using and making it my own source code in the future.
In my SSIS package, I want to see if a particular Oracle view has data in it.
I have a Execute SQL Task with a select count statement:
Select CAST( count(*) as Integer) As Row_Count from OWNER.VIEW_MV
My RowCount is in the Result Set, pointing to my Variable User:TableCount. If I set TableCount to anything except Object, I get an error such as:
Error: 0xC002F309 at Check for data in
view, Execute SQL Task: An error
occurred while assigning a value to
variable "TableCount": "The type of
the value being assigned to variable
"User::TableCount" differs from the
current variable type. Variables may
not change type during execution.
Variable types are strict, except for
variables of type Object.
However, if I do make the data type to Object, then in my next step, a Script Task, when I read that object, I don't see how to convert it to an integer so that I can use and view it. Then I get the error:
Conversion from type 'Object' to type 'Integer' is not valid.
with the code
Dim nTableCount As Int32
nTableCount = CInt(Dts.Variables("TableCount").Value)
Perhaps I am going about this wrong. What is the best way to determine if an Oracle table is empty, and then act on that knowledge? Essentially, we want an error message about the empty table, rather than continuing on in our process.
Update:
I've tried Select CAST( count(*) as char) As Row_Count from OWNER.VIEW_MV where ROWNUM < 2, and sending it to a SSIS variable of type char. I've cast it to Numeric and Integer, with SSIS variables of Int32, Int64. I've tried varchar2(20) with SSIS type String. Everything gives an error except for SSIS datatype Object.
Right now, I'm saving as datatype Object, but when I try to get the value, setting my nTableCount as String, I can use nTableCount = Dts.Variables("TableCount").Value().ToString(), but it returns 0. How do I extract that string out of the Object variable? (Assuming of course, that it actually put it in there.)
I know nothing about SSIS so can't help you there, but, as a note, if you only want to check for the presence of data in a table, it's more efficient to include a ROWNUM clause in the SQL. e.g.
SELECT COUNT(*)
FROM table
WHERE ROWNUM < 2;
(will return 0 if the table is empty, 1 if any rows are in the table)
In this way Oracle can stop reading the results from the table/view as soon as it finds any rows, thus (potentially) finishing execution of the query much sooner.
SSIS has a lot of trouble talking to Oracle. I have faced similar issues before. Did you try using string data type? You will not have trouble converting string to integer in script.
I found a solution. It may or may not be the best solution, but it will work. Other solutions and improvements are always welcome.
First, rather than having a SQL Task, I have a Data Flow task with the Oracle source and the Data Access Mode is a SQL Command with the original shown in the question (although, I'll probably use cagcowboy's advice once this is working properly).
The output is a destination Script Transformation. I selected ROW_COUNT as my input column, didn't mess with the Inputs and Outputs, and for the Script selected my TableCount as ReadWriteVariables. My script is as below:
Public Class ScriptMain
Inherits UserComponent
Dim ScriptCount As Single
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
' Assign the variable to something outside of the sub
ScriptCount = Row.ROWCOUNT
End Sub
Public Overrides Sub PostExecute()
' Variable is only available to set in PostExecute
Me.Variables.TableCount = ScriptCount
End Sub
End Class
My public variable is of type Double, and in my final script task (outside of the data flow), I simply access it with the lines
Dim nTableCount As String
nTableCount = Dts.Variables("TableCount").Value().ToString()
I suspect I should do a bit more with the data types, because it doesn't seem like it should need to be converted to a string at this point. But that's ok, I can now determine if there is data in the view, and fork accordingly.
this is what worked for me. The idea is to convert count(*) to char and use String type variable in SSIS:
SELECT TO_CHAR(COUNT(*)) FROM yourtable;