EDIT: See Below
I have a web service which uses a class of functions in order to return data used in various business processes (via InfoPath).
One of the functions takes a given SQLCommand object and executes it into a SQLDataReader. Now depending on the SQL command text used this may return one or many rows of one or many columns. So what is the best class for this function to return bearing in mind it needs to be serialized by the web service.
My existing code is:
Dim array As New ArrayList
Try
conCMS.Open()
Dim rdr As SqlDataReader = cmdCurrent.ExecuteReader
While rdr.Read
If rdr.VisibleFieldCount > 1 Then
Dim complexType(rdr.VisibleFieldCount - 1) As String
For rowIndex As Integer = 0 To rdr.VisibleFieldCount - 1
complexType(rowIndex) = rdr(rowIndex)
Next
array.Add(complexType)
Else
array.Add(rdr(0))
End If
End While
conCMS.Close()
Return array
Catch ex As Exception
array.Add("ERROR " & ex.Message)
End Try
Return Nothing
Now I know this is not efficient code, but this a work in progress.
As you can probably see this is generating a string array to represent a row with more than one column, however this cannot be serialized by the web service.
So 2 things really;
Some guidance on an effective type to use (without writing a serializable class of my own)
Some advice on improving the code going forward.
Thanks in advance
EDIT:
I have managed to get serialization to work by simply creating a nested arrayList as follows (doh!):
If rdr.VisibleFieldCount > 1 Then
Dim complexType As New ArrayList
For rowIndex As Integer = 0 To rdr.VisibleFieldCount - 1
complexType.Add(rdr(rowIndex))
Next
array.Add(complexType)
Else
array.Add(rdr(0))
End If
However please let me know how this could be improved.
If your code knows the schema of the returned data before the call, then you should return data in the same shape. Have a struct or class with properties of the appropriate type for each column of the returned data. For each row, create an instance of such a struct, and fill in the properties from the returned columns. Then add each instance to a strongly-typed list of that struct, a List(Of T). Then return the list.
OBTW, ArrayList was created before we had generics in .NET. Today, it's better to use strongly-typed collections and not ArrayList, which is basically List(Of anything).
Related
so I'm using SQLite in a VB.net project with a populated database. I'm using the Microsoft.Data.Sqlite.Core and System.Data.SQLite NuGet package libraries. So the problem presents when I'm trying to get the result of a query. At first the SQLiteDataReader gets the response and all the elements of the desired table. I know this cause in the debugger I have a breakpoint after the setting the object and when I check the parameters of the SQLiteDataReader object the Result View shows all the elements of my table, but as soon as I remove the mouse from the object and check it again the Result View turns out empty, without even resuming with the next line of code. Does anyone know if its a known bug or something cause Ive used this exact method of querying a table in another project and it works.
The code:
Public Function RunQuery(com As String)
If CheckConnection() Then
command.CommandText = com
Dim response As SQLiteDataReader
response = command.ExecuteReader
Dim len As Integer = response.StepCount
Dim col As Integer = response.FieldCount
Dim resp(len, col) As Object
For i = 0 To col - 1
Using response
response.Read()
For j = 0 To len - 1
resp(i, j) = response.GetValue(j)
Next
End Using
Next
Debugger with populated result view
Debugger with empty result view
edit: added the for loop to show that its not only on the debugger that the result view is empty. When using response.Read() it throws an exception "System.InvalidOperationException: 'No current row'"
As I have told you in the comment, a DataReader derived class is a forward only retrieval object. This means that when you reach the end of the records returned by the query, that object is not capable to restart from the beginning.
So if you force the debugger to enumerate the view the reader reaches the end of the results and a second attempt to show the content of the reader fails.
The other part of your problem is caused by a misunderstanding on how to work on the reader. You should loop over the Read result not using a StepCount property. This property as far as I know, is not standard and other data providers don't support it. Probably it is present in the SQLite Provider because for them it is relatively easy to count the number of records while other providers don't attempt do calculate that value for performance reasons.
However, there are other ways to read from the reader. One of them is to fill a DataTable object with its Load method that conveniently take a DataReader
Dim data As DataTable = New DataTable()
Using response
data.Load(response)
End Using
' Now you have a datatable filled with your data.
' No need to have a dedicated bidimensional array
A DataTable is like an array where you have Rows and Columns instead of indexes to iterate over.
I'm having trouble understanding what's not happening, in my case, with IEnumerator in vb.net.
I have this code which pulls data from a database and then reads the first two rows, summing up a column from each row into 'numGirls'. Then, with 'resEnum1.Reset()', I expected 'resEnum' to reset itself to the begining of the collection, but it doesn't;
Dim resEnum1 As IEnumerator = iTrack.res_by_gender(False, YearCB.Text).GetEnumerator
resEnum1.Reset()
Dim resSet As Object
Dim numGirls As Integer = 0
For x = 0 To 1
resEnum1.MoveNext()
resSet = resEnum1.Current
numGirls += resSet.qty
Next
resEnum1.Reset()
At this point my code should continue and iterate through the collection starting at the begining of the collection (-1) with a MoveNext;
While resEnum1.MoveNext
'...........'
End While
However, the IEnumerator (resEnum1) does not infact get reset but continues from the third row of the collection. Why?
Obviously I'm not understaing something, unless it is up to me to re-invent the wheel and implement the 'reset' in code?
Not all enumerators implement the Reset method inherited from IEnumerator. You could think of it as a "forward-only" enumerator.
"The Reset method is provided for COM interoperability. It does not necessarily need to be implemented; instead, the implementer can simply throw a NotSupportedException.". - MSDN
Please see the code below:
Public Function ExecuteDynamicQuery(ByVal strSQL As String, ByVal list As List(Of clsType), ByVal tyType As clsType) As List(Of clsType) Implements IGenie.ExecuteDynamicQuery
Dim objParameterValues As New clsParameterValues
Dim iConnectionBLL As iConnectionBLL
Dim objCon As DbConnection
Dim objDR As DbDataReader
Dim paramValues() As DbParameter
objParameterValues = New clsParameterValues
iConnectionBLL = New clsConnectionBLL()
objCon = iConnectionBLL.getDatabaseTypeByDescription("Genie2")
Using objCon
paramValues = objParameterValues.getParameterValues
objDR = clsDatabaseHelper.ExecuteReader(objCon, CommandType.Text, strSQL, paramValues)
Do While objDR.Read
Dim tyType2 As clsType = tyType
tyType.PopulateDataReader(objDR)
list.Add(tyType2)
Loop
objDR.Close()
Return list
End Using
End Function
An SQL statement is passed to the function along with clsType (the base type). A list of types is returned e.g. a list of Persons. For example, in this case strSQL = "SELECT * FROM Persons". A list of 500 persons is returned but they are all the same person (the last person added to the list). I realise this is because the list is referncing the same object for each entry. How do I change this?
This is a situation where making the method generic would be useful. For instance:
Public Function MyGenericMethod(Of T As New)() As List(Of T)
Dim results As New List(Of T)()
For i As Integer = 0 To 9
Dim item As New T()
' Populate item ...
results.Add(item)
Next
Return results
End Function
For what it's worth, though, I see people trying do this kind of thing often, and it never sits well with me. I'm always the first one in line to suggest that common code should be encapsulated and not duplicated all over the place, but, I've never been convinced that creating some sort of data access layer that encapsulates the calls to ADO, but doesn't also encapsulate the SQL, is a good idea.
Consider for a moment that ADO, is in-and-of-itself an encapsulation of that part of the data-access layer. Sure, it can take a few more lines of code than you might like to execute a simple SQL command, but that extra complexity is there for a reason. It's necessary in order to support all of the features of the data source. If you try to simplify it, inevitably, you will one day need to use some other feature of the data source, but it won't be supported by your simplified interface. In my opinion, each data access method should use all of the necessary ADO objects directly rather than trying to some how create some common methods to do that. Yes, that does mean that many of your data access methods will be very similar in structure, but I think you'll be happier in the long run.
I've reduced your original code. The following sample is functionally equivalent to what you posted. Without knowing more about your types, it will hard to give you anything more than this, but maybe the reduction will make the code clear enough for you to spot a solution:
Public Function ExecuteDynamicQuery(ByVal sql As String, ByVal list As List(Of clsType), ByVal type As clsType) As List(Of clsType) Implements IGenie.ExecuteDynamicQuery
Dim paramValues() As DbParameter = New clsParameterValues().getParameterValues()
Using conn As DbConnection = iConnectionBLL.getDatabaseTypeByDescription("Genie2"), _
rdr As DbDataReader = clsDatabaseHelper.ExecuteReader(conn, CommandType.Text, sql, paramValues)
While rdr.Read()
type.PopulateDataReader(rdr)
list.Add(type)
End While
Return list
End Using
End Function
There are a few additional bits of advice I can give you:
You must have some way to accept parameter information for your query that is separate from the query itself. The ExecuteReader() method that you call supports this, but you only ever pass it an empty array. Fix this, or you will get hacked.
A implementation that uses Generics (as posted in another answer) would be much simpler and cleaner. The Genie interface you're relying doesn't seem to be adding much value. You'll likely do better starting over with a system that understands generics.
The problem of re-using the same object over and over can be fixed by creating a new object inside the loop. As written, the only way to do that is with a New clsType (and it seems you may have Option Strict Off, such that this could blow up on you at run time), through some messy reflection code, a switch to using generics as suggested in #2, or a by accepting a Func(Of clsType) delegate that can build the new object for you.
ok this is the thing I have right now which is working quite well except its a bit slow:
Public Function GetList() As List(Of SalesOrder)
Try
Dim list As New List(Of SalesOrder)
Dim ds As DataSet
ds = cls.GetSalesOrderList 'CLS is the data access class
For i = 0 To ds.Tables(0).Rows.Count - 1
Dim row As DataRow = ds.Tables(0).Rows(i)
Dim kk As SalesOrder = New SalesOrder()
kk.ID = Val(row.Item("id") & "")
kk.SalesOrderNo = row.Item("salesorderid") & ""
kk.SalesOrderDate = row.Item("OrderDate") & ""
kk.CustomerId = Val(row.Item("customerid") & "")
list.Add(kk)
Next
Return list
Catch ex As Exception
Throw ex
End Try
End Function
Now once I start retrieving more than 10000 records from the table, the loop takes long time to load values into generic class. Is there any way that I can get rid of loop? Can I do something like the following with the generic class?
txtSearch.AutoCompleteCustomSource.AddRange(Array. ConvertAll(Of DataRow, String)(BusinessLogic.ToDataTable.ConvertTo(WorkOr derList).Select(), Function(row As DataRow) row("TradeContactName")))
I would have thought the problem isn't with doing a loop but with volumes of data. Your loop method seems to process each bit of data only once so there isn't any massive efficiency crash (such as looping over the dataset once and then again for each row or that kind of thing). Any method you choose is at the end of the day going to have to loop through all your data.
Their methods might be slightly more efficient than yours but they aren't going to be that much more so I'd think. I'd look at whether you can do some refactoring to reduce your data set (eg limit it to a certain period or similar) or whether you can do whatever searching or aggregating of that list you intend in the database instead of in code. eg if you're just going to sum the values of that list then you can almost certainly do it better by having a stored procedure that will do the summing on the database rather than in the code.
I know this hasn't directly answered your question but this is mainly because I don't know of a more efficient method. I took the question as asking for optimisation in general though rather than how to do this specific one. :)
Converting the loop into some kind of LINQ construct isn't necessarily going to improve performance if you're still enumerating over every row at once. You could return IEnumerable(Of SalesOrder) if you don't need to give the consumer the ability to add/remove from the list (which it looks like might be the case), and then in that case you could create an enumerator to handle this. That way, the dataset is loaded all at once, but the items are only converted into objects when they're being enumerated over, which may be part of your performance hit.
Something like this:
Return ds.Tables(0).Rows.Select(Function(dr As DataRow) Return New SalesOrder ... );
My VB with LINQ is a little rusty, but something to that effect, where the ... is the code to instantiate a new SalesOrder. That will only create a new SalesOrder object as the IEnumerable(Of SalesOrder) is being enumerated over (lazy, if you will).
Hey Paul, You mean something like below code
Dim list As New List(Of SalesOrder)
Dim kk As SalesOrder = New SalesOrder()
Function DrToOrder(dr as datareader)
kk.ID = Val(dr.Item("id") & "")
kk.SalesOrderNo = dr.Item("salesorderid") & ""
list.Add(kk)
End function
Function LoadData()
datareader.Rows.Select(DrToOrder)
End function
Are you talking about something like above code?
This seems like a really basic thing that I'm doing, yet I'm tearing my hair out trying to make it work.
My situation is this: I have a project which contains a large number of lookup tables, and I have all of these lookup tables represented in a single typed DataSet, which contains TableAdapters for each lookup. I've designed an editor for these lookup tables, which should allow editing of one of these at a time. My front-end is written in VB and WinForms, the back-end is a SOAP web service; I can successfully pass the changes to the DataSet back to the web service, but can't find a way to use a TableAdapter to update the single table that has been changed.
What I'm trying to do is instantiate the appropriate TableAdapter for the updated DataTable by sending the name of the table back to the web service along with the DataSet, then referring to the TableAdapter with a dynamic name. The normal way to instantiate a TableAdapter is this:
Dim ta As New dsLookupsTableAdapters.tlkpMyTableTableAdapter
What I'd like to do is this, but of course it doesn't work:
strTableName = "tlkpMyTable"
Dim ta As New dsLookupsTableAdapters(strTableName & "TableAdapter")
Is there any way to achieve this, or am I taking the wrong approach altogether? My other alternative is to write separate code for each table, which I'd prefer to avoid!
You can use Activator to create an instance of your TableAdapter from its string name, just like you want:
object adapter = Activator.CreateInstance(Type.GetType("My.Namespace.MyDataSetTableAdapters." + myTable.Name + "TableAdapter"));
Then, because TableAdapters don't have a common interface, you should use reflection to call its Update method:
adapter.GetType().GetMethod("Update").Invoke(adapter, null);
http://msdn.microsoft.com/en-us/library/system.type.getmethod.aspx
This is from memory, but roughly close enough. You can also use GetProperty to get the connection property and set it as required.
Not sure I 100% understand, do you have a single DataTable in your DataSet, or one DataTable per lookup table?
Anyway, perhaps you could you this approach to filter by lookup table?
It's pretty easy to create types at runtime given the (string) type name.
Here's a self-contained VB class which illustrates one way to do it: use System.Activator.CreateInstance to create instances of types using a string representation of the type name. Then you can cast it to a DataAdapter base class and use it like any other DataAdapter.
Public Class dsLookupsTableAdapters
Public Function CreateInstance(ByVal strName As String) As Object
CreateInstance = Nothing
For Each a As System.Reflection.Assembly In System.AppDomain.CurrentDomain.GetAssemblies()
Try
Dim strAssemblyName As String() = a.FullName.Split(New Char() {","c})
Dim strNameTemp As String = strAssemblyName(0) & "." & strName
Dim instance As Object = System.Activator.CreateInstance(a.FullName, strNameTemp)
If instance IsNot Nothing Then
Dim handle As System.Runtime.Remoting.ObjectHandle
handle = CType(instance, System.Runtime.Remoting.ObjectHandle)
Dim o As Object = handle.Unwrap()
CreateInstance = o
Exit For
End If
Catch ex As System.Exception
Continue For ' ignore exception, means type isn't there
End Try
Next
End Function
Public Class tlkpMyTableTableAdapter
Inherits System.Data.Common.DataAdapter
End Class
Public Sub Test()
' define type name. note that, in this sample, tlkpMyTableTableAdapter is a nested
' class and dsLookupsTableAdapters is the containing class, hence the "+". If, however,
' dsLookupsTableAdapters is a namespace, replace the "+" with a "."
Dim typeName As String = "dsLookupsTableAdapters+tlkpMyTableTableAdapter"
Dim adapter As System.Data.Common.DataAdapter
Dim o As Object = CreateInstance(typeName)
adapter = CType(o, System.Data.Common.DataAdapter)
End Sub
End Class
If you are using VB.Net 2008, then use the tableadaptermanager (http://msdn.microsoft.com/en-us/library/bb384426.aspx). I think this would be much easier to code against :)
Wade