Can I use the the power of Generics to solve my issue - sql

Question

I have a wierd issue. I am loading 1k invoice objects, header first then details in my DAL. I am using VB.NET on this project. I am able to get the invoice headers just fine. When I get to loading the details for each invoice I am getting a timeout on SQL Server. I increased the timeout to 5 minutes but still the same thing. If I reduce the invoice count to 200 it works fine.
Here is what I am doing
//I already loaded the invoice headers. I am now iterating each invoice to get it's detail
For Each invoice As Invoice In invoices
drInvoiceItems = DBSqlHelperFactory.ExecuteReader(CONNECTION_STRING, CommandType.StoredProcedure, "dbo.getinvoiceitem", _
New SqlParameter("#invoicenumber", invoice.InvoiceNumber))
While drInvoiceItems.Read()
invoice.LineItems.Add(New InvoiceLine(drInvoiceItems("id"), drInvoiceItems("inv_id"), drInvoiceItems("prodid"), drInvoiceItems("name"), drInvoiceItems("barcode"), drInvoiceItems("quantity"), drInvoiceItems("costprice")))
End While
Next
Return invoices
I am aware that I am firing 1k connections to the DB due to the iterations. Can't I load all the line items with one select statement and then do something like
For Each invoice As Invoice In invoices
invoice.Items.Add(invoiceItems.Find(Function(i as InvoiceItem),i.InvoiceNumber = invoice.InvoiceNumber))
Next
I get the error whenusing the lambda funcion above
Error 1 Value of type 'System.Collections.Generic.List(Of BizComm.InvoiceLine)' cannot be converted to 'BizComm.InvoiceLine'. C:\Projects\BizComm\InvoiceDAL.vb 75 35 BizComm

Answer 1

One thing I have done when iterating through items in the past is use the same Connection object for all the necessary read activities. It seems to greatly enhance performance.
I'd also look at the database to see whether the dbo.getinvoiceitem procedure can be improved, or if another procedure can be written which will give you all the line items for a group of invoices (perhaps by date or customer/vendor) rather than just one header at a time. Then you can more effectively apply your iteration over the invoice collection and add the lines to the headers.
You can also check to see whether there is an effective index on column that the #invoicenumber parameter references.

Answer 2

From your code, it looks like you are not closing the connections and datareaders. See if you can place your connections and datareaders in a USING statement:
Using con As New SqlConnection(connectionString)
....
End Using
The DBSqlHelperFactory opens a connection, but can't close it since the connection is needed after its return. I'd modify the code, so that you open one connection and pass it to DBSqlHelperFactory as a parameter.
To quickly pick up these issues, I always debug with:
Max Pool Size=1;
added to the end of the connection string. That will quickly throw an error any time you forget to close a connection.

Answer 3

Why load InvoiceItems before hand? Can't you load it on demand?
i.e. when you need to get the Items, call a method on Invoice instance (myInvoice.GetItems)
EDIT: It will be better to understand the full picture of what you are trying to do.
Is it really required to get all Invoices as well?

Answer 4

Why not select all the line items for all the invoices you need in a single query. then split the results up into multiple invoice objects?
Re: how do I map between collections?
One implementation could be: create 1000 anemic Invoice object, chuck them in a Dictionary which goes from Id to Invoice. Then when you select the line items you include the invoice id, look up the anemic invoice and add the line to it.