A legacy application is copying 100K records in one DataTable into another like so:
For index = 0 To dataTable.Rows.Count() - 1
Dim column1 = CType(dataTable.Rows(index).Item("column1"), Integer)
Dim column2 = CType(dataTable.Rows(index).Item("column2"), Integer)
Dim column3 = CType(dataTable.Rows(index).Item("column3"), Integer)
Dim dataRow = ds.Tables("MyTable").NewRow
dataRow("column1") = column1
dataRow("column2") = column2
dataRow("column3") = column3
ds.Tables("MyTable").Rows.Add(dataRow)
Next
This seems to be very slow as we need to iterate 100K times and add a new row. Are there any .NET APIs to bulk copy rows to decrease the time it takes to copy everything? The source DataTable has a lot more columns whereas the destination is a subset. We could refactor the code to only use the source, but this is a complex app and it will require regression testing since both source and destination tables are global variables and used in many places.
How about it ? "dataTable.Copy()"
https://learn.microsoft.com/en-gb/dotnet/api/system.data.datatable.copy?view=netframework-4.8
// Create an object variable for the copy.
DataTable copyDataTable;
copyDataTable = table.Copy();
I'm not sure by how much but I would expect this to be faster:
Dim newTable = oldTable.DefaultView.ToTable(False,
{"column1",
"column2",
"column3"})
Related
i have datatable "dataitems" contain 98000 Rows And Datagridview have 11000 row
want to add the quantity form the datatable to the datagridview every item in it's row in the datagrid view
i am using this code but its take too too too much time to run and sometimes stop responding i need ideas to make it faster to run
Dim dt As New DataTable = najrndataset.dataitems
Dim Total As Integer
for x = 0 to datagridview1.rows.count -1
Dim b = datagridview1.rows(x).cells(1)
Dim c = 3
Total = (From r As DataRow In dt.AsEnumerable
Where r.Field(Of String)("Item_Number") = b And r.Field(Of Integer)("SOP_Type") = c
Select r.Field(Of Integer)("Quantity")).Sum
datagridview1.Rows(x).Cells(0).Value = Total
next
Hopefully you appreciate that if you have 11000 rows in a datagridview (terrible idea, by the way) and 98000 rows in a datatable, and you're running a loop that searches the table for each of the 11K items, and it does this by starting at number 1, and searching 98000 items for it, then going to 2 and searching 98000 items for it til you reach 11000 and searching 98000 items for it... At the end of the operation you're going to have performed 11000 x 98000 operations.. i.e. you're going to have performed 1.078 BILLION operations. This is why "its take too too too much time to run and sometimes stop responding" :)
You can speed this up by using a dictionary to track the sums, and it'll probably be fastest to index the 98000 items then set the dictionary from them:
Dim d as New Dictionary(Of String, Integer)
For Each di In najrndataset.dataitems.Where(Function(r) r.SOP_Type = 3)
If Not d.ContainsKey(di.Item_Number) Then
d(di.Item_Number) = di.Quantity
Else
d(di.Item_Number) += di.Quantity
End If
Next di
Then edit your grid (ugh; this isn't how datagridview are supposed to be used)
datagridview1.SuspendDrawing()
For x = 0 to datagridview1.rows.count - 1
Dim b = datagridview1.rows(x).cells(1)
Dim s As Integer
If d.TryGetValue(b, s) Then
datagridview1.rows(x).cells(0) = s
End If
Next x
datagridview1.ResumeDrawing()
You could also have a logic of "loop over the datagrid, putting 11000 items in the dictionary, loop over the datatable accumulating sums into the dictionary if the keys are present in the dictionary, loop over the datagriview putting the sums into the datagridview".
Not an answer, but too long for a traditional comment. I'm working to provide an answer separately.
I can't tell you how many times I've seen code like this:
Dim someVariable As New SomeObject
someVariable = someMethodToReturnSomeObject()
This pattern is bad code!
The New operator in the first line tells the compiler you want to allocate memory and run the constructor for your type. However, the next line assigns a new object... it throws away and forgets the memory and constructor work from the previous line. That's wasteful.
Instead, you want this:
Dim someVariable As SomeObject = someMethodToReturnSomeObject()
Worse, this speaks to a profound lack of understanding of the difference between a reference and an object, and what the computer is doing with your code in memory. This is a core principle of how many programming environments work. Code like the first example gives me big doubts of the capability of the programmer who writes it.
I have a Query qryRuleSets which outputs a table with 19 fields (that I do not want to save into an access table before that is suggested). I would like to get the field names and store them into an array so I can use that array in a for loop later on.
To find the number of fields that in the query result (to use in for loop later on) I have implemented the following, where the number of fields is stored in the variable numberfields -
numberfields = CurrentDb.QueryDefs("qryrulesets").Fields.Count
To actually get the name of these fields and store them in an array I am running into 2 problems:
1. Getting the field names from the query result
2. Setting up a dynamic array so that if the query ends up returning a table with more or less than 19 fields, it will still work
For my first problem:
I have tried to follow the steps in the following link: Get Column name from a query table but I can't figure it out.
To get the field names from the qry result I have tried the following but I'm not overly knowledgeable in vba/access so finding it hard to understand, even after a whole lot of googling:
Dim qry As QueryDef
Dim fieldNames As QueryDef
Dim firstcol As String
Set fieldNames = CurrentDb.CreateQueryDef(qry.qryrulesets)
firstcol = fieldNames.field(0).Name
For my second problem:
To store values in an array I have tried the following (as a test) and it works but I have to define the size of the array. Is there a way where it can be dynamic, i.e based on the value of the number of fields (found above stored in numberfields) :
Dim vardata(30) As Variant
For i = 1 To numberfields
vardata(i) = "hello"
Next i
I tried making the '30' above to a variable value but it didn't like that.
Any and all help will be appreciated. Thanks!
You can do like this:
Public Function GetFieldNames(ByVal QueryName As String) As String()
Dim Query As DAO.QueryDef
Dim FieldNames() As String
Dim Index As Integer
Set Query = CurrentDb.QueryDefs(QueryName)
ReDim FieldNames(0 To Query.Fields.Count - 1)
For Index = LBound(FieldNames) To UBound(FieldNames)
FieldNames(Index) = Query.Fields(Index).Name
Next
GetFieldNames = FieldNames()
End Function
I am trying to pass a query and existing datatable into a function. The function will query the passed datatable using the passed query and return the result.
Unfortunately, I am unable to return any data. I have posted my code below. Can anyone help me fix it? I don't know what I am doing wrong.
Public Function ExecQueryTest(Query As String, DT As DataTable) As DataTable
Dim Result() As DataRow
'initialize the table to have the same number of columns of the table that is passed into the function
Dim LocalTable As DataTable = DT
'initialize counting variables
Dim x, y As Integer
'use the select command to run a query and store the results in an array
Result = DT.Select(Query)
'remove all items from the localtable after initial formatting
For x = 0 To LocalTable.Rows.Count - 1
LocalTable.Rows.RemoveAt(0)
Next
'for loop to iterate for the amount of rows stored in result
For x = 0 To Result.GetUpperBound(0)
'add each array row into the table
LocalTable.Rows.Add(Result(x))
Next
ExecQueryTest = LocalTable
End Function
If there is a better way to accomplish my goal, I don't mind starting from scratch. I just want to be able to handle dynamic tables, queries, and be able to return the information in a datatable format.
The problem is here:
Dim LocalTable As DataTable = DT
That code does not do what you think it does. DataTable is a reference type, which means assigning DT to the LocalTable variable only assigns a reference to the same object. No new table is created, and nothing is copied. Therefore, this later code also clears out the original table:
'remove all items from the localtable after initial formatting
For x = 0 To LocalTable.Rows.Count - 1
LocalTable.Rows.RemoveAt(0)
Next
Try this instead:
Public Function ExecQueryTest(Query As String, DT As DataTable) As DataTable
ExecQueryTest = New DataTable() 'create new DataTable object to hold results
For Each row As DataRow In DT.Select(Query)
ExecQueryTest.LoadDataRow(row.ItemArray, True)
Next
End Function
Though you may also need to clone each DataRow record.
You can clear a table with just
LocalTable.Clear()
instead of using that cycle, Also the results of your select can be directly converted to datatable using
LocalTable = Result.CopyToDataTable
I have 2 datatable with identical structure and some rows with duplicate keys. So I want to use lambda expression to get all the records in table 1 that have its keys duplicates with any row's keys in table 2. I tried to use this: assume that item1 and item2 is keys
Dim result as IEnumerable(Of DataRow) = table1.Asenumerable.Where(function(t1) _
table2.AsEnumerable().Any(function(t2) t1("item1") = t2("item1") _
andalso t1("item2") = t2("item2")))
But this code snippet always give me no result (result.count = 0) despite the fact that there's duplicate between 2 tables
P/S: sorry for my bad English
I would try following:
' prepare HashSet from keys from table2 '
Dim table2Keys = new HashSet(Of Tuple(Of String, String))
set.UnionWith(table2.AsEnumerable().Select(Function(x) Tuple.Create(x("item1"), x("item2"))
' search table1 for duplicates '
Dim result = table1.Where(Function(x) table2Keys.Contains(Tuple.Create(x("item1", y("item2")))
It should have better performance then using Any, because HashSet lookup can be done in O(1).
I have a datatable that contains the rows of a database table. This table has a primary key formed by 2 columns.
The components are assigned this way: datatable -> bindingsource -> datagridview. What I want is to search a specific row (based on the primary key) to select it on the grid. I cant use the bindingsource.Find method because you only can use one column.
I have access to the datatable, so I do manually search on the datatable, but how can I get bindingsource row position based on the datatable row? Or there is another way to solve this?
Im using Visual Studio 2005, VB.NET.
I am attempting to add an answer for this 2-year old question. One way to solve this is by appending this code after the UpdateAll method(of SaveItem_Click):
Me.YourDataSet.Tables("YourTable").Rows(YourBindingSource.Position).Item("YourColumn") = "YourNewValue"
Then call another UpdateAll method.
Well, I end up iterating using bindingsource.List and bindingsource.Item. I didnt know but these properties contains the data of the datatable applying the filter and sorting.
Dim value1 As String = "Juan"
Dim value2 As String = "Perez"
For i As Integer = 0 To bsData.Count - 1
Dim row As DataRowView = bsData.Item(i)
If row("Column1") = value1 AndAlso row("Column2") = value2 Then
bsData.Position = i
Return
End If
Next