I have two dataTables which I want to union to make one final dataTable.
Both are results of different functions.
I tried this :
dtfinal = dt1.Copy()
dtfinal = dt2.Copy()
But here dt2 data is replaced by dt1. What should be used to get union of both into the final dt.
You can use DataTable.Merge:
Dim allTables() As DataTable = {dt1, dt2}
Dim dtfinal = new DataTable("dtfinal")
dtfinal.BeginLoadData() ' Turns off notifications, index maintenance, and constraints while loading data
For Each t As DataTable in allTables
dtfinal.Merge(t) ' same as table.Merge(t, false, MissingSchemaAction.Add)
Next
dtfinal.EndLoadData()
If you don't have primary keys specified you could end up with repeating rows where you actually want to merge them. Then either specify the PKs or use this method i have provided here(needs conversion from C#):
Combining n DataTables into a Single DataTable
Related
Simple question:
When I call SqlDataAdapter.Fill(DataGridView.DataSource) the second time after initially creating first Data it does not update the contained rows. It simply adds all rows returned by the select command to my DataGridView.
If I call it a third, fourth (so on) it will also just add the returned rows.
Am I understanding the .Fill(DataTable) function wrong? How do I update the already existing DataTable correctly? Which line of code is responsible for that?
Turns out it has to be a code problem;
DataGridView1.AutoGenerateColumns = False
Dim sql = "select * from myTable"
oDtSource = New DataTable
oAdapter = New SqlDataAdapter
oCon = sqlCon("serverName\Instance", "myDataBase") ' Returns a SqlConnection
oCmd = New SqlCommand(sql, oCon)
oCon.Open()
oDtSource.Clear()
oAdapter.MissingSchemaAction = MissingSchemaAction.AddWithKey
oAdapter.SelectCommand = oCmd
oAdapter.Fill(oDtSource)
DataGridView1.DataSource = oDtSource
For refreshing I use oAdapter.Fill(oDtSource)
The PrimaryKey is set in the database
From MSDN:
You can use the Fill method multiple times on the same DataTable. If a
primary key exists, incoming rows are merged with matching rows that
already exist. If no primary key exists, incoming rows are appended to
the DataTable.
So either define a primary key or clear the table first.
Dim table = CType(DataGridView.DataSource, DataTable)
table.Clear()
' fill ...
To define primary key(s) manually read this. To let it create automatically if they are defined in the database you need to set the MissingSchemaAction to AddWithKey:
' ...
dataAdapter.MissingSchemaAction = MissingSchemaAction.AddWithKey
' fill ...
The edit code doesnt show the PrimaryKey being defined for the DataTable. This will configure the DataAdapter to perform updates and enabled refreshing the DataTable. The code uses MySQL but the Provider objects all work the same in this regard:
' persistant form level vars
Private daSample As MySqlDataAdapter
Private dtSample As DataTable
...
Elsewhere:
' there are caveats with WHERE clauses
Dim sql = "SELECT Id, Name, Country, Animal FROM SAMPLE WHERE Color = 'onyx'"
' using the ctor overload, no need for a DbCommand or Connection object
daSample = New MySqlDataAdapter(sql, MySQLConnStr)
' initialize the CommandBuilder, get other commands
Dim cbSample = New MySqlCommandBuilder(daSample)
daSample.UpdateCommand = cbSample.GetUpdateCommand
daSample.InsertCommand = cbSample.GetInsertCommand
daSample.DeleteCommand = cbSample.GetDeleteCommand
dtSample = New DataTable()
daSample.FillSchema(dtSample, SchemaType.Source)
dtSample.PrimaryKey = New DataColumn() {dtSample.Columns("id")}
daSample.Fill(dtSample)
dgv1.DataSource = dtSample
To pick up changes made to the db from other client apps:
daSample.Fill(dtSample)
Initial display:
After I change a row to "onyx" from a UI browser and Update the changed row shows up:
WHERE clauses can be a bit of an issue. Since it restricts the subset of data pulled back, Update is only going to compare rows in the new result set. So, if I change an onlyx row to "blue" it wont be removed.
One solution is to use .DefaultView.RowFilter on the table, but that can slow things down since it requires returning all rows to the client to be filtered there. Its not perfect.
I have a DataTable containing about 30 rows and I need to extract all rows having a date field bigger than a date stored into a variable.
(This code will be executed a lot of times)
I found three ways to do this but I would like to know how to choose because I don't know the differences between various codes.
Here is what I was able to write (and my worries):
1st way (DataTable.Select)
Dim SelectedRows() As DataRow = DT_DBData.Select("In_Date=#" & _
LastDate.ToString("yyyy-MM-dd") & "#")
Using New_Dt As DataTable = SelectedRows.CopyToDataTable
'Some code
End Using
I'm worried about the string format: I'm afraid that some rows may be not extracted because of a date formatting error.
2nd way (query Linq)
Using New_Dt As DataTable = (From DBData In DT_DBData.AsEnumerable() _
Where DBData.Field(Of Date)("In_Date") >= LastDate).CopyToDataTable
'Some code
End Using
I never used Linq and so I don't know what kind of issues can it give me.
3rd way (For Each Loop + If Then)
Using New_Dt As DataTable = DT_DBData.Clone
For Each dr As DataRow In DT_DBData.Rows
If dr("In_Date") >= LastDate Then
New_Dt.Rows.Add(dr.ItemArray)
End If
Next
'Some code
End Using
I'm not really worried about this code. I only think that the others could be better or faster (but I can't answer to this)
Faster is kind of irrelevant when dealing with 30 rows.
The first one is kind of wasteful. You start with a DataTable, Select to get a subset, then convert the result into a new DataTable. Time to extract matching Rows: 8 ms.
You can work with the SelectedRows array without putting it into a new DataTable. If it goes back to the DB after "some code", I would not extract it from the DT.
By the way, there is no reason to worry about matching date formats as long as the DB column is a date type (and therefore, the DataTable column will be also). Dates do not have a format; formats are just how computers (and by extension, us) display them to users.
Dim drs = dt.Select(String.Format("StartDate > '{0}'", dtTgt.Date), "")
The date type I pass will compare/filter just fine with the DateTime data for that column. Formats only come into play when you convert them to string, which is mostly only needed for those pesky users.
One option you missed might be especially useful if this will be done over and over: A DataView:
dt.Load(cmd.ExecuteReader)
' create dataview
Dim dv As New DataView(dt)
dv.RowFilter = String.Format("StartDate > '{0}'", dtTgt.Date)
dv.Sort = "StartDate asc"
' show/iterate/whatever
dgv.DataSource = dv
If the data goes back to the DB, using this method, the rows will retain all the rowstate values.
I have a situation where I want to Insert into access DB table from MS SQL table.
Same columns and everything.
I have both data sets and both table adapter. I can do what ever I want inside each dataset - any manipulation but I cannot insert from one table to another.
I tried creating an Insert query for destination tableadapter but I cannot get the from working. Tried linking, nothing works.
Searched for days, simply cannot find it.
Thank you for your answer. Can you help me on my example. I'm having trouble setting this up. This is what i got:
Dim myToTableTableAdapter As FirstDataSetTableAdapters.ToTableTableAdapter
myToTableTableAdapter = New FirstDataSetTableAdapters.ToTableTableAdapter()
Dim myFromTableTableAdapter As SecondDataSetTableAdapters.FromTableTableAdapter
myFromTableTableAdapter = New SecondDataSetTableAdapters.FromTableTableAdapter()
myFromTableTableAdapter = myToTableTableAdapter.Clone
'but it doesnt work from here`
What I wanted to do is:
For each drfrom As DataRow In myFromTableTableAdapter.GetData
myToTableTableAdapter.InsertInto(drfrom.item(column01), drfrom.item(column02), drfrom.item(andSoOn))
Next
But it seem to me that this would take so much longer then a "Insert Into From Select" script.
You cannot insert a row from one table into another table, but there are a couple of ways to do what you want. One way (a little verbose) is this:
' sets it up with same schema but empty rows
mOutTable = inTable.Clone
' Now insert the rows:
For Each rowIn In inTable.Rows
r = mOutTable.NewRow()
For Each col In inTable.Columns
r(col.ColumnName) = rowIn(col.ColumnName)
Next
mOutTable.Rows.Add(r)
Next
mOutTable.AcceptChanges
A second way, which is more concise, is this:
outTable = inTable.Clone
For Each inRow As DataRow In inTable.Rows
outTable.LoadDataRow(inRow.ItemArray, False)
End If
outTable.AcceptChanges
Note that both inTable and outTable are ADO.NET DataTable objects. You cannot implement my suggestion on the DataAdapter objects. You must use the DataTable objects. Each DataTable can be associated with a DataAdapter in the standard fashion for ADO.NET:
Dim t as New DataTable()
a.Fill(t);
where a is the ADO.NET DataAdapter. I hope this helps!
Jim
I have a Source Data table of type System.Data.DataTable from which i have to generate Destination Data Table of mapped column(Consider same SqlDBType for mapped columns.)
I have a list of MappingInfo class in which each Source Datatable Column mapped with New Column Name which will be in destination data table.
Public Class MappingInfo
Public Property SourceFieldName As String
Public Property DestinationFieldName As String
End Class
I have to evaluate a condition in source datatable to allow row data copy in destination table.
I did this using following code snippet:
''Prepare destination table.
For Each oMapping In oMappingInfo
DestinationDataTable.Columns.Add( _
New DataColumn(oMapping.DestinationFieldName))
Next
For Each oRow In SourceDataTable.Rows ''Copy data.
If oRow("IsActive") Then
oDataRow = DestinationDataTable.NewRow
For Each oMapping In oMappingInfo
oDataRow(oMapping.DestinationFieldName) = _
oRow(oMapping.SourceFieldName)
Next
DestinationDataTable.Rows.Add(oDataRow)
End If
Next
The main drawback is that here i have minimum 40k records in source datatable and data is not possible to fetch from database as all changes with data committed only when user save his work. The generated destination table is been assigned as data source to grid control and to report for preview.
How can i achieve this efficiently using Linq or do anyone please suggest me best way to achieve this requirement.
I've not tried this, so I can't say for sure that it'll be faster, but it seems to me that you'd get much better speed using something like the following:
Dim l_destinationTable As DataTable
' This creates a copy of the structure and content
l_destinationTable = SourceTable.Copy()
For Each l_column As DataColumn in l_destinationTable.Columns
Dim l_columnMap = oMappingInfo.FirstOrDefault( _
Function (c) c.SourceFieldName = l_column.ColumnName )
If l_columnMap IsNot Nothing Then
' Rename the column if it is mapped
l_column.ColumnName = l_columnMap.DestinationFieldName
Else
' Drop the column if it is not mapped
l_destinationTable.Columns.Remove( l_column )
End If
Next
NOTE: This method will fail if an unmapped column is part of a relationship or another column's expression depends on this column. Also, if you are swapping the name of two columns (for example, A will be named B and B will be named A) then you will get an exception as two columns may not have the same name at the same time.
I am dealing with a legacy application which is written in VB.Net 2.0 against a SQL 2000 database.
There is a single table which has ~125,000 rows and 2 pairs of fields with similar data.
i.e. FieldA1, FieldB1, FieldA2, FieldB2
I need to process a combined, distinct list of FieldA, FieldB.
Using SQL I have confirmed that there are ~140,000 distinct rows.
Due to a very restrictive framework in the application I can only retrieve the data as either 2 XML objects, 2 DataTable objects or 2 DataTableReader objects. I am unable to execute custom SQL using the framework.
Due to a very restrictive DB access policy I am unable to add a View or Stored Proc to retrieve as a single list.
What is the most efficient way to combine the 2 XML / DataTable / DataTableReader objects into a single, distinct, IEnumerable object for later processing?
I may have missed something here but could you not combine both DataTables using Merge?
DataTableA.Merge(DataTableB)
You can then use DataTableA.AsEnumerable()
Then see this answer on how to remove duplicates or
You can do this with a DataView as follows: dt.DefaultView.ToTable(True,[Column names])
This is the solution I came up with.
Combine the 2 DataTables using .Merge (thanks to Matt's answer)
Using this as a base I came up with the following code to get distinct rows from the DataTable based on 2 columns:
Private Shared Function GetDistinctRows(sourceTable As DataTable, ParamArray columnNames As String()) As DataTable
Dim dt As New DataTable
Dim sort = String.Empty
For Each columnName As String In columnNames
dt.Columns.Add(columnName, sourceTable.Columns(columnName).DataType)
If sort.Length > 0 Then
sort = sort & ","
End If
sort = sort & columnName
Next
Dim lastValue As DataRow = Nothing
For Each dr As DataRow In sourceTable.Select("", sort)
Dim add As Boolean = False
If IsNothing(lastValue) Then
add = True
Else
For Each columnName As String In columnNames
If Not (lastValue(columnName).Equals(dr(columnName))) Then
add = True
Exit For
End If
Next
End If
If add Then
lastValue = dr
dt.ImportRow(dr)
End If
Next
Return dt
End Function