I have 2 Data Table in Vb.Net
First Table Table1 Like
Accno || Name || databirth
100 || Hema || 10-may-1990
200 || Chand ||
300 || arul ||
Second Table table2 like
Accno databirth
100 10-may-1990
200 23-Aug-1990
300 5-Jan-1989
But I need Table1 like below Struture
Accno Name databirth
100 Hema 10-may-1990
200 Chand 23-Aug-1990
300 arul 5-Jan-1989
for i as byte = 0 to Table1.Rows.count-1
dim row as datarow = Table1.rows(i)
dim accno as integer = row("Accno")
dim dr() as datarow = Table2.Select("Accno=" & accno)
if dr.lenght>0 then
if not string.isnullorempty(row("databirth").tostring) then
row("databirth") = dr(0)("databirth")
end if
end if
next
next
Related
Description of the current situation:
I have an excel file of approximately 315 columns and 4000 rows. The file contains the answers to a 300-question questionnaire. The data format is as follows:
(Headers) A | B | C | D | E | F | Q.1 | Q.2 | ... | Q.300 |
(FirstRow) Info of first participant | AnswerCode for every Q |
The columns A to F contain contain info on every participant, while the columns Q.1 to Q.300 contain the respective answer code to each question. After storing the file as a large DataTable:
I need to load all 4000 rows on an existing database table, but before I do that I must edit the data format. The end result must become:
ParticipantCode | QuestionCode | AnswerCode | DateOfRegistration
00001 | 0001 | 1234567 | yyyy-MM-dd HH:mm:ss
... | ... | ... | ...
00001 | 0300 | 1234567 | yyyy-MM-dd HH:mm:ss
00002 | 0001 | 1234567 | yyyy-MM-dd HH:mm:ss
... | ... | ... | ...
04000 | 0300 | 1234567 | yyyy-MM-dd HH:mm:ss
So every row of the original ExcelDataTable is transformed into 300 rows in the FinalDataTable. In this way, the FinalDataTable will have about 1.2 million rows.
What Have I implemented so far:
Private Function MyFunction()
For Each ExcelRow As DataRow In ExcelDataTable.Rows
For Each ExcelColumn As DataColumn In ExcelDataTable.Columns
QuestionCodeFound = False
ExcelColumnNameRaw = ExcelColumn.ColumnName.ToString.Trim
If ExcelColumnNameRaw.StartsWith("Q") Then
' Correct the headers
ExcelColumnSplit = ExcelColumnNameRaw.Split("#")
ExcelColumnName = String.Concat(ExcelColumnSplit(0), ExcelColumnSplit(1))
SelectedRowFromDT = QuestionCodeAndQuestionIDDataTable.Select("QuestionID = '" + ExcelColumnName + "'")
' Search for "_", because some questions are different
If SelectedRowFromDT.Length > 0 Then
QuestionCodeFound = True
Else
Dim ExcelColumnSplitForMult As String()
ExcelColumnSplitForMult = ExcelColumnName.Split("_")
SelectedRowFromDT = QuestionCodeAndQuestionIDDataTable.Select("QuestionID = '" + ExcelColumnSplitForMult(0).ToString + "'")
If SelectedRowFromDT.Length > 0 Then
QuestionCodeFound = True
End If
End If
If QuestionCodeFound Then
Dim QuestionCode As String
Dim QuestionTypeDataTable As DataTable
Dim QuestionType As String
' Get the Question Type from the respective table
QuestionType = String.Empty
QuestionCode = SelectedRowFromDT(0).Item("QuestionCode").ToString
QuestionTypeDataTable = SearchInSql(My.Settings.ConnectionString, SQLString)
If QuestionTypeDataTable.Rows.Count > 0 Then
QuestionType = QuestionTypeDataTable.Rows(0).Item(0).ToString.Trim
End If
' Fix the Date Format
DateRaw = ExcelRow.Item(1).ToString
DateSplit = DateRaw.Split("/")
If DateSplit(0).Length = 1 Then
DateSplit(0) = String.Concat("0", DateSplit(0))
End If
If DateSplit(1).Length = 1 Then
DateSplit(1) = String.Concat("0", DateSplit(1))
End If
DateText = String.Concat(DateSplit(0), "/", DateSplit(1), "/", DateSplit(2))
DateRegistration = DateTime.ParseExact(DateText, "MM/dd/yyyy", CultureInfo.InvariantCulture)
DateRegistrationReformed = DateRegistration.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)
DateRegFinal = DateTime.ParseExact((DateRegistrationReformed + " " + "10:00:00").ToString, "yyyy-MM-dd HH:mm:ss", CultureInfo.InvariantCulture)
Dim AnswerValue As String
Dim AnswerCode As String
Dim AnswerCodeDataTable As DataTable
Dim QuestionWasAnswer As String
Dim AnswerValueRow() As DataRow = ExcelDataTable.Select("ParticipantCode = '" + ExcelRow.Item(2).ToString + "'")
AnswerCodeDataTable = New DataTable
AnswerValue = ""
QuestionWasAnswer = "0"
' Complete "QuestionWasAnswer" field for all questions and retrieve the AnswerCode for the answer given by each participant
If AnswerValueRow.Length > 0 And AnswerValueRow(0).Item(ExcelColumnNameRaw).GetType IsNot GetType(DBNull) Then
If Not (QuestionType.Equals("02") Or QuestionType.Equals("03")) Then
AnswerValue = AnswerValueRow(0).Item(ExcelColumnNameRaw)
QuestionWasAnswer = "1"
ElseIf QuestionType.Equals("02") Or QuestionType.Equals("03") Then
Dim ExcelColumnSplitForMultSecond As String()
Dim MultAnswerValue As String
ExcelColumnSplitForMultSecond = ExcelColumnName.Split("_")
MultAnswerValue = AnswerValueRow(0).Item(ExcelColumnNameRaw).ToString.Trim
AnswerValue = ExcelColumnSplitForMultSecond(1).ToString
If MultAnswerValue.Equals("1") Then
QuestionWasAnswer = "1"
ElseIf MultAnswerValue.Equals("2") Then
QuestionWasAnswer = "2"
End If
End If
' Search in the Answers table for the existing AnswerCode
SQLString = String.Format("SELECT Answers.AnswerCode
FROM Answers
WHERE Answers.QuestionCode = '{0}'
AND (Answers.AnswerNumber = '{1}' OR Answers.Answer = '{1}')", QuestionCode, AnswerValue)
AnswerCodeDataTable = SearchInSql(My.Settings.ConnectionString, SQLString)
If AnswerCodeDataTable.Rows.Count > 0 Then
AnswerCode = AnswerCodeDataTable.Rows(0).Item(0).ToString
FormattedDataTable.Rows.Add(ParticipantAnswerCode, ExcelRow.Item(2), QuestionCode, AnswerCode, QuestionWasAnswer, DateRegFinal)
ParticipantAnswerCode = Convert.ToInt32(ParticipantAnswerCode + 1).ToString.PadLeft(ParticipantAnswerCodeFieldLength, "0")
Else
' If a given answer does not exist, save it in the respective table and then try again
Dim AnswerCodeLength = GetLengthFromSqlDataBase(My.Settings.ConnectionString, "Answers", "AnswerCode")
Dim NextAnswerCode = CalculateNextAnswerCode(AnswerCodeLength)
Dim NestAnswerNumber = CalculateNextAnswerNumber(QuestionCode)
SaveNewAnswer(NextAnswerCode, QuestionCode, NestAnswerNumber, AnswerValue)
SQLString = String.Format("SELECT Answers.AnswerCode
FROM Answers
WHERE Answers.QuestionCode = '{0}'
AND Answers.Answer = '{1}'", QuestionCode, AnswerValue)
AnswerCodeDataTable = SearchInSql(My.Settings.ConnectionString, SQLString)
If AnswerCodeDataTable.Rows.Count > 0 Then
AnswerCode = AnswerCodeDataTable.Rows(0).Item(0).ToString
FormattedDataTable.Rows.Add(ParticipantAnswerCode, ExcelRow.Item(2), QuestionCode, AnswerCode, QuestionWasAnswer, DateRegFinal)
ParticipantAnswerCode = Convert.ToInt32(ParticipantAnswerCode + 1).ToString.PadLeft(ParticipantAnswerCodeFieldLength, "0")
End If
End If
End If
End If
End If
Next
Next
Return FormattedDataTable
End Function
After that, I bulk insert the FinalDataTable on the DB.
The problem I am facing:
Using the current program I built, every row in the ExcelDataTable takes about 40 seconds to transform into 300 rows in the FinalDataTable. If I try to load all 4000 rows, it will take more than 40 hours to transform the entire datatable. I need to find a faster way to do this.
As mentioned, there isn't much to go off of on this with what has been provided.
I'm sure there are more helpful fixes to consider but I wanted to put my two cents in about the For Loops.
I recommend switching the
For Each
statements with
For i as integer = 0 to ExcelDataTable.Rows.Count - 1
I've read that For Each is not as performance-friendly as it gathers each "row" as a collection, therefore increasing the overhead per loop.
Here is a SO post about this subject:
Major difference between 'for each' and 'for' loop in .NET
Not sure if that will make a difference for you but thought I would recommend it anyway.
I Have a datatable with following records
ID NAME VALUE CONTENT
1 AAA 10 SYS, LKE
2 BBB 20 NOM
1 AAA 15 BST
3 CCC 30 DSR
2 BBB 05 EFG
I want to write a VB.NET/LINQ query to have a output like below table: -
ID NAME SUM CONTENT (as CSV)
1 AAA 25 SYS, LKE, BST
2 BBB 25 NOM, EFG
3 CCC 30 DSR
Please provide me LINQ query to get the desired result. Thanks.
I have tried concatenation using below query
Dim grouped = From row In dtTgt.AsEnumerable() _
Group row By New With {row.Field(Of Int16)("ID"), row.Field(Of String)("Name")} _
Into grp() _
Select ID, Name, CONTENT= String.Join(",", From i In grp Select i.Field(Of String)("CONTENT"))
This query will give you the expected output:-
Dim result = From row In dt.AsEnumerable()
Group row By _group = New With {Key .Id = row.Field(Of Integer)("Id"),
Key .Name = row.Field(Of String)("Name")} Into g = Group
Select New With {Key .Id = _group.Id, Key .Name = _group.Name,
Key .Sum = g.Sum(Function(x) x.Field(Of Integer)("Value")),
Key .Content = String.Join(",", g.Select(Function(x) x.Field(Of String)("Content")))}
Thanks for your answers.
However, I have managed to get the desired result using simple code (Without LINQ): -
Dim dt2 As New DataTable
dt2 = dt.Clone()
For Each dRow As DataRow In dt.Rows
Dim iID As Integer = dRow("ID")
Dim sName As String = dRow("Name")
Dim sContt As String = dRow("Content")
Dim iValue As Integer = dRow("Value")
Dim rwTgt() As DataRow = dt2.Select("ID=" & iID)
If rwTgt.Length > 0 Then
rwTgt(0)("Value") += iValue
rwTgt(0)("Content") += ", " & sContt
Else
rw = dt2.NewRow()
rw("ID") = iID
rw("Name") = sName
rw("Value") = iValue
rw("Content") = sContt
dt2.Rows.Add(rw)
End If
Next
I want to update a table using numbers from another table, I want to do the following formla
Y-X(1)-(A1+A2+A3)
X is form table1 while Y,A1,A2,A3 from table2. The new record wil be updated in column X where the ID has only one of the records = 0
Table1
ID || X
-------------
1 || **0**
1 || 155
2 || 4
3 || 0
3 || 0
3 || 234
4 || 0
4 || 0
Table2
ID || Y || A1 || A2 || A3
--------------------------------------
1 || 228 || 1 || 3 || 4
2 || 112 || 6 || 7 || 7
3 || 4 || 22 || 1 || 0
4 || 78 || 76 || 6 || 2
from the above example tables the only column that will match the certiria is ID=1 where only one of them is 0 and the count = 2 so column X
where =0 will be updated ( X2= 228-155-(1+3+4))
Updated table1
ID || X
-------------
1 || **65**
1 || 155
2 || 4
3 || 0
3 || 0
3 || 234
4 || 0
4 || 0
My code is
Private Sub GET_TWO_INJLINE_EST()
DoCmd.SetWarnings False
DoCmd.RunSQL "UPDATE table1 I, table2 P" _
& " SET I.X = IIf(I.X = 0, DLookup(""P.Y-I.X-(P.A1 + P.A2 + P.A3)"" , " _
& " ""table1"", ""[ID]="" & [ID] & "" AND I.X <> 0""), I.X)" _
& " WHERE I.[ID] IN (SELECT I.[ID] FROM table1 I" _
& " GROUP BY I.[ID] HAVING (Count(I.[ID]) = 2)" _
& " AND (Min(I.ID) <> Max(I.X))" _
& " AND (Min(I.X) = 0 OR Max(I.X) = 0))" _
& " AND I.ID=P.ID"
DoCmd.SetWarnings True
End Sub
when I run it show me "Unknown" error.
Consider an UPDATE ... INNER JOIN query and reconciling your DLookUp as you attempt to pass I.X within the string expression. Furthermore, your subquery's Min() <> Max() equality does not compare to same X.
Finally, consider saving the update query as a stored Access query without concatenating in a VBA string which avoids the double quoting and forces you to check syntax error prior to saving. Plus, the database engine caches and pre-compiles for best execution plan.
SQL (save as Access stored query; see DMax inside larger expression)
UPDATE table1 I
INNER JOIN table2 P ON I.ID = P.ID
SET I.X = IIF(I.X=0, (P.Y - DMax("X", "table1", "ID=" & I.ID) - (P.A1+P.A2+P.A3)), I.X)
WHERE I.[ID] IN
(SELECT sub.[ID]
FROM table1 sub
GROUP BY sub.[ID]
HAVING (Count(sub.[ID]) = 2)
AND (Min(sub.X) <> Max(sub.X))
AND (Min(sub.X) = 0 OR Max(sub.X) = 0))
VBA
Private Sub GET_TWO_INJLINE_EST()
DoCmd.SetWarnings True
DoCmd.OpenQuery "mySavedUpdateQuery"
DoCmd.SetWarnings True
End Sub
try changing the second table1 I to table1 J and update all the references to the second table1
UPDATE table1 I, table2 P
Set I.X = IIf(I.X = 0,
DLookup(""P.Y-I.X-(P.A1 + P.A2 + P.A3)"",
""table1"",
""[ID]="" & [ID] & "" AND I.X <> 0""
),
I.X
)
WHERE I.[ID] IN (
SELECT J.[ID] FROM table1 J
GROUP BY J.[ID]
HAVING (Count(J.[ID]) = 2)
AND (Min(J.ID) <> Max(J.X))
AND (Min(J.X) = 0 OR Max(J.X) = 0))
AND J.ID=P.ID
I have a table like this:
MAName feldtext
------------------
karl fieldtext1
karl fieldtext2
karl fieldtext1
karl fieldtext3
karl fieldtext4
karl fieldtext2
karl fieldtext5
karl fieldtext3
karl fieldtext3
susi fieldtext1
susi fieldtext4
john fieldtext2
john fieldtext5
john fieldtext5
and I need:
MAName fieldtext1 fieldtext2 fieldtext3 fieldtext4 fieldtext5 FehlerJeMA
karl 2 2 3 1 1 9
susi 1 0 0 1 0 2
john 0 1 0 0 2 3
The columns fieldtext can go from fieldtext1 to fieldtextn, it's dynamic, depending on query.
I was looking here for solutions and found, so my approach:
Dim dt2 As New DataTable
Dim nn As Integer = 0
Dim Zeile As DataRow
dt2.Columns.Add("MAName")
' fieldtext distinct
Dim query2 = (From dr In (From d In newTable2.AsEnumerable Select New With {.feldtext1 = d("feldtext")}) Select dr.feldtext1 Distinct)
For Each Feldtext In query2
dt2.Columns.Add(Feldtext)
Next
column = New DataColumn()
column.DataType = System.Type.GetType("System.Int32")
column.ColumnName = "FehlerJeMA"
dt2.Columns.Add(column)
' MAName distinct
Dim query3 = (From dr In (From d In newTable2.AsEnumerable Select New With {.MAName2 = d("MAName")}) Select dr.MAName2.ToString.ToLower Distinct)
For Each Mitarbeiter In query3
Zeile = dt2.NewRow()
Zeile(0) = Mitarbeiter.ToString.ToLower
MA2 = Mitarbeiter.ToString.ToLower
nn = 1
For Each colName2 In query2
Fehler2 = colName2
Dim AnzahlFehler As String = (From row In newTable2.Rows Select row Where row("MAName").ToString.ToLower = MA2 And row("feldtext") = Fehler2).Count
If AnzahlFehler = 0 Then
AnzahlFehler = ""
End If
Zeile(nn) = AnzahlFehler
nn += 1
If AnzahlFehler <> "" Then
FehlerJeMA += CInt(AnzahlFehler)
End If
Next
Zeile(nn) = FehlerJeMA
dt2.Rows.Add(Zeile)
Next
This works, but is very slow...
It could be the case that in my table has more than 10.000 rows...
So my question is: what is fastest approach to get the result?
Is it some kind of cross table with linq? Other approaches?
In C# you will be able to use the code, try to translate it for your problem:
var pivotData = data.GroupBy(x => new {x.MAName, x.feldtext}, (key, group) => new { MAName = key.Column1, feldtext = key.Column2, count = group.Count() });
I am having two datasets,I have to check if plant code is 1st dataset and 2nd dataset then i have to set some value for a variable.If plant code is in 1st dataset and not in second dataset then have to set some value.I have try this with following code,but i m not getting output as i want
If Not dsCheckForBOM Is Nothing AndAlso dsCheckForBOM.Tables.Count > 0 Then
If dsCheckForBOM.Tables(0).Rows.Count > 0 Then
For count1 As Int32 = 0 To dsCheckForBOM.Tables(0).Rows.Count - 1
For count2 As Int32 = 0 To dsTmpMat.Tables(0).Rows.Count - 1
If dsTmpMat.Tables(0).Rows(count2)("PLANT_CODE").ToString() = dsCheckForBOM.Tables(0).Rows(count1)("PLANT_CODE").ToString() Then
dsTmpMat.Tables(0).Rows(count2)("BOM_IND") = 1
dsTmpMat.Tables(0).Rows(count2)("BOM_DESC") = "BOM CREATED"
If count2 < dsTmpMat.Tables(0).Rows.Count - 1 AndAlso count1 < dsCheckForBOM.Tables(0).Rows.Count - 1 Then
count2 = count2 + 1
count1 = count1 + 1
End If
Else
dsTmpMat.Tables(0).Rows(count2)("BOM_IND") = 0
dsTmpMat.Tables(0).Rows(count2)("BOM_DESC") = "BOM NOT CREATED"
If count2 < dsTmpMat.Tables(0).Rows.Count - 1 Then
count2 = count2 + 1
End If
End If
Next
Next
Else
For i As Int32 = 0 To dsTmpMat.Tables(0).Rows.Count - 1
dsTmpMat.Tables(0).Rows(i)("BOM_IND") = 0
dsTmpMat.Tables(0).Rows(i)("BOM_DESC") = "BOM NOT CREATED"
Next
End If
End If
I can't test this, but I think you need to use a different approach.
First, there is no need to have two loop nested. Use the Select method to find the rows in the temporary table.
Second, if you don't find the row in the temporary table then I think you need to create it, not to set the last row of the temporary table to zero
If Not dsCheckForBOM Is Nothing AndAlso dsCheckForBOM.Tables.Count > 0 Then
' To remove the clutter, assign the two tables
' to two local variables and work with them
Dim tempTable = dsTmpMat.Tables(0)
Dim checkTable = dsCheckForBOM.Tables(0)
If checkTable.Rows.Count > 0 Then
' Loop on the primary table
For count1 As Int32 = 0 To checkTable.Rows.Count - 1
' Get the key to search for in the temporary table
Dim plantCode = checkTable.Rows(count1)("PLANT_CODE").ToString()
' Use Select to return an array of rows that match the condition.
' I suppose that PLANT_CODE is a primary key value here, so just one
' row should be returned
Dim foundRows = tempTable.Select("PLANT_CODE = '" plantCode + "'")
if foundRows.Count > 0
foundRows(0)("BOM_IND") = 1
foundRows(0)("BOM_DESC") = "BOM CREATED"
Else
' No row found, so create a new row and add it to the temporary table
Dim newRow = tempTable.NewRow()
newRow("BOM_IND") = 0
newRow("BOM_DESC") = "BOM NOT CREATED"
tempTable.Rows.Add(newRow)
End If
Next
Else
For i As Int32 = 0 To dsTmpMat.Tables(0).Rows.Count - 1
dsTmpMat.Tables(0).Rows(i)("BOM_IND") = 0
dsTmpMat.Tables(0).Rows(i)("BOM_DESC") = "BOM NOT CREATED"
Next
End If
End If