OleDbDataAdapter - read tab delimited file - vb.net

(I don't need alternatives to OleDbDataAdapter.)
The code below finds and reads the file OK but the DGV has four columns (as expected) but all the data rows just have text in the first column.
Dim sDir As String = "c:\temp\"
Dim sConn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & sDir & ";Extended Properties='text;HDR=Yes;FMT=TabDelimited';"
Dim dt As New DataTable()
Using adapt As New OleDbDataAdapter(String.Format("SELECT TOP 100 * FROM robo.txt"), sConn)
adapt.Fill(dt)
End Using
DataGridView1.DataSource = dt
I would think the Extended Properties would be the only requirement. I've tried add a Schema.ini to no avail - I don't think it is even being read as the column headers never match the schema.
The header row in the most successful pass used commas as separator - this resulted in four columns with the proper names but the tab separated data all in Col1. If I use tabs in the header row I get some system assign columns (3) which sort of corresponds to a data row with two commas.
What am I doing wrong?
Here are the first few rows with the tab character being replaced by <tab> . I since noticed that I have an extra column in the data. The fix to the header row below did not fix the problem - all data is dumped into the first field.
Use a tab separator in the header, instead of commas, results in all header text and the data being dumped into the first field.
col1,state,col3,size,path
<tab> same<tab><tab> 102912<tab>\\APCD04T\Data\Thumbs.db
<tab> same<tab><tab> 22016<tab>\\APCD04T\Data\APCD Topical Info\APCD_Boards&Committees_List.doc
<tab> same<tab><tab> 4.3 m<tab>\\APCD04T\Data\APCD Topical Info\LOSSAN-LAtoSLORailCorridorStrategicPlan.pdf

Learned several things while trying to load a RoboCopy log into a DataTable using OLEDB.
log file needs to have a .txt or .csv (or ?) extension, .log fails.
Schema.ini seems to be needed for tab delimited robocopy log, good for column definition anyway.
Datagridview takes a long time to display 30MB of data so I used
filters
I borrowed code from the net to create a Schema.ini as noted below
(SO bug: code will not paste from Visual Studio anymore. Code tool flips to other web page for Java.)
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Try
Cursor = Cursors.WaitCursor
'http://ss64.com/nt/robocopy.html can suppress header and summary
Dim sFile As String = "c:\temp\robo.txt" ' seems to need a .txt or .csv, .log didn't work
CreateRoboLogSchema(sFile) ' recreates each pass, no needed once things work
Dim sConn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & IO.Path.GetDirectoryName(sFile) & ";Extended Properties='text';"
' use Schema.ini for: HDR=Yes;FMT=TabDelimited' and column definitions
Dim dt As New DataTable()
Dim SQL As String = "SELECT * FROM " & IO.Path.GetFileName(sFile)
'SQL &= " WHERE State <> 'Same'"
Using adapt As New OleDbDataAdapter(SQL, sConn)
adapt.Fill(dt)
End Using
Debug.Print("|" & dt.Rows(0)(1) & "|") ' show import trimmed leading spaces (trims trailing too)
' DGV slow to load large files, use filter to display target rows
Dim dv As New DataView(dt)
dv.RowFilter = "State <> 'Same'" ' not case sensitive
DataGridView1.DataSource = dv
DataGridView1.Columns(0).Visible = False
DataGridView1.AutoResizeColumns()
Catch ex As Exception
MsgBox(ex.Message)
Finally
'Cursor=Cursors.Default
End Try
End Sub
Private Function CreateRoboLogSchema(ByVal strFileName As String) As Boolean
' edit http://www.vb-tips.com/CSVDataSet.aspx
Dim ascii As System.Text.Encoding = System.Text.Encoding.ASCII
Dim swSchema As System.IO.StreamWriter = Nothing
Dim blnReturn As Boolean
Dim strSchemaPath As String = System.IO.Path.GetFileName(strFileName)
Try
strSchemaPath = IO.Path.GetDirectoryName(strFileName) & "\Schema.ini"
swSchema = My.Computer.FileSystem.OpenTextFileWriter(strSchemaPath, False, ascii)
Dim strFile As String = System.IO.Path.GetFileName(strFileName)
swSchema.WriteLine("[" & IO.Path.GetFileName(strFileName) & "]")
swSchema.WriteLine("ColNameHeader=False")
swSchema.WriteLine("Format=TabDelimited")
swSchema.WriteLine("Col1=Value1 Text") ' file specific
swSchema.WriteLine("Col2=State Text")
swSchema.WriteLine("Col3=DirChanges Text")
swSchema.WriteLine("Col4=Size Text")
swSchema.WriteLine("Col5=Filepath Text")
'Continue for all fields
blnReturn = True
Catch ex As Exception
blnReturn = False
Finally
If swSchema IsNot Nothing Then
swSchema.Close()
End If
End Try
Return blnReturn
End Function

Related

Retrieve column header from specific row in excel using ADO and VB.net

I have an app where a user uploads a spreadsheet and specifies the sheetname and row number for the header row. I need the app to extract the column names from that specified row. I was able to get it to work returning the top row. How would i speficy that the column names i want should be on row(x)
Dim ExcelConn As System.Data.OleDb.OleDbConnection
Dim ExcelTable As DataTable = Nothing
Dim dr As DataRow
Dim sheet_found As Boolean = False
ExcelConn = New system.Data.OleDb.OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & file & ";Extended Properties=Excel 12.0;")
End If
'open the file
ExcelConn.Open()
ExcelTable = ExcelConn.GetOleDbSchemaTable(System.Data.OleDb.OleDbSchemaGuid.Tables, New Object() {Nothing, Nothing, Nothing, "Table"})
'make sure there is a matching sheet name
For Each dr In ExcelTable.Rows
If dr("TABLE_NAME").ToString() = sheet & "$" Then
sheet_found = True
Exit For
End If
Next
If sheet_found = False Then
MesgBox("the sheet name specified in the header (" + sheet + ") was not found")
ExcelConn.Close()
Exit Sub
Else
Dim sheet_name As String = Nothing
sheet_name = "[" & sheet & "$]"
Dim cmd1 As New System.Data.OleDb.OleDbCommand("Select * From " & sheet_name, ExcelConn)
Dim da As New OleDbDataAdapter("Select * From " & sheet_name, ExcelConn)
Dim ds As DataSet = New DataSet()
Dim dc As DataColumn
da.Fill(ds)
For Each dc In ds.Tables(0).Columns 'this returns col names fine from first row. how would i tell it to get names from 2nd or 3rd row, etc. The integer var is passed in. i just need to know how to specify that it is row(x)
header_row = LCase(RTrim(header_row + "|" + dc.ColumnName))
Next
MsgBox(header_row)
ExcelConn.Close()
End If
as far as i know (checked that issue in the past) there is no way to select a table with System.Data.OleDb from excel file using SQL query if headers are not placed in row 1. the solution for me is to delete all the rows above the header row before querying the worksheet - just opening the workbook with Microsoft.Office.Interop deleting the extra rows, closing it and than querying it.
Excel is a very powerful tool but was never designed to behave like database (sql server or access file for example).
There are some known limitations to use the JET/ACE drivers to access data in Excel sheets, as jonathana has pointed out.
As an alternative, I'd like to offer our Excel ADO.NET Provider. With it, you get all of the SQL access to your Excel data that you're accustomed to from the JET/ACE drivers, but with more flexibility in how that data is arranged in Excel.
In your example, you could submit a query like the following to denote that the headers are placed in row 4:
SELECT * FROM Sheet1#A4:**
Using our provider, your code would look similar to the following:
Dim ExcelConn As System.Data.CData.Excel.ExcelConnection
Dim ExcelTable As DataTable = Nothing
Dim dr As DataRow
Dim sheet_found As Boolean = False
ExcelConn = New System.Data.CData.Excel.ExcelConnection("Excel File=" & file & ";")
'open the file
ExcelConn.Open()
ExcelTable = ExcelConn.GetSchema("Tables")
'make sure there is a matching sheet name
For Each dr In ExcelTable.Rows
If dr("Table_Name").ToString() = sheet Then
sheet_found = True
Exit For
End If
Next
If sheet_found = False Then
MesgBox("the sheet name specified in the header (" + sheet + ") was not found")
ExcelConn.Close()
Exit Sub
Else
Dim sheet_name As String = Nothing
'Here, I assume that header_row indicates the row that contains the headers
sheet_name = "[" & sheet & "#A" & header_row & ":**]"
Dim cmd1 As New System.Data.CData.Excel.ExcelCommand("Select * From " & sheet_name, ExcelConn)
Dim da As New System.Data.CData.Excel.ExcelDataAdapter("Select * From " & sheet_name, ExcelConn)
Dim ds As DataSet = New DataSet()
Dim dc As DataColumn
da.Fill(ds)
For Each dc In ds.Tables(0).Columns
'I wasn't sure what this code was meant to accomplish, but at this point,
'dc.ColumnName contains the column names from header_row
Next
ExcelConn.Close()
End If
We have a blog post on our site with more information on our provider and you can download a free trial from our site as well.

Unwanted data truncating from Excel

When I try to do an import from an Excel document the comments get truncated. I have checked the usually issue that the Table would be limited but is set as:
Comments ... nvarchar(MAX)
Sample of the code, please note even running the code in Debug mode I can see the parameter is truncated before it even goes to the stored procedure.
Dim excelConnectionString As String = (Convert.ToString("Provider=Microsoft.ACE.OLEDB.12.0; Data Source=") & vFileNameFolder) + "; Extended Properties='Excel 12.0;HDR=YES;IMEX=1;';"
'#### Upload, Rename and save file
'#### Open Excel to Parse here
Dim ds As New DataSet
Dim oleda As New OleDbDataAdapter()
Dim cmdExcel As New OleDbCommand()
'#### End - Open Excel to Parse here
Dim vActionRef As String = ""
Try
Dim excelConnection As New OleDbConnection(excelConnectionString)
With cmdExcel
.CommandText = "Select * from [Portal$A1:BB9999]" 'Names we want to select and the name of the sheet
.CommandType = CommandType.Text
.Connection = excelConnection
End With
excelConnection.Open()
oleda = New OleDbDataAdapter(cmdExcel)
oleda.Fill(ds, "dataExcel")
If ds.Tables("dataExcel").Rows.Count > 0 Then
'#### Stored procedure details
Dim connection As SqlConnection
Dim commandSQL As New SqlCommand
Dim FRAUPRN As String = ""
Dim ConnectionString As String = System.Configuration.ConfigurationManager.ConnectionStrings("SQLLocal").ToString()
'########### End - Stored procedure details
'Set date once
Dim vDate As Date
vDate = DateTime.Now.AddDays(0)
connection = New SqlConnection(ConnectionString)
connection.Open()
'Dims for error handling and checking for invalid characters
Dim iImported As Integer
For j As Integer = 0 To ds.Tables("dataExcel").Rows.Count - 1 ' counted rows so loop through, ignores first row with names in
If (IsDBNull(ds.Tables("dataExcel").Rows(j)("UPRN"))) Then
'skip
Else
iImported = iImported + 1
'Bring the data across, the rows(i)("xxx") must match a name on the Excel sheet but DOES NOT have to be in order
With commandSQL
.Parameters.Clear()
.Connection = connection
.CommandText = "spAddCSVDataLine" 'Stored procedure here
If Trim(ds.Tables("dataExcel").Rows(j)("Comments")) = "0" Then
.Parameters.AddWithValue("Comments", " ")
Else
' .Parameters.AddWithValue("Comments", If(IsDBNull(ds.Tables("dataExcel").Rows(j)("Comments")), "", Trim(ds.Tables("dataExcel").Rows(j)("Comments"))))
Dim vComments As String
vComments = ds.Tables("dataExcel").Rows(j)("Comments")
.Parameters.AddWithValue("Comments", vComments)
Session.Item("Comments") = Session.Item("Comments").ToString & "//" & vComments
End If
I have looked at similar questions such as ADO is truncating Excel data which talks about numerical issues but am struggling to find the reason why I am losing data before I export the data. 'Common sense' says excel is not passing over more than 255 characters but then this is programming!
I've had all sorts of problems with the JET/Ace DB engine truncating and doing other sorry-ass guesses at data type. Check out this Microsoft article that talks a bit about how JET uses only the first 8 records to determine field length (http://support.microsoft.com/kb/189897/en-us). You can edit a registry setting to change how many records it will scan to determine field length, but the results still seem to be hit or miss for folks.
You might also find some luck in creating a dummy record at the top of the excel sheet that contains a comment with the maximum number of characters of any of your comments. Then just delete that one record after it comes through. Again... results seem to be mixed here.

Searching Excel Document Columns with Visual Basic and Interop

I'm struggling with a problem that involves interop and excel.
Basically, I have excel files with columns that contain "headers" and the rows beneath the columns have the data. For example, the column Age will have 12,14,etc underneath it. I am new to Interop and I'm trying to allow the user to enter the name of the column header they wish to extract data from, so if they enter "Age", it'll find age is colum B for example and then extract all the data from the proceeding rows.
I've Googled extensively and haven't found anything solid, all rather context orientated and being new to Interop makes this a little tricky.
What I've got so far:
Public Sub getExcelData(ByVal directory As String)
Dim excelAppFirstFile As Excel.Application = Nothing
excelAppFirstFile = CreateObject("Excel.Application")
Try
excelAppFirstFile.Workbooks.Open(directory)
Dim excelSheet As Excel.Worksheet = excelAppFirstFile.Worksheets(1)
Catch ex As Exception
MsgBox("There was a problem: " + ex.Message)
End Try
End Sub
I know it isn't much but I've gone in circles with ranges,etc and can't figure out how to get where I need to.
EDIT:
I forgot to add that the Column name being searched for is a variable called field which is set at an earlier stage by the user.
If all you want to do is read data in the Excel file, I suggest you to use OleDb instead of interop (which is much faster):
Dim filePath As String = "C:\Book1.xls"
Dim connectionString As String = (Convert.ToString("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=") & filePath) + ";Extended Properties=""Excel 8.0"";"
Dim connection As New OleDbConnection(connectionString)
Dim cmdText As String = "SELECT * FROM [Sheet1$]"
Dim command As New OleDbCommand(cmdText, connection)
command.Connection.Open()
Dim reader As OleDbDataReader = command.ExecuteReader()
If reader.HasRows Then
While reader.Read()
Console.WriteLine("{0}" & vbTab & "{1}", reader(0).ToString(), reader(1).ToString())
End While
End If

"No data exists for the row/column" when connecting to SQL database from VB.net

I'm trying to create a program which has a datagridview, when the user clicks on a cell in the view, it then looks in a SQL database, grabs information from other fields in the same record, and automatically fills corresponding text boxes (done by manipulating the name of the field) in the form.
For some reason however, I'm getting an error message saying:
"InvalidOperationException was unhandled"
"No Data exists for the row / column"
Here is the code relevant to this part of the program:
Private Sub DataGridView1_CellMouseClick(sender As Object, e As DataGridViewCellMouseEventArgs) Handles dgvResults.CellMouseClick
' Set values in the text boxes to both corresponding to the film.
Dim strFields() As String = {"ID", "fName", "fGenre", "fSynopsis", "fAgeRating", "fActorsActresses", "fWebRating", "fNoReservations", "fWeeksRunning"}
Dim Con = New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=ApplicationData.accdb;Persist Security Info=False;")
Con.Open() 'Open the connection
Dim Cmd As New OleDbCommand(StringBuilderCommand("*", "Films", dgvResults.CurrentCell.Value, "fName"), Con) 'Create a string by calling the StringBuilderCommand to combine the parameters together with quotes.
Cmd.CommandType = CommandType.Text
Dim Rdr As OleDbDataReader = Cmd.ExecuteReader()
Dim intCount As Integer = 4 ' Create a loop variable.
Do While Rdr.Read() Or intCount < 6 ' While this statement is 'TRUE', e.g. there is a valid record.
strResult = "txt" & strFields(intCount).Replace("f", "") 'Remove any instances of 'f', e.g. the prefix of the string.
txtActorsActresses.Text = StringBuilderCommand("*", "Films", dgvResults.CurrentCell.Value, "fName")
Me.Controls(strResult).Text = Rdr.Item(strFields(intCount)) ' Suspect the error lies here.
'Set the text-box to the correct value from the database.
'This will allow me to go through several text boxes, and grab their corresponding values from the database.
intCount = intCount + 1
'Current error is because it cannot find any data beyond the first field taken.
'I have no idea why this is. But if I change the starting intCount value, it will successfully take a different value.
Loop
Rdr.Close() 'Cleaning up.
Cmd.Dispose()
Con.Close()
WebBrowser1.Navigate(dgvResults.CurrentCell.Value.Replace(" ", ".") & ".movie.poster.new.jpg.to") 'Grab the movie poster off the internet corresponding to the films name.
End Sub
Private Function StringBuilderCommand(Field, Table, CurrentCellValue, SearchParameter)
'Creates a suitable SQL string.
Dim MyStringBuilder As New StringBuilder("SELECT ")
MyStringBuilder.Append("*") ' Append the parameter 'Field'.
MyStringBuilder.Append(" FROM ") ' Append the SQL command 'FROM'.
MyStringBuilder.Append(Table) ' Append the parameter 'Table'.
MyStringBuilder.Append(" WHERE ") ' Append the SQL command 'WHERE'.
MyStringBuilder.Append(SearchParameter) ' Append the parameter 'SearchParameter'.
MyStringBuilder.Append("=""")
MyStringBuilder.Append(CurrentCellValue) ' Append the parameter 'CurrentCellValue', representing the cell selected.
MyStringBuilder.Append("""") 'Append a quotation mark.
Return MyStringBuilder.ToString() ' Return it to the main program.
End Function
Database table being connected to:
A view of the error as it looks in Visual Studio 2012 Express:
The value of 'dgvResults.CurrentCell.Value' is the name of a film taken from the database (e.g. "12 Years a Slave").
What am I doing wrong?
Thanks,
C.
The problem is caused by the value of strFields(intCount) you are passing to the reader. It is not a valid column index.
You probably want to loop on the fields before looping again on DataReader(), like:
Do While Rdr.Read()
For intCount as Integer = 4 to 6
strResult = "txt" & strFields(intCount).Replace("f", "")
txtActorsActresses.Text = StringBuilderCommand("*", "Films", dgvResults.CurrentCell.Value, "fName")
Me.Controls(strResult).Text = Rdr.Item(strFields(intCount))
Next
Loop
I removed the Dim intCount As Integer = 4 because it is no longer needed because of the for next loop.

convert csv data to DataTable in VB.net

I am trying to import a large array of integers stored as a csv file into a VB.Net DataTable called BeamMap. The .csv file consists only of integers, with a delimiter of ,, no quotes around the data (ie., 1,3,-2,44,1), and an end of line character of line feed and carriage return. All I want to do is get each integer into a DataTable cell with the appropriate rows and columns (there are the same number of columns for each row) and be able to reference it later on in my code. I really don't want anything more than absolutely necessary in the code (no titles, captions, headings, etc.), and I need it to be fairly efficient (the csv array is approx. ~1000 x ~1000).
Thanks!
Use OleDb provider to read CSV and pouplate the DataTable.
Dim folder = "c:\location\of\csv\files\"
Dim CnStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & folder & ";Extended Properties=""text;HDR=No;FMT=Delimited"";"
Dim dt As New DataTable
Using Adp As New OleDbDataAdapter("select * from [nos.csv]", CnStr)
Adp.Fill(dt)
End Using
Here's a simple approach which requires a strict format (as you've mentioned):
Dim lines = IO.File.ReadAllLines(path)
Dim tbl = New DataTable
Dim colCount = lines.First.Split(","c).Length
For i As Int32 = 1 To colCount
tbl.Columns.Add(New DataColumn("Column_" & i, GetType(Int32)))
Next
For Each line In lines
Dim objFields = From field In line.Split(","c)
Select CType(Int32.Parse(field), Object)
Dim newRow = tbl.Rows.Add()
newRow.ItemArray = objFields.ToArray()
Next
Getting the file from a mapped drive and putting the retrieved data in a dataset:
Dim folder = "Z:\"
Dim CnStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & folder & ";Extended Properties=""text;HDR=No;FMT=Delimited"";"
Dim dssample As New DataSet
Using Adp As New OleDbDataAdapter("select * from [samplecsv.csv]", CnStr)
Adp.Fill(dssample)
End Using
If dssample.Tables.Count > 0 Then
'some code here
End If
Also, don't forget to include the
Imports System.Data.OleDb
And if you wish to link to a DataGridView (after read):
Dim bs As New BindingSource
bs.DataSource = dt
DataGridView1.DataSource = bs