I'm being presented with a CSV file that i would like to import to a datatable. The challenge I have is that the file has 2 different delimters. The first few columns are delimited with a "tab" and the rest with a";". I can handle the one easily but not sure how to handle both. The code that I have so far but struggling to find a way to expand this to import it the single step:
Public Function LoadFileToDatatable(ByVal FullFilePath As String)
'Load the Testfile into an datatable
Dim folder As String = System.IO.Path.GetDirectoryName(FullFilePath)
Dim filename As String = System.IO.Path.GetFileName(FullFilePath)
Dim con = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & folder & ";Extended Properties=""text;HDR=No;FMT=Delimited"";"
Dim dt As New DataTable
Using Adp As New OleDbDataAdapter("Select * From " & filename, con)
Adp.Fill(dt)
'Remove the first row as it contains the header data
Dim theRow As DataRow = dt.Rows(0)
dt.Rows.Remove(theRow)
End Using
Return dt
End Function
Related
I have a .CSV file that im filling a datatable in my application with.
one of the cells is a number, for example: 5720358152
these values seem to be getting skipped and nothing is put into the datatable.
here is my code:
Dim csvFile As String = "I:\STOCK.csv"
Dim folder = "I:\"
Dim csvStr As String = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & folder & ";Extended Properties=""text;HDR=No;FMT=Delimited"";"
Dim csvSQL = "SELECT * FROM [STOCK.csv]"
Dim MyDT As DataTable
Using csvCn = New OleDbConnection(csvStr),
cmd As New OleDbCommand(csvSQL, csvCn)
Using da As New OleDbDataAdapter(cmd)
MyDT = New DataTable
da.Fill(MyDT)
End Using
End Using
StockTakeDGV.DataSource = MyDT
Here is a list of numbers that get left out.
5720358152
5720358150
5720358146
5720350121
5720324303
5720308119
5720308118
5720308115
5720308114
5720308110
5720308104
But these numbers are fine:
4021021135
4021021132
4021021126
1320203187
1320023154
at first i thought it might have been the number of digits but other numbers with this many digits work, but i assume it's more to do with the "type" of number and that the value exceeds the limitation of that number type.
how do i overcome this problem?
I am writing a program that will automatically process files, based on a set of criteria. In order to process these, I need to load the Excel file first, before I can process it.
I create a module called "NewBusAuto". In here, I created a new datagridview. I am trying to set the datasource of this, but whenever I do, it doesn't actually bind. When I try and process each row/column, it is saying it is still empty.
Code used:
Module m_NewBusinessAutomation
Dim dgvImport As DataGridView
Public Sub NewBusAuto(ByVal folderName As String,
ByVal fileName As String,
ByVal executeScript As String)
dgvImport = New DataGridView
If Path.GetExtension(fileName) = ".xls" Or Path.GetExtension(fileName) = ".xlsx" Then
Using cn As New System.Data.OleDb.OleDbConnection
Dim builder As New OleDbConnectionStringBuilder With _
{.DataSource = folderName & "\" & fileName,
.Provider = "Microsoft.ACE.OLEDB.12.0"}
builder.Add("Extended Properties", "Excel 12.0; IMEX=1;HDR=Yes;")
cn.ConnectionString = builder.ConnectionString
cn.Open()
Using cmd As OleDbCommand = New OleDbCommand With {.Connection = cn}
cmd.CommandText = "SELECT * FROM [Sheet1$]"
Dim dr As System.Data.IDataReader = cmd.ExecuteReader
Dim dt As New DataTable
dt.Load(dr)
dgvImport.DataSource = dt
dgvImport.Refresh()
Messagebox.show(dgvImport.RowCount & "-" & dgvImport.ColumnCount)
dr.Close()
End Using
End Using
End If
The messagebox is giving me "0 - 0" as a result.
I need to load the Excel file to the actual database. Any help would be appreciated!
(I don't need alternatives to OleDbDataAdapter.)
The code below finds and reads the file OK but the DGV has four columns (as expected) but all the data rows just have text in the first column.
Dim sDir As String = "c:\temp\"
Dim sConn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & sDir & ";Extended Properties='text;HDR=Yes;FMT=TabDelimited';"
Dim dt As New DataTable()
Using adapt As New OleDbDataAdapter(String.Format("SELECT TOP 100 * FROM robo.txt"), sConn)
adapt.Fill(dt)
End Using
DataGridView1.DataSource = dt
I would think the Extended Properties would be the only requirement. I've tried add a Schema.ini to no avail - I don't think it is even being read as the column headers never match the schema.
The header row in the most successful pass used commas as separator - this resulted in four columns with the proper names but the tab separated data all in Col1. If I use tabs in the header row I get some system assign columns (3) which sort of corresponds to a data row with two commas.
What am I doing wrong?
Here are the first few rows with the tab character being replaced by <tab> . I since noticed that I have an extra column in the data. The fix to the header row below did not fix the problem - all data is dumped into the first field.
Use a tab separator in the header, instead of commas, results in all header text and the data being dumped into the first field.
col1,state,col3,size,path
<tab> same<tab><tab> 102912<tab>\\APCD04T\Data\Thumbs.db
<tab> same<tab><tab> 22016<tab>\\APCD04T\Data\APCD Topical Info\APCD_Boards&Committees_List.doc
<tab> same<tab><tab> 4.3 m<tab>\\APCD04T\Data\APCD Topical Info\LOSSAN-LAtoSLORailCorridorStrategicPlan.pdf
Learned several things while trying to load a RoboCopy log into a DataTable using OLEDB.
log file needs to have a .txt or .csv (or ?) extension, .log fails.
Schema.ini seems to be needed for tab delimited robocopy log, good for column definition anyway.
Datagridview takes a long time to display 30MB of data so I used
filters
I borrowed code from the net to create a Schema.ini as noted below
(SO bug: code will not paste from Visual Studio anymore. Code tool flips to other web page for Java.)
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Try
Cursor = Cursors.WaitCursor
'http://ss64.com/nt/robocopy.html can suppress header and summary
Dim sFile As String = "c:\temp\robo.txt" ' seems to need a .txt or .csv, .log didn't work
CreateRoboLogSchema(sFile) ' recreates each pass, no needed once things work
Dim sConn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & IO.Path.GetDirectoryName(sFile) & ";Extended Properties='text';"
' use Schema.ini for: HDR=Yes;FMT=TabDelimited' and column definitions
Dim dt As New DataTable()
Dim SQL As String = "SELECT * FROM " & IO.Path.GetFileName(sFile)
'SQL &= " WHERE State <> 'Same'"
Using adapt As New OleDbDataAdapter(SQL, sConn)
adapt.Fill(dt)
End Using
Debug.Print("|" & dt.Rows(0)(1) & "|") ' show import trimmed leading spaces (trims trailing too)
' DGV slow to load large files, use filter to display target rows
Dim dv As New DataView(dt)
dv.RowFilter = "State <> 'Same'" ' not case sensitive
DataGridView1.DataSource = dv
DataGridView1.Columns(0).Visible = False
DataGridView1.AutoResizeColumns()
Catch ex As Exception
MsgBox(ex.Message)
Finally
'Cursor=Cursors.Default
End Try
End Sub
Private Function CreateRoboLogSchema(ByVal strFileName As String) As Boolean
' edit http://www.vb-tips.com/CSVDataSet.aspx
Dim ascii As System.Text.Encoding = System.Text.Encoding.ASCII
Dim swSchema As System.IO.StreamWriter = Nothing
Dim blnReturn As Boolean
Dim strSchemaPath As String = System.IO.Path.GetFileName(strFileName)
Try
strSchemaPath = IO.Path.GetDirectoryName(strFileName) & "\Schema.ini"
swSchema = My.Computer.FileSystem.OpenTextFileWriter(strSchemaPath, False, ascii)
Dim strFile As String = System.IO.Path.GetFileName(strFileName)
swSchema.WriteLine("[" & IO.Path.GetFileName(strFileName) & "]")
swSchema.WriteLine("ColNameHeader=False")
swSchema.WriteLine("Format=TabDelimited")
swSchema.WriteLine("Col1=Value1 Text") ' file specific
swSchema.WriteLine("Col2=State Text")
swSchema.WriteLine("Col3=DirChanges Text")
swSchema.WriteLine("Col4=Size Text")
swSchema.WriteLine("Col5=Filepath Text")
'Continue for all fields
blnReturn = True
Catch ex As Exception
blnReturn = False
Finally
If swSchema IsNot Nothing Then
swSchema.Close()
End If
End Try
Return blnReturn
End Function
I have an excel 2007 xlsm file, where on one of the tabs I have several data tables. Using VB.NET, I'm trying to read one table at a time as a named range like so:
Public Function OpeDataFromRange(ByVal Filename as string, ByVal RangeName As String, ByVal bColumnNames As Boolean) as DataTable
' Returns a DataSet containing information from a named range
' in the passed Excel worksheet
Dim sHDR As String
Dim strConn As String
If bColumnNames Then
sHDR = "Yes"
Else
sHDR = "No"
End If
strConn = "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=" & Filename & ";Extended Properties=""Excel 12.0 Macro;HDR=" & sHDR & """;"
Dim objConn _
As New System.Data.OleDb.OleDbConnection(strConn)
objConn.Open()
' Create objects ready to grab data
Dim objCmd As New System.Data.OleDb.OleDbCommand( _
"SELECT * FROM [" & RangeName & "]", objConn)
Dim objDA As New System.Data.OleDb.OleDbDataAdapter()
objDA.SelectCommand = objCmd
' Fill DataSet
Dim objDS As New System.Data.DataSet()
objDA.Fill(objDS)
' Clean up and return DataSet
objConn.Close()
return objDS
End Function
But I'm getting the error at Fill command:
The Microsoft Office Access database engine could not find the object 'MyNamedTable1'. Make sure the object exists and that you spell its name and the path name correctly.
I tried to read the entire sheet in the SELECT, and then to fish out my table through objDS.Tables, but then Tables gets loaded with only one table with everything in it.
Any Recommendations?
You cannot use Microsoft.Jet.OLEDB.4.0 with Excel 12.0 you should use Microsoft.ACE.OLEDB.12.0 instead.
Incidentally, you are filling a DataSet but returning a DataTable you need to change one of those.
Personally I prefer to use a DataTable for this but you may prefer a DataSet. If you want to use a DataTable you can...
Dim objDT As New DataTable
objDT.Load(objCmd.ExecuteReader)
I am trying to import a large array of integers stored as a csv file into a VB.Net DataTable called BeamMap. The .csv file consists only of integers, with a delimiter of ,, no quotes around the data (ie., 1,3,-2,44,1), and an end of line character of line feed and carriage return. All I want to do is get each integer into a DataTable cell with the appropriate rows and columns (there are the same number of columns for each row) and be able to reference it later on in my code. I really don't want anything more than absolutely necessary in the code (no titles, captions, headings, etc.), and I need it to be fairly efficient (the csv array is approx. ~1000 x ~1000).
Thanks!
Use OleDb provider to read CSV and pouplate the DataTable.
Dim folder = "c:\location\of\csv\files\"
Dim CnStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & folder & ";Extended Properties=""text;HDR=No;FMT=Delimited"";"
Dim dt As New DataTable
Using Adp As New OleDbDataAdapter("select * from [nos.csv]", CnStr)
Adp.Fill(dt)
End Using
Here's a simple approach which requires a strict format (as you've mentioned):
Dim lines = IO.File.ReadAllLines(path)
Dim tbl = New DataTable
Dim colCount = lines.First.Split(","c).Length
For i As Int32 = 1 To colCount
tbl.Columns.Add(New DataColumn("Column_" & i, GetType(Int32)))
Next
For Each line In lines
Dim objFields = From field In line.Split(","c)
Select CType(Int32.Parse(field), Object)
Dim newRow = tbl.Rows.Add()
newRow.ItemArray = objFields.ToArray()
Next
Getting the file from a mapped drive and putting the retrieved data in a dataset:
Dim folder = "Z:\"
Dim CnStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & folder & ";Extended Properties=""text;HDR=No;FMT=Delimited"";"
Dim dssample As New DataSet
Using Adp As New OleDbDataAdapter("select * from [samplecsv.csv]", CnStr)
Adp.Fill(dssample)
End Using
If dssample.Tables.Count > 0 Then
'some code here
End If
Also, don't forget to include the
Imports System.Data.OleDb
And if you wish to link to a DataGridView (after read):
Dim bs As New BindingSource
bs.DataSource = dt
DataGridView1.DataSource = bs