Opening a CSV file as a OLEDB Connection converts file text to double - vb.net

I'm opening up a couple of CSV files and reading them in as a DataTable per the example I found here. The issue I am running into it the basic query I'm using to import the data is converting the column of IP addresses into Doubles. So I want to read in 10.0.0.1 and it shows up as 10.001. How can I get this column to read in as a string? I would like to not double process the file if I can.
Query I'm using is basic and is as follows:
SELECT * FROM [ComputerList.csv]
Here is my function to open and read the CSV file into a DataTable
Public Function OpenFile(ByVal strFolderPath as String, ByVal strQuery as String) as DataTable
Dim strConn as String = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & strFolderPath & ";Extended Propteries=""text; HDR=Yes;FMT=Delimited"""
Dim conn as OleDb.OleDbConnection = New OleDb.OleDbConnection(strConn)
Try
conn.Open()
Dim cmd as OleDb.OleDbCommand = New OleDb.OleDbCommand(strQuery, conn)
Dim da as OleDb.OleDbDataAdapter = New OleDb.OleDbDataAdapter()
da.SelectCommand = cmd
Dim ds as DataSet = New DataSet()
da.Fill(ds)
da.Dispose()
return ds.Tables(0)
Catch
return Nothing
Finally
conn.Close()
End Try
End Function

i use this to read csv's and force all to string.
Public Function convert_csv_to_data_table(ByVal File As String, ByVal separator As String) As DataTable
Dim dt As New DataTable
Dim firstLine As Boolean = True
If IO.File.Exists(File) Then
Using sr As New StreamReader(File)
While Not sr.EndOfStream
If firstLine Then
firstLine = False
Dim cols = sr.ReadLine.Split(separator)
For Each col In cols
dt.Columns.Add(New DataColumn(col, GetType(String)))
Next
Else
Dim data() As String = sr.ReadLine.Split(separator)
dt.Rows.Add(data.ToArray)
End If
End While
End Using
End If
Return dt
End Function
EDIT:- this will only work with a separator btw

ok I tried variations of everyone's suggested and have settled on a hybrid between all of them. My goal was to read in a CSV file manipulate it in a DataTable form and them write it back out. Some of my CSV files had multiple lines with in a cell and some had deliminators within a cell. Below you can find my hybrid solution that utilized TextFieldParser to read in the file and break it up.
Public Function OpenFile(ByVal File as String, NyVal delim as String) as DataTable
Dim dt as New DataTable()
Dim firstline as Boolean = True
Using MyReader as New Microsoft.VisualBasic.FileIO.TextFieldParser(File)
MyReader.TextFieldType = FileIO.FieldType.Delimited
MyReader.SetDelimiters(delim)
Dim currentRow as String()
While Not MyReader.EndOfData
Try
currentRow = MyReader.ReadFields()
If firstline
firstline = false
For Each col in currentRow
dt.Columns.Add(New DataColumn(col.ToString(), System.Type.GetType("System.String")))
Next
Else
dt.Rows.Add(currentRow.ToArray())
End If
Catch ex as Microsoft.VisualBasic.FileIO.MalformedLineException
Console.WriteLIne("Line " + ex.Message + " is not valid and will be skipped")
End Try
End While
End Using
return dt
End Function

Related

oracle bulkcopy from vb.net databale

I am loading the CSV file to vb.net datatable .
then I am using Bulk copy command to write to database.
The problem is if one date cell in the data table is empty, i am receiving
ORA:-01840 input value not long enough for date format.
How to resolve this.
Public Class Form1
Dim cn As New OracleConnection("Data Source =(DESCRIPTION =(ADDRESS = (PROTOCOL = TCP)(HOST = ipaddressofserver)(PORT = portofserver))(CONNECT_DATA =(SERVER = DEDICATED)(SERVICE_NAME = oracleservicename)) ) ; User Id=useridofdb;Password=passwordofdb")
cn.Open()
MsgBox("connection opened")
Dim dt As DataTable = ReadCSV("D:\test.csv")
DataGridView1.DataSource = dt
Try
Dim _bulkCopy As New OracleBulkCopy(cn)
_bulkCopy.DestinationTableName = "TEST_TABLE"
_bulkCopy.BulkCopyTimeout = 10800
_bulkCopy.WriteToServer(dt)
cn.Close()
cn.Dispose()
cn = Nothing
MsgBox("Finished")
Catch ex As Exception
MsgBox(ex.Message)
End Try
End Sub
Function ReadCSV(ByVal path As String) As System.Data.DataTable
Try
Dim sr As New StreamReader(path)
Dim fullFileStr As String = sr.ReadToEnd()
sr.Close()
sr.Dispose()
Dim lines As String() = fullFileStr.Split(ControlChars.Lf)
Dim recs As New DataTable()
Dim sArr As String() = lines(0).Split(","c)
For Each s As String In sArr
recs.Columns.Add(New DataColumn())
Next
Dim row As DataRow
Dim finalLine As String = ""
For Each line As String In lines
row = recs.NewRow()
finalLine = line.Replace(Convert.ToString(ControlChars.Cr), "")
row.ItemArray = finalLine.Split(","c)
recs.Rows.Add(row)
Next
Return recs
Catch ex As Exception
Throw ex
End Try
End Function
End Class

When I try to pull nvarchar column value that has an emojis stored in it, the data in VB.net it displays as '??'

When I try to pull the emoji value from the database using VB.net the emoji is displayed as '??'
Is there some sort of conversion I need to do in my stored procedure?
Private Function DisplayEmoji(input As String) As String
Dim output As New StringBuilder()
Dim enumerator = StringInfo.GetTextElementEnumerator(input)
While enumerator.MoveNext()
Dim chunk As String = enumerator.GetTextElement()
If Char.IsSurrogatePair(chunk, 0) Then
output.Append("<img src=""" + "https://abs.twimg.com/emoji/v1/72x72/" + Char.ConvertToUtf32(chunk, 0).ToString("x") + ".png"" style=""height:1.5em; width:1.5em;"">")
Else
output.Append(chunk)
End If
End While
Return output.ToString()
End Function
Or insert your image Blob type and read sql.
Dim filePath As String = Server.MapPath("APP_DATA/test.png")
Dim filename As String = Path.GetFileName(filePath)
Dim fs As FileStream = New FileStream(filePath, FileMode.Open, FileAccess.Read)
Dim br As BinaryReader = New BinaryReader(fs)
Dim bytes As Byte() = br.ReadBytes(Convert.ToInt32(fs.Length))
br.Close()
fs.Close()
Insert to Database yor image
Dim strQuery As String = "insert into tblFiles(Name, ContentType, Data) values (#Name, #ContentType, #Data)"
Dim cmd As SqlCommand = New SqlCommand(strQuery)
cmd.Parameters.Add("#Name", SqlDbType.VarChar).Value = filename
cmd.Parameters.Add("#ContentType", SqlDbType.VarChar).Value = "image/png"
cmd.Parameters.Add("#Data", SqlDbType.Binary).Value = bytes
InsertUpdateData(cmd)
InsertUpdateData
Public Function InsertUpdateData(ByVal cmd As SqlCommand) As Boolean
Dim strConnString As String = System.Configuration.
ConfigurationManager.ConnectionStrings("conString").ConnectionString
Dim con As New SqlConnection(strConnString)
cmd.CommandType = CommandType.Text
cmd.Connection = con
Try
con.Open()
cmd.ExecuteNonQuery()
Return True
Catch ex As Exception
Response.Write(ex.Message)
Return False
Finally
con.Close()
con.Dispose()
End Try
End Function

convert csv data to DataTable in VB.net, capturing column names from row 0

I've adapted the code from the #tim-schmelter answer to question convert csv data to DataTable in VB.net (see below)
I would like to parse in the column titles from row 0 of the csv file
DT|Meter Number|Customer Account Number|Serial Number|Port...
but I'm not having any luck trying to figure out how to do this. any suggestions would be very appreciated.
Public Function csvToDatatable_2(ByVal filename As String, ByVal separator As String)
'////////////////////////////////////////
'Reads a selected txt or csv file into a datatable
'based on code from http://stackoverflow.com/questions/11118678/convert-csv-data-to-datatable-in-vb-net
'////////////////////////////////////////
Dim dt As System.Data.DataTable
Try
dt = New System.Data.DataTable
Dim lines = IO.File.ReadAllLines(filename)
Dim colCount = lines.First.Split(separator).Length
For i As Int32 = 1 To colCount
dt.Columns.Add(New DataColumn("Column_" & i, GetType(String)))
Next
For Each line In lines
Dim objFields = From field In line.Split(separator)
Dim newRow = dt.Rows.Add()
newRow.ItemArray = objFields.ToArray()
Next
Catch ex As Exception
Main.Msg2User(ex.Message.ToString)
Return Nothing
End Try
Return dt
End Function
Just loop thru all the line of the file. Use a boolean to check for the first row.
Public Function csvToDatatable_2(ByVal filename As String, ByVal separator As String)
Dim dt As New System.Data.DataTable
Dim firstLine As Boolean = True
If IO.File.Exists(filename) Then
Using sr As New StreamReader(filename)
While Not sr.EndOfStream
If firstLine Then
firstLine = False
Dim cols = sr.ReadLine.Split(separator)
For Each col In cols
dt.Columns.Add(New DataColumn(col, GetType(String)))
Next
Else
Dim data() As String = sr.Readline.Split(separator)
dt.Rows.Add(data.ToArray)
End If
End While
End Using
End If
Return dt
End Function
Here is a hybrid of the two solutions above, with a few other changes:
Public Shared Function FileToTable(ByVal fileName As String, ByVal separator As String, isFirstRowHeader As Boolean) As DataTable
Dim result As DataTable = Nothing
Try
If Not System.IO.File.Exists(fileName) Then Throw New ArgumentException("fileName", String.Format("The file does not exist : {0}", fileName))
Dim dt As New System.Data.DataTable
Dim isFirstLine As Boolean = True
Using sr As New System.IO.StreamReader(fileName)
While Not sr.EndOfStream
Dim data() As String = sr.ReadLine.Split(separator, StringSplitOptions.None)
If isFirstLine Then
If isFirstRowHeader Then
For Each columnName As String In data
dt.Columns.Add(New DataColumn(columnName, GetType(String)))
Next
isFirstLine = True ' Signal that this row is NOT to be considered as data.
Else
For i As Integer = 1 To data.Length
dt.Columns.Add(New DataColumn(String.Format("Column_{0}", i), GetType(String)))
Next
isFirstLine = False ' Signal that this row IS to be considered as data.
End If
End If
If Not isFirstLine Then
dt.Rows.Add(data.ToArray)
End If
isFirstLine = False ' All subsequent lines shall be considered as data.
End While
End Using
Catch ex As Exception
Throw New Exception(String.Format("{0}.CSVToDatatable Error", GetType(Table).FullName), ex)
End Try
Return result
End Function

The process cannot access the file 'file.csv' because it is being used by another process

I have a code which process each file in a folder and send the data in the file to SQL Database and then deletes the file. But often i get this Exception "The process cannot access the file 'file.csv' because it is being used by another process." Please can anyone point me out in the right direction, will really appreciate that.
The code is shown below :
Dim dirinfo As DirectoryInfo
Dim allFiles() As FileInfo
dirinfo = New DirectoryInfo("E:\SQLUPDATE\CAC")
allFiles = dirinfo.GetFiles("*.csv")
If allFiles.Length <> 0 Then
Try
For Each fl As FileInfo In allFiles
'MsgBox(fl.FullName.ToString())
Dim con As SqlConnection = New SqlConnection(SQL_con2)
Dim sr As StreamReader = New StreamReader(fl.FullName)
Dim line As String = sr.ReadLine
Dim value() As String = line.Split(Microsoft.VisualBasic.ChrW(44))
Dim dt As DataTable = New DataTable
Dim row As DataRow
For Each dc As String In value
dt.Columns.Add(New DataColumn(dc))
Next
While Not sr.EndOfStream
value = sr.ReadLine.Split(Microsoft.VisualBasic.ChrW(44))
If (value.Length = dt.Columns.Count) Then
row = dt.NewRow
row.ItemArray = value
dt.Rows.Add(row)
End If
End While
Dim bc As SqlBulkCopy = New SqlBulkCopy(con.ConnectionString, SqlBulkCopyOptions.TableLock)
bc.DestinationTableName = "[DB].[dbo].[CAC_LData]"
bc.BatchSize = dt.Rows.Count
con.Open()
bc.WriteToServer(dt)
bc.Close()
con.Close()
sr.Close()
System.IO.File.Delete(fl.FullName)
sr.Dispose()
Next
Catch ex As Exception
MsgBox(ex.Message)
End Try
End If

trouble returning datatable within a try statement

I have the following piece of code that I am using to try and rip a csv file and turn it into a datatable. My problem is that the debugger never makes it to the return statement. Everything is appending to the datatable correctly, so I know that part works. Any idea's on what I can do to trouble shoot this further. Also, if you know of a simpler way to turn a import a csv file to a datatable I'd be very interested in learning about it.
Thanks!
Public Function loadCSVTableII() As DataTable
Dim dt As New DataTable("TableII")
Dim line As String = String.Empty
Dim counter As Integer = 0
Dim reader As New StreamReader(pathTableTwo)
Try
While Not IsNothing(line)
line = reader.ReadLine()
Dim lineSep As String() = line.Split(New Char() {","c})
If Not counter = 0 Then
dt.Rows.Add(lineSep)
counter += 1
Else
For Each value As String In lineSep
dt.Columns.Add(value)
Next
counter += 1
End If
End While
'cursor never gets to this code block...
Dim primarykey(0) As DataColumn
primarykey(0) = dt.Columns("Ages")
dt.PrimaryKey = primarykey
Return dt
Catch ex As Exception
Throw
End Try
End Function
Update: It is erroring out on this line in the code.
Dim lineSep As String() = line.Split(New Char() {","c})
It say that the Object reference is not set to an instance of an object. What's weird though is that it works through the whole data table fine. Could it be that the while loop is not terminating at the end of the file?
Try changing your While loop to handle the end of stream condition. It's not very clear what the IsNothing function is doing in your code.
While Not reader.EndOfStream
line = reader.ReadLine
'// Dim lineSep As String() = line.Split(New Char() {","c})
For your line split, in VB.Net, it's simple to just do this:
Dim lineSep As String() = line.Split(",")
You can use OLEDB provider for this.
string query = "SELECT Symbol, [Name of Company], FROM [just file name with extension]";
string connStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + [csv file path without file name] + ";" + "Extended Properties=’text;HDR=YES;’";
//create dataadapter object
OleDbDataAdapter adapter = new OleDbDataAdapter(query, connStr);
// create table
DataTable dtSymbolDetails = new DataTable("ScriptDetails");
dtSymbolDetails.Columns.Add("Symbol");
dtSymbolDetails.Columns.Add("Name of Company");
// fill the table with data using adapter
adapter.Fill(dtDetails);