2-D array from txt in VB.NET - vb.net

I am needing to create a code that is versatile enough where I can add more columns in the future with minimum reconstruction of my code. My current code does not allow me to travel through my file with my 2-D array. If I was to change MsgBox("map = "+ map(0,1) I can retrieve the value easily. Currently all I get in the code listed is 'System.IndexOutOfRangeException' and that Index was outside the bounds of the array. My current text file is 15 rows (down) and 2 columns (across) which puts it at a 14x1. they are also comma separated values.
Dim map(14,1) as string
Dim reader As IO.StreamReader
reader = IO.File.OpenText("C:\LocationOfTextFile")
Dim Linie As String, x,y As Integer
For x = 0 To 14
Linie = reader.ReadLine.Trim
For y = 0 To 1
map(x,y) = Split(Linie, ",")(y)
Next 'y
Next 'x
reader.Close()
MsgBox("map = " + map(y,x))``

Here's a generic way to look at reading the file:
Dim data As New List(Of List(Of String))
For Each line As String In IO.File.ReadAllLines("C:\LocationOfTextFile")
data.Add(New List(Of String)(line.Split(",")))
Next
Dim row As Integer = 1
Dim col As Integer = 10
Dim value As String = data(row)(col)

This is the method suggested by Microsoft. It is generic and will work on any properly formatted comma delimited file. It will also catch and display any errors found in the file.
Using MyReader As New Microsoft.VisualBasic.
FileIO.TextFieldParser(
"C:\LocationOfTextFile")
MyReader.TextFieldType = FileIO.FieldType.Delimited
MyReader.SetDelimiters(",")
Dim currentRow As String()
While Not MyReader.EndOfData
Try
currentRow = MyReader.ReadFields()
Dim currentField As String
For Each currentField In currentRow
MsgBox(currentField)
Next
Catch ex As Microsoft.VisualBasic.
FileIO.MalformedLineException
MsgBox("Line " & ex.Message &
"is not valid and will be skipped.")
End Try
End While
End Using

Essentially what you are asking is how can I take the contents of comma-separated values and convert this to a 2D array.
The easiest way, which is not necessarily the best way, is to return an IEnuemrable(Of IEnumerable(Of String)). The number of items will grow both vertically based on the number of lines and the number of items will grow horizontally based on the values split on a respective line by a comma.
Something along these lines:
Private Function GetMap(path As String) As IEnumerable(Of IEnumerable(Of String)
Dim map = New List(Of IEnumerable(Of String))()
Dim lines = IO.File.ReadAllLines(path)
For Each line In lines
Dim row = New List(Of String)()
Dim values = line.Split(","c)
row.AddRange(values)
map.Add(row)
Next
Return map
End Function
Now when you want to grab a specific cell using the (row, column) syntax, you could use:
Private _map As IEnumerable(Of IEnumerable(Of String))
Private Sub LoadMap()
_map = GetMap("C:/path-to-map")
End Sub
Private Function GetCell(row As Integer, column As Integer) As String
If (_map Is Nothing) Then
LoadMap()
End If
Return _map.ElementAt(row).ElementAt(column)
End Function
Here is an example: https://dotnetfiddle.net/ZmY5Ki
Keep in mind that there are some issues with this, for example:
What if you have commas in your cells?
What if you try to access a cell that doesn't exist?
These are considerations you need to make when implementing this in more detail.

You can consider the DataTable class for this. It uses much more memory than an array, but gives you a lot of versatility in adding columns, filtering, etc. You can also access columns by name rather than index.
You can bind to a DataGridView for visualizing the data.
It is something like an in-memory database.

This is much like #Idle_Mind's suggestion, but saves an array copy operation and at least one allocation per row by using an array, rather than a list, for the individual rows:
Dim data = File.ReadLines("C:\LocationOfTextFile").
Select(Function(ln) ln.Split(","c)).
ToList()
' Show last row and column:
Dim lastRow As Integer = data.Count - 1
Dim lastCol As Integer = data(row).Length - 1
MsgBox($"map = {data(lastRow)(lastCol)}")
Here, assuming Option Infer, the data variable will be a List(Of String())
As a step up from this, you could also define a class with fields corresponding to the expected CSV columns, and map the array elements to the class properties as another call to .Select() before calling .ToList().
But what I really recommend is getting a dedicated CSV parser from NuGet. While a given CSV source is usually consistent, more broadly the format is known for having a number of edge cases that can easily confound the Split() function. Therefore you tend to get better performance and consistency from a dedicated parser, and NuGet has several good options.

Related

New to programming language, need help to read a .dat file into array Vs VB, console .net framework

I am trying to read from a data file called TXT.dat and store the circled values into a separate array's using the type of commands I have used. I cannot use streamreader,streamwriter this is what I am learning at university but we were taught to read an array not certain values as I will be tested using a file which has
2 factorial values from a single file and store int two arrays.
I have spent hours trying before going to paid sites who always answer with i/o streamreader which doesnt help
.dat file i created to practice
MY CODE
Sub Main()
Const i_val As Integer = 6
Dim j As Integer = 6 'loop readers
Dim Arayn_Fact(i_val - 1) As Double 'array for 2nd value per line
Dim Aray_Fact2n(j - 1) As Double 'array for 4th value per line
Read_Values(i_val - 1, Arayn_Fact)
End Sub
Sub Read_Values(ByVal i As Integer, ByRef _A() As Double)
Dim fid1 As Integer = FreeFile()
Dim fid2 As Integer = FreeFile() + 1
Dim tmp As Double
FileOpen(fid1, "TXT.dat", OpenMode.Input, OpenAccess.Read)
For i = 0 To 5 Step 1
Input(fid1, tmp)
Input(fid1, _A(i))
Next i
FileClose(fid1)
Console.ReadKey()
End Sub
Without getting too fancy, here's some very basic code to do what you've outlined:
Sub Main()
Dim fileName As String = "txt.dat"
Dim Aray_Fact2() As Integer 'array for 2nd value per line
Dim Aray_Fact4() As Integer 'array for 4th value per line
Dim list2 As New List(Of Integer)
Dim list4 As New List(Of Integer)
Dim lines() As String = System.IO.File.ReadAllLines(fileName)
For Each line As String In lines
Dim values() As String = line.Split(",")
If values.Length >= 3 Then
Dim f2, f4 As Integer
If Integer.TryParse(values(1), f2) AndAlso Integer.TryParse(values(3), f4) Then
list2.Add(f2)
list4.Add(f4)
Else
Console.WriteLine("Invalid value in line: " & line)
End If
Else
Console.WriteLine("Invalid number of entries in line: " & line)
End If
Next
Aray_Fact2 = list2.ToArray
Aray_Fact4 = list4.ToArray
Console.WriteLine("Aray_Fact2: " & String.Join(",", Aray_Fact2))
Console.WriteLine("Aray_Fact4: " & String.Join(",", Aray_Fact4))
Console.Write("Press Enter to Quit...")
Console.ReadLine()
End Sub
If you are completely against the use of Lists and ToArray, then you'd have to read the file TWICE. Once to determine how many lines are in the file so you can dimension the arrays to the correct size. Then another time to actually read the values and populate the arrays.
First, some style things. Passing the array ByRef is incorrect. Arrays are already reference types, and so passing ByVal still passes the reference to the array object. But don't even pass the array in the first place. It's better practice to let the Read_Values() method create and return the array to the calling function.
And no one uses that ancient/ganky FileOpen() API anymore. I mean that. NO. ONE. It's not even appropriate for teaching, as the Stream types are closer to what the low level operating system/file system are doing. If you can't use StreamReader and friends, there are still better options out there.
Sub Main()
Dim Array_Fact() As Double = Read_Values("TXT.dat", 1)
Dim Array_Fact2n() As Double = Read_Values("TXT.dat", 3)
Console.ReadKey(True)
End Sub
Function Read_Values(filePath As String, position As Integer) As Double()
Dim lines() As String = File.ReadAllLines(filePath) 'Look Ma, no StreamReader
Dim result(lines.Length - 1) As Double
For i As Integer = 0 To Lines.Length - 1
result(i) = Double.Parse(lines(i).Split(",")(position).Trim())
Next line
Return result
End Function
But what I'd really do is keep the 2nd and 4th values in the same collection and read everything in one pass through the file. This uses features like Generics, Tuples, Lambdas+Linq, and more that you won't get to for some time, but I feel like it's useful to show you where you're going:
Sub Main()
Dim Facts = Read_Values("TXT.dat")
Console.WriteLine(String.Join(",",Facts.Select(Function(f) $"({f.Item1}:{f.Item2})")))
Console.ReadKey(True)
End Sub
Function Read_Values(filePath As String) As IEnumerable(Of (Double, Double))
Return File.ReadLines(filePath).Select(
Function(line) 'Poor man's CSV. In real code, I'd pull in a CSV parser from NuGet
Dim fields = line.Split(",")
Return (Double.Parse(fields(1).Trim()), Double.Parse(fields(3).Trim()))
End Function
)
End Function

I am trying to figure out how to read a text file into an array in vb.net

I'm trying to create a program in vb.net (forms) to process data from a UVvis spectrometer.
The txt file output looks as following.
"180809_QuartzRefTrans.spc - RawData"
"Wavelength nm.","T%"
400.00,90.822
401.00,90.800
402.00,90.823
403.00,90.811
404.00,90.803
405.00,90.804
406.00,90.816
407.00,90.811
408.00,90.833
409.00,90.837
410.00,90.847
411.00,90.827
412.00,90.839
413.00,90.851
414.00,90.828
415.00,90.879
416.00,90.846
and so on.
What I am trying to do is read the data it into an array so I can manipulate the columns. I need to be able to skip the first two lines so that all I have is numerical data. I also need it to sort the array from lowest to highest (wavelengths). Sometimes we run from 800->200 nm then accidentally put in 200->800 nm
Imports System.IO
Imports System.Windows.Forms.DataVisualization.Charting
Public Class Form1
Public Class RefTrans
Public Property Wavelength As Double
Public Property Transpercent As Double
End Class
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim strtext As String
OpenFileDialog1.Title = "Open Text Files"
OpenFileDialog1.ShowDialog()
strtext = OpenFileDialog1.FileName
TextBox1.Text = My.Computer.FileSystem.GetName(strtext)
Label1.Text = My.Computer.FileSystem.GetName(strtext)
Dim line1 As String
Dim output1 As New ArrayList
Using sr As New IO.StreamReader(strtext)
sr.ReadLine()
sr.ReadLine()
Do While sr.Peek() >= 0
line1 = sr.ReadLine()
output1.Add(line1)
Loop
End Using
If strtext <> "" Then
Dim SR As New StreamReader(strtext)
SR.ReadLine()
Do Until SR.EndOfStream
TextBox3.Text = TextBox3.Text & SR.ReadLine & vbCrLf
Loop
SR.Close()
End If
Dim data1 = IO.File.ReadLines(strtext).
Skip(2).
Select(Function(line)
Dim parts = line.Split(","c)
Return New RefTrans With {.Wavelength = CDbl(parts(0)),
.Transpercent = CDbl(parts(1))}
End Function).
ToArray() = line.Split(","c)
End Sub
End Class
Here's an example that will get you an array of Tuple(Of Double, Double):
Dim data = IO.File.ReadLines(filePath).
Skip(2).
Select(Function(line)
Dim parts = line.Split(","c)
Return Tuple.Create(CDbl(parts(0)), CDbl(parts(1)))
End Function).
ToArray()
The IO.File.ReadLines method will read the lines of the file. Where the ReadAllLines method reads the entire file and then returns the lines as a String array, the ReadLines method exposes the lines of the file as an enumerable list and the lines only get read from the file as you use them. Particularly for big files, ReadLines is more efficient than ReadAllLines because it does create an intermediate array first that you don't actually want.
The Skip method allows you to access the items in a list starting at a particular index. In the case of reading a file, you need to read every line regardless but call Skip(2) on the result of File.ReadLines means that the first two lines of the file will be discarded after reading and any subsequent processing will only but done on lines from the third.
The Select method is basically a transformation. It says create an output list containing an item for every item in the input list where the output item is the result of a transformation of the input item. The transformation is defined by the function you provide. In this case, a line from the file is the input and the transformation is to split that line on the comma, convert the two substrings to Double values and then create a Tuple containing those two Double values.
The ToArray method takes any enumerable list, i.e. any object that implements IEnumerable(Of T), and returns an array containing the items from that list. In this case, Select returns an IEnumerable(Of Tuple(Of Double, Double)) so ToArray returns a Tuple(Of Double, Double) array.
If you wanted to write that all out long-hand then it would look like this:
'Get all the lines of the file.
Dim step1 As IEnumerable(Of String) = IO.File.ReadLines(filePath)
'Skip the first two lines.
Dim step2 As IEnumerable(Of String) = step1.Skip(2)
'Split each line into two substrings.
Dim step3 As IEnumerable(Of String()) = step2.Select(Function(line) line.Split(","c))
'Convert substrings to numbers and combine.
Dim step4 As IEnumerable(Of Tuple(Of Double, Double)) = step3.Select(Function(parts) Tuple.Create(CDbl(parts(0)), CDbl(parts(1))))
'Create an array of Tuples.
Dim data As Tuple(Of Double, Double)() = step4.ToArray()
A Tuple is basically a general-purpose object for grouping values together. You can use a Tuple for one-off cases rather than defining your own class. You may prefer to define your own class in all cases though, and you should define your own class where you will use the data extensively. In this case, your own class might look like this:
Public Class RefTrans
Public Property Wavelength As Double
Public Property TransPercent As Double
End Class
and the code would then become:
Dim data = IO.File.ReadLines(filePath).
Skip(2).
Select(Function(line)
Dim parts = line.Split(","c)
Return New RefTrans With {.Wavelength = CDbl(parts(0)),
.TransPercent = CDbl(parts(1))}
End Function).
ToArray()
In that case, you have properties Wavelength and TransPercent on each element of your array. If you use Tuples then the properties have the generic names Item1 and Item2.
Once you have the array, you can use any appropriate overload of Array.Sort to do the sorting, e.g.
Array.Sort(data, Function(d1, d2) d1.Item1.CompareTo(d2.Item1))
That will sort the Tuples by comparing their Item1 properties, which contain the wavelength values. If you're using your own class then obviously you specify your own property, e.g. the Wavelength property in the RefTrans class I showed.

Reading data to multidimensional array

I have to read data from database to multidimensional array of unknown size.
Dim myarray As String()
Using reader = mcmd.ExecuteReader()
While (reader.Read())
myarray = TryCast(reader("mydataarray"), String())
End While
End Using
Here I got in myarray array of strings of last readed row only.
How to get that in 'myarray' will be readed data of all rows from sql result?
Here is code to do what I think you mean:
Using reader = mcmd.ExecuteReader()
Dim myOuterList as New List(of String())
While (reader.Read())
Dim myInnerList as New List(of String)
'For loop retrieves all columns of data as string
For i = 0 to reader.FieldCount - 1
myInnerList.Add(reader.GetString(i))
Next
myOuterList.Add(myInnerList.ToArray)
End While
End Using
dim myarray = myOuterList.ToArray
Resulting myarray will be of type String()() I.E. a two-dimensional array of strings.
Though this is possible, I would seriously consider using a typed solution as it will be much easier to understand code dealing with a list of a type than a two-dimensional array.
The typed version would be something like this:
Dim myList As List(Of myType) = reader.OfType(Of IDataRecord) _
.Select(Function(data) New myType _
With {.mydataarray = data.item("mydataarray")})
'Additionaly properties if needed.

VB.NET: How to find a word after a specific word in a line and store it

I have a continuous string of words coming from a machine to hyper terminal of my system, for which I am using USB to serial cable. I want to find some of the values which comes after a specific word in that string and then store it.
I used threads and splitting concepts to do it but as per the requirement and operation of the machine it will not work properly in the runtime.
The values which I want to capture comes from a specific word. I want to skip that words and just store the values. How to do it?
I have given the example of that string below:
MEAN 49 50
SD 500 10
MIN 100 5
MAX 50 45.56
In this I just want to store the values e.g. 49 and 50, then discard MEAN. Then discard SD and store 500 and 10 and so on.
You can use a StreamReader object to read the stream one line at a time. Then, you can easily parse the line using the String.Split method. I would recommend creating one or more classes that represent the data being read, like this:
Public Class LineData
Public Property Label As String
Public Property Value1 As Decimal
Public Property Value2 As Decimal
End Class
Public Function ReadNextLine(stream As Stream) As LineData
Dim reader As New StreamReader(stream)
Dim line As String = reader.ReadLine()
Dim data As LineData = Nothing
If line IsNot Nothing Then
Dim words() As String = line.Split(New Char() {" "c}, StringSplitOptions.RemoveEmptyEntries)
If words.Length = 3 Then
data = New LineData()
data.Label = words(0)
data.Value1 = Decimal.Parse(words(1))
data.Value2 = Decimal.Parse(words(2))
End If
End If
Return Data
End Function
Note, this is a very simple example based on the example data you provided. If different lines have different numbers of numeric parameters, that will further complicate the logic. In my example, the method returns Nothing if no data can be read. Also, the method will throw an exception if the last two words in the line are not numeric. Therefore, you would need to wrap it in some additional exception handling.
This may be what you're looking for. Although personally I'd create a class which stores the type(mean, median etc), firstvalue and secondvalue.
By the sounds of it though, all you want it to do is dump the values into some sort of storage, therefore this will suffice.
Dim Values as New List(Of Decimal)
'Use a streamreader to read each line of text
Using reader As StreamReader = New StreamReader(*your text source*)
'Read the line
Dim linetext as string = reader.ReadLine
Dim myValue as decimal
'Split the line
Dim splitText() = linetext.Split(" ")
'Analyze each section of the line, if its possible to parse the value as a decimal then add it to the list of values to be stored.
For Each txt in splitText
If Decimal.TryParse(txt, myValue) then Values.Add(myValue)
Next
End Using

VB.NET - Load a List of Values from a Text File

I Have a text file that is like the following:
[group1]
value1
value2
value3
[group2]
value1
value2
[group3]
value3
value 4
etc
What I want to be able to do, is load the values into an array (or list?) based on a passed in group value. eg. If i pass in "group2", then it would return a list of "value1" and "value2".
Also these values don't change that often (maybe every 6 months or so), so is there a better way to store them instead of a plain old text file so that it makes it faster to load etc?
Thanks for your help.
Leddo
This is a home work question?
Use the StreamReader class to read the file (you will need to probably use .EndOfStream and ReadLine()) and use the String class for the string manipulation (probably .StartsWith(), .Substring() and .Split().
As for the better way to store them "IT DEPENDS". How many groups will you have, how many values will there be, how often is the data accessed, etc. It's possible that the original wording of the question will give us a better clue about what they were after hear.
Addition:
So, assuming this program/service is up and running all day, and that the file isn't very large, then you probably want to read the file just once into a Dictionary(of String, List(of String)). The ContainsKey method of this will determine if a group exists.
Function GetValueSet(ByVal filename As String) As Dictionary(Of String, List(Of String))
Dim valueSet = New Dictionary(Of String, List(Of String))()
Dim lines = System.IO.File.ReadAllLines(filename)
Dim header As String
Dim values As List(Of String) = Nothing
For Each line As String In lines
If line.StartsWith("[") Then
If Not values Is Nothing Then
valueSet.add(header, values)
End If
header = GetHeader(line)
values = New List(Of String)()
ElseIf Not values Is Nothing Then
Dim value As String = line.Trim()
If value <> "" Then
values.Add(value)
End If
End If
Next
If Not values Is Nothing Then
valueSet.add(header, values)
End If
Return valueSet
End Function
Function GetHeader(ByVal line As String)
Dim index As Integer = line.IndexOf("]")
Return line.Substring(1, index - 1)
End Function
Addition:
Now if your running a multi-threaded solution (that includes all ASP.Net solutions) then you either want to make sure you do this at the application start up (for ASP.Net that's in Global.asax, I think it's ApplicationStart or OnStart or something), or you will need locking. WinForms and Services are by default not multi-threaded.
Also, if the file changes you need to restart the app/service/web-site or you will need to add a file watcher to reload the data (and then multi-threading will need locking because this is not longer confined to application startup).
ok, here is what I edned up coding:
Public Function FillFromFile(ByVal vFileName As String, ByVal vGroupName As String) As List(Of String)
' open the file
' read the entire file into memory
' find the starting group name
Dim blnFoundHeading As Boolean = False
Dim lstValues As New List(Of String)
Dim lines() As String = IO.File.ReadAllLines(vFileName)
For Each line As String In lines
If line.ToLower.Contains("[" & vGroupName.ToLower & "]") Then
' found the heading, now start loading the lines into the list until the next heading
blnFoundHeading = True
ElseIf line.Contains("[") Then
If blnFoundHeading Then
' we are at the end so exit the loop
Exit For
Else
' its another group so keep going
End If
Else
If blnFoundHeading And line.Trim.Length > 0 Then
lstValues.Add(line.Trim)
End If
End If
Next
Return lstValues
End Function
Regarding a possible better way to store the data: you might find XML useful. It is ridiculously easy to read XML data into a DataTable object.
Example:
Dim dtTest As New System.Data.DataTable
dtTest.ReadXml("YourFilePathNameGoesHere.xml")