What would be the best way to parse this file? - vb.net

Question

I was just wondering if anyone knew of a good way that I could parse the file at the bottom of the post.
I have a database setup with the correct tables for each section eg Refferal Table,Caller Table,Location Table. Each table has the same columns that are show in the file below
I would really like something that is fairly genetic so if the file layout changes it won't mess me around to much. At the moment I am just reading the file in a line at a time and just using a case statement to check which section i'm in.
Is anyone able to help me with this?
PS. I am using VB but C# or anything else will be fine, also the x's in the document are just personal info I have blanked
Thanks,
Nathan
File:--->
DIAL BEFORE YOU DIG
Call 1100, Fax [phone]
PO Box 7710 MELBOURNE, VIC 8004
Utilities are requested to respond within 2 working days and reference the Sequence number.
[REFFERAL DETAILS]
FROM= Dial Before You Dig - Web
TO= Technical Services
UTILITY ID= xxxxxx
COMPANY= {Company Name}
ENQUIRY DATE= 02/10/2008 13:53
COMMENCEMENT DATE= 06/10/2008
SEQUENCE NO= xxxxxxxxx
PLANNING= No
[CALLER DETAILS]
CUSTOMER ID= 403552
CONTACT NAME= {Name of Contact}
CONTACT HOURS= 0
COMPANY= Underground Utility Locating
ADDRESS= {Address}
SUBURB= {Suburb}
STATE= {State}
POSTCODE= 4350
TELEPHONE= xxxxxxxxxx
MOBILE= xxxxxxxxxx
FAX TYPE= Private
FAX NUMBER= xxxxxxxxxx
PUBLIC ADDRESS= xxxxxxxxxx
PUBLIC TELEPHONE=
EMAIL ADDRESS= {Email Address}
[LOCATION DETAILS]
ADDRESS= {Location Address}
SUBURB= {Location Suburb}
STATE= xxx
POSTCODE= xxx
DEPOSITED PLAN NO= 0
SECTION & HUNDRED NO= 0
PROPERTY PHONE NO=
SIDE OF STREET= B
INTERSECTION= xxxxxx
DISTANCE= 0-200m B
ACTIVITY CODE= 15
ACTIVITY DESCRIPTION= xxxxxxxxxxxxxxxxxx
MAP TYPE= StateGrid
MAP REF= Q851_63
MAP PAGE=
MAP GRID 1=
MAP GRID 2=
MAP GRID 3=
MAP GRID 4=
MAP GRID 5=
GPS X COORD=
GPS Y COORD=
PRIVATE/ROAD/BOTH= B
TRAFFIC AFFECTED= No
NOTIFICATION NO= 3082321
MESSAGE= entire intersection of Allora-Clifton rd , Hillside
rd and merivale st
MOCSMESSAGE= Digsafe generated referral
Notice: Please DO NOT REPLY TO THIS EMAIL as it has been automatically generated and replies are not monitored. Should you wish to advise Dial Before You Dig of any issues with this enquiry, please Call 1100
(See attached file: 3082321_LLGDA94.GML)

Answer 1

Google has the answers, once you know that the file-format is called '.ini'
Edit: That is, it's an .ini plus some extra leading/trailing gunk.

Answer 2

You could read each line of the file sequentially. Each line is essentially a name value pair. Place each value in a map (hash table) keyed by name. Use a map for each section. When done parsing the file you'll have maps containing all the name value pairs. Iterate over each map and populate your database tables.

Answer 3

I would head to Python for any type of string parsing like this. I'm not sure how much of this information you want to retain, but I would perhaps use Python's split() function to split on = to get rid of the equals sign, then strip the whitespace out of the second piece of the pie.
First, I would mask out the header/footer info I know I don't need, then do something akin to the following:
Let's take a chunk and save it in test1.txt:
ADDRESS= {Location Address}
SUBURB= {Location Suburb}
STATE= xxx
POSTCODE= xxx
DEPOSITED PLAN NO= 0
SECTION & HUNDRED NO= 0
PROPERTY PHONE NO=
Here's a small python snippet:
>>> f = open("test1.txt", "r")
>>> l = f.readlines()
>>> l = [line.split('=') for line in l]
>>> for line in l:
print line
['ADDRESS', '{Location Address}']
['SUBURB', '{Location Suburb}']
['STATE', 'xxx']
['POSTCODE', 'xxx']
['DEPOSITED PLAN NO', '0']
['SECTION & HUNDRED NO', '0']
['PROPERTY PHONE NO', '']
This would essentially give you a [Column, Value] tuple you could use to insert the data into your database (after escaping all strings, etc etc, SQL Injection warning).
This is assuming the email input and your DB will have the same column names, but if they didn't, it'd be fairly trivial to set up a column mapping using a dictionary. On the flip side, if the email and columns are in sync, you don't need to know the names of the columns to get the parsing down.
You could iterate through the pseudo-dictionary and print out each key-value pair in the right spot in your parameterized sql string.
Hope this helps!
Edit: While this is in Python, C#/VB.net should have the same/similar abilities.

Answer 4

Using f As StreamReader = File.OpenText("sample.txt")
Dim g As String = "undefined"
Do
Dim s As String = f.ReadLine
If s Is Nothing Then Exit Do
s = s.Replace(Chr(9), " ")
If s.StartsWith("[") And s.EndsWith("]") Then
g = s.Substring("[".Length, s.Length - "[]".Length)
Else
Dim ss() As String = s.Split(New Char() {"="c}, 2)
If ss.Length = 2 Then
Console.WriteLine("{0}.{1}={2}", g, Trim(ss(0)), Trim(ss(1)))
End If
End If
Loop
End Using