FileHelpers Library - Append Multiple Records on Transform - vb.net
Using the FileHelpers library to do some great things in VB.NET. Parsing files with dynamic classes built from text file templates. One thing I can't find: A way to read in a single record and determine that it should result in the generation of two records.
Current Code:
Dim FromType As Type = Dynamic.ClassBuilder.ClassFromSourceFile(MyFilePath, MyDynamicTypeName, NetLanguage.VbNet)
Dim FromRecords() As Object
FromRecords = FileHelpers.CommonEngine.ReadString(FromType, MyStringBuilder.ToString)
'... maybe code here to check for certain values
Dim engine As New FileTransformEngine(Of ITransformable(Of MyDestinationClass), MyDestinationClass)
' Ideally in this next line I would like it to see certain conditions and be able to generate two records from a single source line.
Dim PayRecords() As Object = engine.TransformRecords(FromRecords)
Alternately, if there is a way to implement the "ITransformable(Of ..." TransformTo() and have it return multiple records, I could put the logic in the dynamic class definition TransformTo() method.
Thoughts?
Here is a sample of my source dynamic class:
Imports FileHelpers ' Never forget
_
Public NotInheritable Class MyDynamicClass
Implements ITransformable(Of MyDestinationClass)
_
Public Name As String
<FieldQuoted(""""c, QuoteMode.OptionalForRead, MultilineMode.AllowForRead)> _
Public KeyType As String
Public Hours As Double
Public Function TransformTo() As MyDestinationClass Implements ITransformable(Of MyDestinationClass).TransformTo
Dim res As New MyDestinationClass
res.ContactName = Name
' Here is where I would like to say... instead of Return res
If KeyType="ABCD" Then
Dim newRes as New MyDestinationClass
newRes.Contactname = Name + " 2nd contact"
Dim resArray() as MyDestinationClass
redim resArray(1)
resArray(0) = res
resArray(1) = newRes
End If
Return resArray
' Or alternately refer to the engine, but it is not in scope for the dynamic record (is it?). Something like...
engine.AppendToDestination(new MyDestinationClass(...))
End Function
End Class
I think the main problem you're going to run into is that ITransformable.TransformTo() is spec'ed to return a single T value.
This is called in a loop when you call engine.TransformRecords(), where one output record is added for each input record.
I don't think this would be a big change if you didn't mind doing your own version of FileTransformEngine, but I don't see a clean way to do this as-is.
Related
Convert an unknown structure to an untyped Object in VB.NET
I'd like to convert an unknown basic structure to an Object (no type here).
I'm building a library that will be used by many users to extract data from my system but don't want to do a new function for everyone of them. They have to know what will be the result.
In vb, it is possible to create an Object with some properties and use it as it is a regular Class like so:
Dim myObj as New With { .name = "Matt", .age = "28" }
MsgBox( myObj.name & " is now " & myObj.age & " years old.")
So far, so good.
Next step : my user will give me some instructions that I need to extract data from various DBs, and I've no idea of what the result will be.
What I know after the execution is a list of String containing the columns of the result set and, of course a (set of) rows.
And here is the problem of course
My function (for a single row) so far:
Public Function GetData(ByVal instructions as String) as Object ' User is supposed to know what will be inside, instructions is as XML describing DB, table, query, ...
' Do what is needed to retrieve data
' Here I have a variable cols As List(Of String) ' e.g. ("BP", "NAME", "VAT")
Dim o As New With ???
Return o
End Function
What I've tried: build a fake JSon on the fly, and try to Deserialize to Object.
But even if it seems to work, I (and the user) can't access the property as in my top piece of code like:
MsgBox(o.BP)
I know that I could do
Public Function GetData(Of T As {New})(ByVal instructions as String) As T
Dim o As T
' Use some Reflexion to TryInvokeMember of T
Return o
End Function
But I wanted to remove the hassle to create a class to use my code.
Plus, My librairy will be use in a webservice and the class of the user is then unknown.
One approach could be - to use Dictionary(Of String, Object)
Public Function GetData(instructions as String) As Dictionary(Of String, Object)
Dim data = ' Load data
Dim columns As String() = { "BP", "NAME", "VAT" }
Return columns.ToDictionary(
Function(column) column,
Function(column) data.GetByColumnName(column)
)
End Function
` Usage
Dim result = GetDate("instructions: ['BP', 'NAME']")
' Because user knows it is Integer
Dim bpValue = DirectCast(result.Item("BP"), Integer)
Thanks to #GSerg, #Fabio and a few other searches about ExpandoObject, I did it ! Imports System.Dynamic Dim o As Object = New ExpandoObject() For Each col In cols DirectCast(o, IDictionary(Of String, Object)).Add(col, row.GetString(col)) Next
How to order list of files by file name with number?
I have a bunch of files in a directory that I am trying to get based off their type. Once I have them I would like to order them by file name (there is a number in them and I would like to order them that way) My files returned are: file-1.txt file-2.txt ... file-10.txt file-11.txt ... file-20.txt But the order I get them in looks something more closely to this: file-1.txt file-10.txt file-11.txt ... file-2.txt file-20.txt Right now I am using Directory.GetFiles() and attempting to using the linq OrderBy property. However, I am failing pretty badly with what I would need to do to order my list of files like the first list above. Directory.GetFiles() seems to be returning a list of strings so I am unable to get the list of file properties such as filename or name. Here is my code currently: documentPages = Directory.GetFiles(documentPath, "*.txt").OrderBy(Function(p) p).ToList() Would anyone have any ideas?
It sounds like you might be looking for a "NaturalSort" - the kind of display File Explorer uses to order filenames containing numerals. For this you need a custom comparer:
Imports System.Runtime.InteropServices
Partial Class NativeMethods
<DllImport("shlwapi.dll", CharSet:=CharSet.Unicode)>
Private Shared Function StrCmpLogicalW(s1 As String, s2 As String) As Int32
End Function
Friend Shared Function NaturalStringCompare(str1 As String, str2 As String) As Int32
Return StrCmpLogicalW(str1, str2)
End Function
End Class
Public Class NaturalStringComparer
Implements IComparer(Of String)
Public Function Compare(x As String, y As String) As Integer Implements IComparer(Of String).Compare
Return NativeMethods.NaturalStringCompare(x, y)
End Function
End Class
Use it to sort the results you get:
Dim myComparer As New NaturalStringComparer
' OP post only shows the filename without path, so strip off path:
' (wont affect the result, just the display)
Dim files = Directory.EnumerateFiles(path_name_here).
Select(Function(s) Path.GetFileName(s)).ToList
Console.WriteLine("Before: {0}", String.Join(", ", files))
' sort the list using the Natural Comparer:
files.Sort(myComparer)
Console.WriteLine("After: {0}", String.Join(", ", files))
Results (one-lined to save space):
Before: file-1.txt, file-10.txt, file-11.txt, file-19.txt, file-2.txt, file-20.txt, file-3.txt, file-9.txt, file-99.txt
After: file-1.txt, file-2.txt, file-3.txt, file-9.txt, file-10.txt, file-11.txt, file-19.txt, file-20.txt, file-99.txt
One of the advantages of this is that it doesnt rely on a specific pattern or coding. It is more all-purpose and will handle more than one set of numbers in the text:
Game of Thrones\4 - A Feast For Crows\1 - Prologue.mp3
Game of Thrones\4 - A Feast For Crows\2 - The Prophet.mp3
...
Game of Thrones\4 - A Feast For Crows\10 - Brienne II.mp3
Game of Thrones\4 - A Feast For Crows\11 - Sansa.mp3
A Natural String Sort is so handy, is is something I personally dont mind polluting Intellisense with by creating an extension:
' List<string> version
<Extension>
Function ToNaturalSort(l As List(Of String)) As List(Of String)
l.Sort(New NaturalStringComparer())
Return l
End Function
' array version
<Extension>
Function ToNaturalSort(a As String()) As String()
Array.Sort(a, New NaturalStringComparer())
Return a
End Function
Usage now is even easier:
Dim files = Directory.EnumerateFiles(your_path).
Select(Function(s) Path.GetFileName(s)).
ToList.
ToNaturalSort()
' or without the path stripping:
Dim files = Directory.EnumerateFiles(your_path).ToList.ToNaturalSort()
I'm assuming the file and .txt parts are mutable, and just here as placeholders for file names and types that can vary. I don't use regular expressions very often, so this may need some work yet, but it's definitely the direction you need to go: Dim exp As String = "-([0-9]+)[.][^.]*$" documentPages = Directory.GetFiles(documentPath, "*.txt").OrderBy(Function(p) Integer.Parse(Regex.Matches(p, exp)(0).Groups(1).Value)).ToList() Looking again, I see I missed that you are filtering by *.txt files, which can help us narrow the expression: Dim exp As String = "-([0-9]+)[.]txt$" Another possible improvement brought by the other answer that includes test data is to allow for whitespace between the - and numerals: Dim exp As String = "-[ ]*([0-9]+)[.]txt$" It's further worth noting that the above will fail if there are text files that don't follow the pattern. We can account for that if needed: Dim exp As String = "-[ ]*([0-9]+)[.][^.]*$" Dim docs = Directory.GetFiles(documentPath, "*.txt") documentPages = docs.OrderBy( Function(p) Dim matches As MatchCollection = Regex.Matches(p, exp) If matches.Count = 0 OrElse matches(0).Groups.Count < 2 Then Return 0 Return Integer.Parse(matches(0).Groups(1).Value) End Function).ToList() You could also use Integer.MaxValue as your default option, depending on whether you want those to appear at the beginning or end of the list.
How to call a function which name is stored in a sql table (VB.net)
Maybe someone of you can help me with that problem. I have written a background task which gets several workers out of a database. In the database I added to each worker the name of the function which should get called. But I am not sure how to call that function from vb.net. It would be awesome if someone of you can give me a hint :) thanks Cheers Arthur
The namespace System.Reflection has numerous methods that enable this functionality, such as this one:
From MSDN, the example in the link above:
Imports System
Imports System.Reflection
Public Class MagicClass
Private magicBaseValue As Integer
Public Sub New()
magicBaseValue = 9
End Sub
Public Function ItsMagic(preMagic As Integer) As Integer
Return preMagic * magicBaseValue
End Function
End Class
Public Class TestMethodInfo
Public Shared Sub Main()
' Get the constructor and create an instance of MagicClass
Dim magicType As Type = Type.GetType("MagicClass")
Dim magicConstructor As ConstructorInfo = magicType.GetConstructor(Type.EmptyTypes)
Dim magicClassObject As Object = magicConstructor.Invoke(New Object(){})
' Get the ItsMagic method and invoke with a parameter value of 100
Dim magicMethod As MethodInfo = magicType.GetMethod("ItsMagic")
Dim magicValue As Object = magicMethod.Invoke(magicClassObject, New Object(){100})
Console.WriteLine("MethodInfo.Invoke() Example" + vbNewLine)
Console.WriteLine("MagicClass.ItsMagic() returned: {0}", magicValue)
End Sub
End Class
' The example program gives the following output:
'
' MethodInfo.Invoke() Example
'
' MagicClass.ItsMagic() returned: 900
I would suggest having a dictionary of delegates in your application code. Store the keys in the database, rather than the function names. When you retrieve the key from the database, look it up in the dictionary and, if present, execute it. You don't want to allow arbitrary methods to be executed based on names stored in a database.
They keys could be strings or integers. I'd prefer the latter just for space savings and ease of lookup, but strings would be easier for debugging, perhaps. So you'd have a dictionary like this:
Private m_WorkerDelegates As New Dictionary(Of String, Action)()
Somewhere else, you'd fill it up with the available workers:
m_WorkerDelegates.Add("worker1", AddressOf WorkerMethod1)
m_WorkerDelegates.Add("worker2", AddressOf WorkerMethod2)
And then, when retrieving from the database, you'd look up the method in your dictionary:
Public Sub ExecuteWorker(ByVal row As DataRow)
Dim key As String = CStr(row("worker_key"))
If Not m_WorkerDelegates.ContainsKey(key) Then
' either throw exception or report the error in some more effective way '
Throw New Exception("Invalid worker key specified")
End If
' actually call the worker method '
m_WorkerDelegates(key)()
End Sub
parse several field using file helper
Good day to all, i have a problem regarding CSV parse using File Helper. My CSV is look like this ,,,026642,0,00336,05,19,"WATERMELON *",19,"1 ",,,,,,,,0,,001.99.,0,,,,,0,,0,0,,,,,,,,,,,,,,,,,,51,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, ,,,026645,0,00338,05,19,"ONION ",19,"*1 ",,,,,,,,0,,002.99.,0,,,,,0,,0,0,,,,,,,,,,,,,,,,,,51,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, ,,,026687,0,00380,05,19,"MUSHROOM ",19," (BLACK FUNGUS) ",,,,,,,,0,,021.90.,0,,,,,0,,0,0,,,,,,,,,,,,,,,,,,51,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, which this CSV have 116 column / field. The problem was i just need only 4 field from whole line which Field 4 = 026687, Field 9 = "WATERMELON *", Field 11 = "(BLACK FUNGUS)", Field 21 = 002.99. When i using wizard which a very good solution to make instance class, and i put FieldValueDiscarded _ on top of every other fields i don't need it, it just print a same as CSV input. Please give me some advise how do i extract only needed field to write into output. Thank you UPDATED : After research some more, i finally getting clear about this error. This error happen because class non-inherited so i cannot get specific field into another class. By declare mapping class as inherited class, i can get specific field. However, i stuck at how to retrieve functions inside the class who inherited from mapping class. This is my code `Imports System.Text Imports System.IO Imports FileHelpers Public Class ProcessField : Inherits InputCSV Public Function MyPLUc(ByVal value As Integer) Dim PLUc As String MyBase.PLU = value PLUc = (0 + 0 + value) Return (0 + 0 + value) End Function Public Function MyStatusc(ByVal value As Integer) MyBase.Status = value Dim Status As Integer If value > 0 Then Status = 800 Else Status = 900 End If Return (0 + Status) End Function Public Function MyUnitPricec(ByVal value As Integer) MyBase.UnitPrice = value Dim UP As Integer UP = value Dim bytes As Byte() = System.Text.Encoding.Unicode.GetBytes(UP) Return (0 + 0 + 0 + UP) End Function Public Function MyLabelFormat(ByVal value As Integer) Return (1 + 1) End Function Public Function MyEAN(ByVal value As Integer) Return (0 + 6) End Function Public Function MyCName(ByVal value As String) MyBase.CName1 = value Dim hexString As String = Hex(value) Return (hexString) End Function Public Function MyBCC(ByVal value As String) value = (0 + 0) Return (0 + 0) End Function End Class` and the problem is how i retrieve all return value in these function
I'm not sure I understand your question, but I think you are making things more complicated than you need.
You can just create your FileHelpers class with 116 string fields. They should be (public) fields, NOT properties. You should NOT subclass the FileHelpers class. Do not try to use the FileHelpers class as a normal class (i.e., with properties, functions, etc, subclasses). Think of the FileHelpers class as a 'specification' of your CSV format only.
To import, you write something like:
Dim engine As New FileHelperEngine(GetType(RecordSpec))
' To Read Use:
Dim results As RecordSpec() = DirectCast(engine.ReadFile("FileIn.txt"), RecordSpec())
Then you have an array of RecordSpec. Each RecordSpec will have all 116 fields populated, but just ignore the fields you don't need. Then loop through the results and do whatever you want with the values. For instance, perhaps you need to map the imported fields to another (more normal) MyProduct class with properties instead of fields and perhaps with additional logic.
For Each recordSpec As RecordSpec In results
Dim myProduct = New MyProduct()
myProduct.Id = recordSpec.Field4
myProduct.Name = recordSpec.Field9
myProduct.Category = recordSpec.Field23
' etc.
Next
In summary: the RecordSpec class should only be used to define the structure of the CSV file. You use it to populate a simple array with all the values from the file. You then map the values to a more useful class MyProduct.
How to write a simple Expression-like class in .NET 2.0?
I'm currently working in .NET 2.0 Visual Basic. The current project is an Active Directory Wrapper class library within which I have a Searcher(Of T) generic class that I wish to use to search the underlying directory for objects.
In this Searcher(Of T) class I have the following methods:
Private Function GetResults() As CustomSet(Of T)
Public Function ToList() As CustomSet(Of T)
Public Function Find(ByVal ParamArray filter() As Object) As CustomSet(Of T)
// And some other functions here...
The one that interests me the most is the Find() method to which I can pass property and values and would like to parse my LDAP query from this filter() ParamArray parameter. Actually, all I can figure out is this:
Public Sub SomeSub()
Dim groupSearcher As Searcher(Of Group) = New Searcher(Of Group)()
Dim groupsSet as CustomSet(Of Group) = groupSearcher.Find("Name=someName", "Description=someDescription")
// Working with the result here...
End Sub
But what I want to be able to offer to my users is this:
Public Sub SomeSub()
Dim groupSearcher As Searcher(Of Group) = New Searcher(Of Group)()
Dim groupsSet As CustomSet(Of Groupe) = groupSearcher.Find(Name = "someName", Guid = someGuid, Description = "someDescription")
// And work with the result here...
End Sub
In short, I want to offer some kind of Expression feature to my users, unless it is too much work, as this project is not the most important one and I don't have like 2 years to develop it. I think that the better thing I should do is to write something like CustomExpression that could be passed in parameters to some functions or subs.
Thanks for any suggestions that might bring me to my goal!
Interesting question. This is a language dependent feature, so I don't see this happening without some clever trickery of the IDE/compiler.
You could however have optional overloads on your Find method (vb.net is good for this), then make the search string manually to obtain the result.
Finally you could make use of lambda functions, but only in .net 3.5 and above. Even still, it would require your searcher to expose a preliminary set of data so you can recover the expression tree and build up the find string.
UPDATE
I've just been playing around with Reflection to see if I can retrieve the parameters passed, and build up a string dynamically depending on if they exist. This doesn't appear to be possible, due to the fact that compiled code doesn't reference the names.
This code just used was:
'-- Get all the "parameters"
Dim m As MethodInfo = GetType(Finder).GetMethod("Find")
Dim params() As ParameterInfo = m.GetParameters()
'-- We now have a reference to the parameter names, like Name and Description
Hmm. http://channel9.msdn.com/forums/TechOff/259443-Using-SystemReflection-to-obtain-parameter-values-dynamically/
Annoyingly it's not (easily) possible to recover the values sent, so we'll have to stick with building up the string in a non-dynamic fashion.
A simple optional method would look like:
Public Sub Find( _
Optional ByVal Name As String = "", _
Optional ByVal Description As String = "")
Dim query As String = String.Empty
If Not String.IsNullOrEmpty(Name) Then
query &= "Name=" & Name
'-- ..... more go here with your string seperater.
End If
End Sub