parse several field using file helper - vb.net

Good day to all,
i have a problem regarding CSV parse using File Helper. My CSV is look like this
,,,026642,0,00336,05,19,"WATERMELON *",19,"1 ",,,,,,,,0,,001.99.,0,,,,,0,,0,0,,,,,,,,,,,,,,,,,,51,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,,,026645,0,00338,05,19,"ONION ",19,"*1 ",,,,,,,,0,,002.99.,0,,,,,0,,0,0,,,,,,,,,,,,,,,,,,51,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,,,026687,0,00380,05,19,"MUSHROOM ",19," (BLACK FUNGUS) ",,,,,,,,0,,021.90.,0,,,,,0,,0,0,,,,,,,,,,,,,,,,,,51,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
which this CSV have 116 column / field.
The problem was i just need only 4 field from whole line which
Field 4 = 026687, Field 9 = "WATERMELON *", Field 11 = "(BLACK FUNGUS)", Field 21 = 002.99.
When i using wizard which a very good solution to make instance class, and i put FieldValueDiscarded _ on top of every other fields i don't need it, it just print a same as CSV input.
Please give me some advise how do i extract only needed field to write into output.
Thank you
UPDATED : After research some more, i finally getting clear about this error. This error happen because class non-inherited so i cannot get specific field into another class. By declare mapping class as inherited class, i can get specific field.
However, i stuck at how to retrieve functions inside the class who inherited from mapping class. This is my code
`Imports System.Text
Imports System.IO
Imports FileHelpers
Public Class ProcessField : Inherits InputCSV
Public Function MyPLUc(ByVal value As Integer)
Dim PLUc As String
MyBase.PLU = value
PLUc = (0 + 0 + value)
Return (0 + 0 + value)
End Function
Public Function MyStatusc(ByVal value As Integer)
MyBase.Status = value
Dim Status As Integer
If value > 0 Then
Status = 800
Else
Status = 900
End If
Return (0 + Status)
End Function
Public Function MyUnitPricec(ByVal value As Integer)
MyBase.UnitPrice = value
Dim UP As Integer
UP = value
Dim bytes As Byte() = System.Text.Encoding.Unicode.GetBytes(UP)
Return (0 + 0 + 0 + UP)
End Function
Public Function MyLabelFormat(ByVal value As Integer)
Return (1 + 1)
End Function
Public Function MyEAN(ByVal value As Integer)
Return (0 + 6)
End Function
Public Function MyCName(ByVal value As String)
MyBase.CName1 = value
Dim hexString As String = Hex(value)
Return (hexString)
End Function
Public Function MyBCC(ByVal value As String)
value = (0 + 0)
Return (0 + 0)
End Function
End Class`
and the problem is how i retrieve all return value in these function

I'm not sure I understand your question, but I think you are making things more complicated than you need.
You can just create your FileHelpers class with 116 string fields. They should be (public) fields, NOT properties. You should NOT subclass the FileHelpers class. Do not try to use the FileHelpers class as a normal class (i.e., with properties, functions, etc, subclasses). Think of the FileHelpers class as a 'specification' of your CSV format only.
To import, you write something like:
Dim engine As New FileHelperEngine(GetType(RecordSpec))
' To Read Use:
Dim results As RecordSpec() = DirectCast(engine.ReadFile("FileIn.txt"), RecordSpec())
Then you have an array of RecordSpec. Each RecordSpec will have all 116 fields populated, but just ignore the fields you don't need. Then loop through the results and do whatever you want with the values. For instance, perhaps you need to map the imported fields to another (more normal) MyProduct class with properties instead of fields and perhaps with additional logic.
For Each recordSpec As RecordSpec In results
Dim myProduct = New MyProduct()
myProduct.Id = recordSpec.Field4
myProduct.Name = recordSpec.Field9
myProduct.Category = recordSpec.Field23
' etc.
Next
In summary: the RecordSpec class should only be used to define the structure of the CSV file. You use it to populate a simple array with all the values from the file. You then map the values to a more useful class MyProduct.

Related

Convert an unknown structure to an untyped Object in VB.NET

I'd like to convert an unknown basic structure to an Object (no type here).
I'm building a library that will be used by many users to extract data from my system but don't want to do a new function for everyone of them. They have to know what will be the result.
In vb, it is possible to create an Object with some properties and use it as it is a regular Class like so:
Dim myObj as New With { .name = "Matt", .age = "28" }
MsgBox( myObj.name & " is now " & myObj.age & " years old.")
So far, so good.
Next step : my user will give me some instructions that I need to extract data from various DBs, and I've no idea of what the result will be.
What I know after the execution is a list of String containing the columns of the result set and, of course a (set of) rows.
And here is the problem of course
My function (for a single row) so far:
Public Function GetData(ByVal instructions as String) as Object ' User is supposed to know what will be inside, instructions is as XML describing DB, table, query, ...
' Do what is needed to retrieve data
' Here I have a variable cols As List(Of String) ' e.g. ("BP", "NAME", "VAT")
Dim o As New With ???
Return o
End Function
What I've tried: build a fake JSon on the fly, and try to Deserialize to Object.
But even if it seems to work, I (and the user) can't access the property as in my top piece of code like:
MsgBox(o.BP)
I know that I could do
Public Function GetData(Of T As {New})(ByVal instructions as String) As T
Dim o As T
' Use some Reflexion to TryInvokeMember of T
Return o
End Function
But I wanted to remove the hassle to create a class to use my code.
Plus, My librairy will be use in a webservice and the class of the user is then unknown.
One approach could be - to use Dictionary(Of String, Object)
Public Function GetData(instructions as String) As Dictionary(Of String, Object)
Dim data = ' Load data
Dim columns As String() = { "BP", "NAME", "VAT" }
Return columns.ToDictionary(
Function(column) column,
Function(column) data.GetByColumnName(column)
)
End Function
` Usage
Dim result = GetDate("instructions: ['BP', 'NAME']")
' Because user knows it is Integer
Dim bpValue = DirectCast(result.Item("BP"), Integer)
Thanks to #GSerg, #Fabio and a few other searches about ExpandoObject, I did it !
Imports System.Dynamic
Dim o As Object = New ExpandoObject()
For Each col In cols
DirectCast(o, IDictionary(Of String, Object)).Add(col, row.GetString(col))
Next

How to set 2 arguments on Let function in a class in vba (for excel)?

I am creating a Class in vba (for excel) to process blocks of data. After some manipulation of a text file I end up with blocks of data (variable asdatablock() ) which I want to process in a For Loop
I created my own Class called ClDataBlock from which I can get key data by a simple call of the property required. 1st pass seems to work and I am now trying to expand my Let function to 2 argument but it’s not working. How do I specify the 2nd argument?
Dim TheDataBlock As New ClDataBlock
For i = 0 to UBound(asdatablock)
asDataBlockLine = Split(asdatablock(i), vbLf) ‘ split block into line
TheDataBlock.LineToProcess = asDataBlockLine(5) ‘allocate line to process by the class
Dvariable1 = TheDataBlock.TheVariable1
‘and so on for the key variables needed base don the class properties defined
Next i
In the Class Module the Let function takes 2 arguments
Public Property Let LineToProcess(stheline As String, sdataneeded As String)
code extract of what I am looking at -
'in the class module
Dim pdMass As Double
Private pthelineprocessed As String
Public Property Let LineToProcess(stheline As String, sdataneeded As String)
pthelineprocessed = DeleteSpaces(Replace(stheline, vbLf, ""))
Dim aslinedatafield() As String
Select Case sdataneeded
'THIS IS AN EXTRACT FROM THE FUNCTION
'THERE ARE AS NUMBER OF CASES WHICH ARE DEALT WITH
Case Is = "MA"
aslinedatafield() = Split(pthelineprocessed, " ")
pdbMass = CDbl(aslinedatafield(2))
End select
End function
Public Property Get TheMass() As Double
TheMass = pdMass
End Property
'in the "main" module
Dim TheDataBlock As New ClDataBlock
For i = 0 to UBound(asdatablock)
TheDataBlock.LineToProcess = asDataBlockLines(5) 'Need to pass argument "MA" as well
dmass = TheDataBlock.TheMass
'and so on for all the data to be extracted
Next i
When a Property has 2 or more arguments, the last argument is what is getting assigned. In other words, the syntax is like this:
TheDataBlock.LineToProcess("MA") = asDataBlockLine(5)
This means you need to change the signature of your property:
Public Property Let LineToProcess(sdataneeded As String, stheline As String)

SSRS Group Variables SUM & VB.NET Custom Code

I have an SSRS report built using group variables with expressions that do field calculations. The group varaibles are providing correct results between 3 datsets using lookups, arithmatic, and logic operations.
I need to SUM total the results of the variable textbox expressions in the tablix footer. But variables will not work in this way and when I try different expressions in the tablix footer I get errors.
So I did online search of summing group variables and I came to custom code solution using VB.NET variables with functions to aggregate then display the values. But the custom code is not quite working. Please see the custom code at the bottom of the page. And here are some issues I've observed
Custom Code issues
If I use variable as Public Dim the total values changes to 0 when exported to excel (e.g.- "1,214,284" on the screen; "0" when exported to excel.)
If I change declare the variables as Public Dim to Public Shared Dim then the values on screen are the same and they will export to excel.
The problem is Public Shared Dim seems to work great in Visual Studio. But when executed on the Report Server, the variable accumulates every time the report is executed (i.e., ExecEvent #1: "150 value" on the screen & excel; ExecEvent #2: "300 value‬" on the screen & excel; ExecEvent #3: "450 value‬" on the screen & excel).
Any help? How can I make these values aggregate and export? How to get the custom code VB variables to behave correctly. Particularly the variable initialization on the server getting correctly set and reset.
Custom Code correction notes
In the "add" function I added return thisValue to fix an issue where the details variable values with blank (not printing)
References
SSRS Summing Group Variables outside of the group
SSRS code variable resetting on new page
https://social.msdn.microsoft.com/Forums/sqlserver/en-US/d4a3969a-f3fc-457b-8430-733d7b590ff6/use-reportitems-to-sum-all-textboxes-in-a-tablix?forum=sqlreportingservices&prof=required
TablixDesign-WithGroupVaraibles
Group row, Group footer row
Tablix Footer: group variable expression errors outside of group
NOTE: These expressions are not allowed
Variables!varExpenditureLifetime.Value --ERROR: Expressions can only refer to a Variable declared within the same grouping scope, a containing grouping scope, or those declared on the report. Letters in the names of variabels must use the correct cast.
Sum(Variables!varExpenditureLifetime.Value) --ERROR: Variables cannot be used in aggregate functions
REPORT CODE:
Group Variable ("varLWSBegin_LifetimeExpense")
= Code.addLWSBeginLifetimeExpense(CDbl(
IIF(Parameters!boolUseLwsBalance.Value = false, 0,
Lookup(Fields!ProjectId.Value, Fields!ProjectId.Value, Fields!LWSBegin_LifetimeExpense.Value, "dsCAPEXAmountsCustomDataUpload"))
))
Tablix group row
Variables!varLWSBegin_LifetimeExpense.Value
Tablix footer row
Code.getTotalLWSBeginLifetimeExpense()
VB.NET Custom Code
'
' Add group variable values to an external variable declared on the VB Code.
' In Tablix Group Properties variable the value will be the call to the addValue function
' Then on your textbox you call the getValue function:
' Group Var: Code.addValue(Fields!my_field).
' Textbox: Code.getValue()
'
' variable1, addTotalBudget, getTotalBudget
' variable2, addLWSBeginLifetimeExpense, getLWSBeginLifetimeExpense
' variable3, addExpenditureLifetime, getExpenditureLifetime
'
'TEMPLATE
Public totalMyField2 as Integer
Public totalMyFieldName as Integer
Public Function addMyFieldName(ByVal thisMyFieldName AS integer)
totalMyFieldName = totalMyFieldName + thisMyFieldName
End Function
Public Function getMyFieldName()
return totalMyFieldName
End Function
'MyVariables
Public Shared Dim totalTotalBudget as Integer
Public Shared Dim totalLWSBeginLifetimeExpense as Integer
Public Shared Dim totalExpenditureLifetime as Integer
Public Shared Function Initialize()
totalTotalBudget = 0
totalLWSBeginLifetimeExpense = 0
totalExpenditureLifetime = 0
End Function
'Functions
Public Function addTotalBudget(ByVal thisValue AS Integer )
totalTotalBudget = totalTotalBudget + thisValue
return thisValue
End Function
Public Function addLWSBeginLifetimeExpense(ByVal thisValue AS Integer )
totalLWSBeginLifetimeExpense = totalLWSBeginLifetimeExpense + thisValue
return thisValue
End Function
Public Function addExpenditureLifetime(ByVal thisValue AS Integer )
totalExpenditureLifetime = totalExpenditureLifetime + thisValue
return thisValue
End Function
Public Function getTotalBudget()
return totalTotalBudget
' Dim myval as Integer = totalTotalBudget
' totalTotalBudget = 0
' return myval
End Function
Public Function getTotalLWSBeginLifetimeExpense()
return totalLWSBeginLifetimeExpense
' Dim myval as Integer = totalLWSBeginLifetimeExpense
' totalLWSBeginLifetimeExpense = 0
' return myval
End Function
Public Function getTotalExpenditureLifetime()
return totalExpenditureLifetime
' Dim myval as Integer = totalExpenditureLifetime
' totalExpenditureLifetime = 0
' return myval
End Function
'<END>
Easiest approach to be able to use variables like you are trying to is to use Report Variables instead of Group Variables. If you use Report Variables, it can be accessed from any scope.
You might have to change your dataset a little bit so that the aggregation is unmistakably grouped when calculated at the report level. I could not really guide you with that because I don't know your data like you do. But as an easier route to the scope of your aggregation, check this SO question. You can use your table group as the scope of your aggregate functions.
I've had to do something along the lines of what you are trying to accomplish and Report Variables came to the rescue for me. Let me know if this would work in your scenario.

Saving multiple file types into one potobuf-net'ized file in VB.net

I'm writing a program that saves 'map' files to the HD so that I can open them later and display the same data. My maps originally saved only one data type, a set of my own custom objects with the properties: id, layer, x, and y. You can see the code I did for that, here:
<ProtoContract()> _
Public Class StrippedElement
<ProtoMember(1)> _
Public Property X() As Integer
Get
Return m_X
End Get
Set(ByVal value As Integer)
m_X = value
End Set
End Property
Private m_X As Integer
<ProtoMember(2)> _
Public Property Y() As Integer
Get
Return m_Y
End Get
Set(ByVal value As Integer)
m_Y = value
End Set
End Property
Private m_Y As Integer
<ProtoMember(3)> _
Public Property Id() As Long
Get
Return m_Id
End Get
Set(ByVal value As Long)
m_Id = value
End Set
End Property
Private m_Id As Long
<ProtoMember(4)> _
Public Property Layer() As String
Get
Return m_Layer
End Get
Set(ByVal value As String)
m_Layer = value
End Set
End Property
Private m_Layer As String
End Class
Basically, I'm just serializing tons and tons of these classes into one file. Now I've come to find out that I have to save new parts of the map that aren't necessarily the same class type.
Is it possible to save multiple types to the same file and still read from it just as simply? Here is my code to write and read to/from the file:
Public Shared Sub Save(ByVal File As String, ByVal Map As RedSimEngine.Map)
Dim nPBL As New List(Of StrippedElement)
For z As Integer = 0 To Grid.LAYERLIMIT
For x As Integer = 0 To Grid.GRIDLIMIT
For y As Integer = 0 To Grid.GRIDLIMIT
Dim currentCell As GridElement = Map.Level.getCell(z, x, y)
If currentCell IsNot Nothing Then
If currentCell.Archivable Then
Dim nStEl As New StrippedElement
nStEl.Id = currentCell.getId()
nStEl.Layer = currentCell.getLayer()
nStEl.X = currentCell.X
nStEl.Y = currentCell.Y
nPBL.Add(nStEl)
End If
End If
Next
Next
Next
Serializer.Serialize(New FileStream(File, FileMode.Create), nPBL)
End Sub
Public Shared Function Load(ByVal File As String) As RedSimEngine.Map
Dim nMap As New Map
Dim nListOfSE As List(Of StrippedElement) = Serializer.Deserialize(Of List(Of StrippedElement))(New FileStream(File, FileMode.Open))
For Each elm As StrippedElement In nListOfSE
Dim nElm As GridElement = GridElement.createElementByIdAndLayer(elm.Layer, elm.Id)
nElm.X = elm.X
nElm.Y = elm.Y
nMap.Level.setCell(nElm)
Next
Return nMap
End Function
I have to add 3 or more class types to the save file, I'd rather not have it split up because then it would get confusing for my clients.
Basically, I have to add things similar to the following:
A class with X, Y, and Value
A class with Name and Value
A class with Name, ENUMVALUE, X, Y, INTEGERVALUE, and a couple other things (This one will have to contain quite a bit of data).
I'm using VB.net so all .net answers are acceptable. Thanks! If you need any clarification, just say so in the comments.
You have 3 options here:
The first option is to write a wrapper class with 3 containers:
[ProtoContract] public class MyData {
[ProtoMember(1)] public List<Foo> SomeName {get;set;} // x,y,value
[ProtoMember(2)] public List<Bar> AnotherName {get;set;} // name,value
[ProtoMember(3)] public List<Blap> ThirdName {get;set;} // etc
}
and serialize an instance of that; note, however, that the order will be lost here - i.e. after deserializing there is no difference between Foo0, Bar0, Foo1 and Foo0, Foo1, Bar0 - either will result in SomeName with Foo0 and Foo1, and AnotherName with Bar0. This option is compatible with your existing data, as internally there is no difference between serializing "a list of Foo" vs "a wrapper class with a list of Foo as field 1".
The second option is to mimic the above, but with a manual type-check during deserialization -this involves using the non-generic API and providing a delegate that maps field numbers (1,2,3) to types (Foo,Bar,Blap). This is useful for very large streams, or selectively plucking objects from the stream, as it allows you to process individual objects (or ignore them) individually. With this approach you can also build the file cumulatively rather than building an entire list. The example is a bit more complex, though, so I'd rather not add one unless it is of interest. This approach is compatible with your existing data.
The third approach is inheritance, i.e.
[ProtoContract, ProtoInclude(1, typeof(Foo))]
[ProtoInclude(2, typeof(Bar)), ProtoInclude(3, typeof(Blap))]
public class SomeBaseType {}
[ProtoContract] public class Foo : SomeBaseType { /* properties etc*/ }
[ProtoContract] public class Bar: SomeBaseType { /* properties etc*/ }
[ProtoContract] public class Blap: SomeBaseType { /* properties etc*/ }
then serialize a List<SomeBaseType> which happens to contain instances that are Foo / Bar / Blap. This is simple and convenient, and preserves order nicely; but it is not quite compatible with data serialized simply as a List<Foo> - if existing serialized data is an issue you will need to migrate between the formats.
i find "Serializer.Serialize" very unclean in this kind of cases, so here is how i would proceed :
i would write variables manually one at time!
for example, here is how i would write and read a StrippedElement :
Sub WriteStrippedElement(ByVal Stream As IO.Stream, ByVal SE As StrippedElement)
Stream.Write(BitConverter.GetBytes(SE.X), 0, 4) 'Write X:integer (4 bytes)
Stream.Write(BitConverter.GetBytes(SE.Y), 0, 4) 'Write Y:integer (4 bytes)
Stream.Write(BitConverter.GetBytes(SE.Id), 0, 8) 'Write Id:Long (8 bytes)
Dim LayerBuffer() As Byte = System.Text.Encoding.Default.GetBytes(SE.Layer) 'Converting String To Bytes
Stream.Write(BitConverter.GetBytes(LayerBuffer.Length), 0, 4) 'Write The length of layer, since it can't have a fixed size:integer (4 bytes)
Stream.Write(LayerBuffer, 0, LayerBuffer.Length) 'Write The Layer Data
Stream.Flush() 'We're Done :)
End Sub
Sub ReadStrippedElement(ByVal Stream As IO.Stream, ByRef SE As StrippedElement)
Dim BinRead As New IO.BinaryReader(Stream) 'Making reading Easier, We can also use a BinaryWriter in the WriteStrippedElement For example
SE.X = BinRead.ReadInt32 'Read 4 Bytes
SE.Y = BinRead.ReadInt32 'Read 4 Bytes
SE.Id = BinRead.ReadInt64 'Read 8 Bytes
Dim Length As Integer = BinRead.ReadInt32 'Read 4 Bytes, the length of Layer
Dim LayerBuffer() As Byte = BinRead.ReadBytes(Length) 'Read Layer Bytes
SE.Layer = System.Text.Encoding.Default.GetString(LayerBuffer) 'Convert Back To String, and done.
End Sub
So if you like to write alot of those StrippedElement, just write down the number of elements (int32, 4bytes) to know how much to read from the file next time.

FileHelpers Library - Append Multiple Records on Transform

Using the FileHelpers library to do some great things in VB.NET. Parsing files with dynamic classes built from text file templates. One thing I can't find: A way to read in a single record and determine that it should result in the generation of two records.
Current Code:
Dim FromType As Type = Dynamic.ClassBuilder.ClassFromSourceFile(MyFilePath, MyDynamicTypeName, NetLanguage.VbNet)
Dim FromRecords() As Object
FromRecords = FileHelpers.CommonEngine.ReadString(FromType, MyStringBuilder.ToString)
'... maybe code here to check for certain values
Dim engine As New FileTransformEngine(Of ITransformable(Of MyDestinationClass), MyDestinationClass)
' Ideally in this next line I would like it to see certain conditions and be able to generate two records from a single source line.
Dim PayRecords() As Object = engine.TransformRecords(FromRecords)
Alternately, if there is a way to implement the "ITransformable(Of ..." TransformTo() and have it return multiple records, I could put the logic in the dynamic class definition TransformTo() method.
Thoughts?
Here is a sample of my source dynamic class:
Imports FileHelpers ' Never forget
_
Public NotInheritable Class MyDynamicClass
Implements ITransformable(Of MyDestinationClass)
_
Public Name As String
<FieldQuoted(""""c, QuoteMode.OptionalForRead, MultilineMode.AllowForRead)> _
Public KeyType As String
Public Hours As Double
Public Function TransformTo() As MyDestinationClass Implements ITransformable(Of MyDestinationClass).TransformTo
Dim res As New MyDestinationClass
res.ContactName = Name
' Here is where I would like to say... instead of Return res
If KeyType="ABCD" Then
Dim newRes as New MyDestinationClass
newRes.Contactname = Name + " 2nd contact"
Dim resArray() as MyDestinationClass
redim resArray(1)
resArray(0) = res
resArray(1) = newRes
End If
Return resArray
' Or alternately refer to the engine, but it is not in scope for the dynamic record (is it?). Something like...
engine.AppendToDestination(new MyDestinationClass(...))
End Function
End Class
I think the main problem you're going to run into is that ITransformable.TransformTo() is spec'ed to return a single T value.
This is called in a loop when you call engine.TransformRecords(), where one output record is added for each input record.
I don't think this would be a big change if you didn't mind doing your own version of FileTransformEngine, but I don't see a clean way to do this as-is.