FileHelpers: The delimiter '|' can´t be found after the field - filehelpers

I have huge CSV file (1GB) which looks like that:
1140|2017|0213065852|2001|99|Ä‚/ŮŌçlš˝Ă_Âá'ÄǸ|-3858,18|4015,36|100,17|19,34|0,00|0,00|0,00|40,00|0,00|0,00|0,00|32,57|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|12|12|74|18|24620,65|11903,96|29385,35|10993,94|0,00|-2520,88|0|0100|301|02|0302|N|N|N| | |2001-12-27 00:00:00|N|1140|1313|BG4CSXF0| |2002-01-01 00:28:13| |0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00
1140|2017|0213065852|2002|99|ľë¸Â‰_gÂýľ)Ľ¸ů_°Ë
…p'™m'î'Ä_|-3799,76|3982,22|461,66|1,48|0,00|0,00|0,00|0,00|0,00|0,00|0,00|100,23|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|27|43|189|20|40140,74|48835,00|69605,32|19312,00|0,00|-3427,87|0|0100|301|02|0303|N|N|N| | |2002-12-27 00:00:00|N|1140|1313|BG4CSXF0| |2003-01-01 01:35:56| |0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00|0,00
When I try to read this file using FileHelpers using following code:
FileHelperEngine engine = new FileHelperEngine(typeof(BGDTMDF));
BGDTMDF[] tab = (BGDTMDF[])engine.ReadFile(path);
I get an error:
The delimiter '|' can´t be found after the field 'MDF_SALDOS_MEDIOS' at line 2 (the record has less fields, the delimiter is wrong or the next field must be marked as optional).
My class looks like:
[DelimitedRecord("|")]
public class BGDTMDF
{
public string MDF_ENTIDAD;
public string MDF_CENTRO_ALTA;
public string MDF_CUENTA;
public string MDF_ANIO;
public string MDF_MES;
[FieldNullValue("def")]
public string MDF_SALDOS_MEDIOS;
[FieldNullValue("def")]
public string MDF_SALDO_DISPUE;
public string MDF_SALDO_MAX_DEU;
public string MDF_INTE_COBRAD;
public string MDF_INTE_PAGADO;
public string MDF_IMPU_RETENI;
}
(there is only part of this class (so huge).)
Any help?

My solution is mark all fileds in class as [FieldOptional].

Related

Jpql query to extract array value

My database contain a column which is the type of varchar.
In that column i am storing value in the form [1,2] .
I need to get first value '1'.
I need to write jpql query using #query of spring data jpa.
I have decided to remove the braces of the value by using substring function then i need to convert it to array.
So, I have tried like this for the substring extraction.
SUBSTRING(u.output,1,LENGTH(u.output-2))
Is this syntax correct for substring extraction? how convert it to array.
If you cannot normalise the database then I would normalise the domain model via a JPA Converter.
You can these just load the entity and navigate the array as normal.
https://www.baeldung.com/jpa-attribute-converters
#Entity
public MyEntity{
#Convert(converter = MyArrayConverter.class)
int [] values;
}
Create a JPA Converter:
#Converter
public class MyArrayConverter implements
AttributeConverter<Integer [], String> {
#Override
public String convertToDatabaseColumn(int [] values) {
//convert int [] to string "[1,2,3]"
}
#Override
public Integer [] convertToEntityAttribute(String dbValue) {
//convert string "[1,2,3]" to int []
}
}

How to log multiple fields in EventSource

I am using EventSource to log event. I need to record 30 fields for one event. It works if I pass all 30 fields as string one by one as shown below. I was able to log those events. But when I trying to pass a string[] into the WriteEvent(). The isEnabled() returns false. Is there a better way to pass multiple fields into the WriteEvent() method?
The following is my source code for my event provider.
sealed class NemoTraceLoggingProvider : EventSource
{
public static NemoTraceLoggingProvider Log = new NemoTraceLoggingProvider();
[Event(1, Version = 12)]
public void SendTraceLoggingEvent(string Host, string Machine, string NemoVersion, string TraceId, string Operation, string HttpResult, string CallDuration, string Segment, string InputCells,
string InputTargetCells, string InputTargetContext, string IdentifyEntitySurfaceFormsIterations, string IdentifyEntitySurfaceFormsMaxDuration,
string IdentifyEntitySurfaceFormsInstancesCount, string ResolveMethod2Iterations, string ResolveMethod2MaxDuration,
string ResolveMethod2InstancesCount, string ResolveMethod3Iterations, string ResolveMethod3MaxDuration, string ResolveMethod3InstancesCount,
string ExtractPatternIterations, string ExtractPatternMaxDuration, string ExtractPatternInstancesCount, string NormalizeTextIterations,
string NormalizeTextMaxDuration, string NormalizeTextInstancesCount, string MaxDocumentTopics, string MaxSurfaceTopics, string MaxSurfaceWords,
string MaxTopicWords)
{
if (IsEnabled())
{
Console.ForegroundColor = ConsoleColor.Blue;
Console.WriteLine("TraceLogging: event is send out");
Console.ResetColor();
WriteEvent(1, Host, Machine, NemoVersion, TraceId, Operation, HttpResult, CallDuration, Segment, InputCells,
InputTargetCells, InputTargetContext, IdentifyEntitySurfaceFormsIterations, IdentifyEntitySurfaceFormsMaxDuration,
IdentifyEntitySurfaceFormsInstancesCount, ResolveMethod2Iterations, ResolveMethod2MaxDuration,
ResolveMethod2InstancesCount, ResolveMethod3Iterations, ResolveMethod3MaxDuration, ResolveMethod3InstancesCount,
ExtractPatternIterations, ExtractPatternMaxDuration, ExtractPatternInstancesCount, NormalizeTextIterations,
NormalizeTextMaxDuration, NormalizeTextInstancesCount, MaxDocumentTopics, MaxSurfaceTopics, MaxSurfaceWords,
MaxTopicWords);
}
else
{
Console.ForegroundColor = ConsoleColor.Blue;
Console.WriteLine("Not Enabled!!!");
Console.ResetColor();
}
}
}
}
Thanks!

Update Specific Record Embedded In document

This is my first time using MongoDB and Spring-data-mongo.
There is a Company object
#Document
public class Company {
#Id
private String id;
private String name;
private String registrationNumber;
private List<Vehicle> vehicles;
}
public class Vehicle {
private String vehicleOwner;
#Indexed(unique = true)
private String plateNumber;
private Double speedLimit;
private GeoJsonPoint currentLocation;
}
I would like to update currentLocation field for a vehicle with a given plateNumber
Obviousl, i'm stuck here
mongoTemplate.find(Query.query(Criteria.where("vehicles"))), how to go to `plateNumber` field? And how to update `currentLocation` field for that particular matching `Vehicle` for the `Company`
This should work, frame the query to fetch the required document and then create an Update object and populate values as shown below,
Query query = new Query(Criteria.where("name").is("company_name")
.and("vehicles")
.elemMatch(Criteria.where("vehicleOwner").is("owner_name")));
Update update = new Update();
update.set("vehicles.$.plateNumber", "NEW NUMBER");
//You can use findAndModify or updateMulti based on your need.
mongoTemplate.updateFirst(query, update, Company.class);
You can try the same for updating currentLocation.

Issue with Antlr Stringtemplate V4 ArrayList iteration

heres my test in java
public class person
{
public String name;
public int age;
public String getName() { return name; }
public int getAge() { return age; }
}
In my function I create a number of person objects
and add it into an list
ArrayList<person> arr = new ArrayList<person>()
arr.add(person1);
arr.add(person2); etc etc
in the string template group file I have got
test(arr) ::= <<
<table>
$arr: {a|
<tr><td>$a.name$</td><td>$a.age$</td></tr>
}$
</table>
>>
this is called from my list template
list (arr) ::= <<
$test(arr)$
... and other page details etc
>>
in version4 for I get template not found message with some stack trace as follows
Caused by: java.lang.ClassCastException: java.util.ArrayList
at org.antlr.runtime.tree.RewriteRuleTokenStream.nextNode(RewriteRuleTokenStream.java:58)
at org.stringtemplate.v4.compiler.STParser.subtemplate(STParser.java:1563)
at org.stringtemplate.v4.compiler.STParser.mapTemplateRef(STParser.java:3692)
but I can list an array as follows and it prints the object representation in string format
(also I can use a map - ie key value pairs works ok too)
in string template
test(arr) :: =<<
<p>
$arr; seperator="</br>"$
</p>
>>
how do we iterate list of object to print field values using string template grop in V4
am I using it in a wrong format/syntax?
any help/points would be greateful
note: we set $ as delimiter using new STGroupString("", templateGroup, '$', '$');
You are missing the terminating $ on your expression: $a.age should be $a.age$

Get name of the parsed file?

I would like to parse a file where the first line may or may not contain a definition of a "project" name (like with Pascal's program keyword), and if not, use the name of the file that is being parsed as default. Simplified:
#members{ String projectName; }
project : {projectName = ...} // name of parsed file as default
( PROJECTNAMEKEYWORD ID {projectName = $ID.text;} )?
otherstuff {/*...*/};
Is this even possible? I found this surprisingly hard to find out using google or the antlr manual. By its documentation, input.getSourceName() should be what I am looking for, but it always returns null, and debugging lead me to the class ANTLRStringStream that has a name field whose value is returned by this method but never set.
Simply create a custom constructor in your parser that takes a string that represents your file.
A demo:
grammar T;
#parser::members {
private String projectName;
public TParser(String fileName) throws java.io.IOException {
super(new CommonTokenStream(new TLexer(new ANTLRFileStream(fileName))));
projectName = fileName;
}
public String getProjectName() {
return projectName;
}
}
parse
: (PROJECT ID {projectName = $ID.text;})? EOF
;
PROJECT : 'project';
ID : ('a'..'z' | 'A'..'Z')+;
SPACE : (' ' | '\t' | '\r' | '\n') {skip();};
which can be tested with:
import org.antlr.runtime.*;
public class Main {
public static void main(String[] args) throws Exception {
// 'empty.txt' is, as the name suggests, an empty file
TParser parser = new TParser("empty.txt");
parser.parse();
System.out.println(parser.getProjectName());
// 'test.txt' contains a single line with the words: 'project JustATest'
parser = new TParser("test.txt");
parser.parse();
System.out.println(parser.getProjectName());
}
}
If you run the Main class, the following will be printed to your console:
empty.txt
JustATest