I'm having an issue with Quickfixn and I'm hoping someone with more experience working with it can shed some light on an issue I'm facing. For some reason, messages are getting rejected by the QuickFix engine because of repeating tags... I expect to have repeating tags and so I set the UseDataDictionary flag = Y in my config file but messages are still getting rejected. Has anyone experienced a similar issue ?
The message I'm receiving looks like :
8=FIXT.1.1 9=421 35=AE 34=8 1128=8 49=XXX 56=YYY 52=20130501-15:45:53 552=1 54=2 37=130501-5 11=NOREF 826=0 78=1 79=default 80=1000000.00 5967=12167800.00 453=4 448=ITXT 452=3 447=D 448=TEST 452=1 447=D 448=LMEB 452=16 447=D 448=FRTB 452=11 447=D 571=6718487 150=F 32=1000000.00 15=USD 1056=12167800.00 31=12.1678 194=12.1678 195=0 64=20130503 63=0 60=20130501-00:00:00 75=20130501 1057=Y 460=4 167=FOR 65=SP 55=USD/MXN 10=203
8=FIXT.1.1 9=124 35=3 34=8 49=XXX 52=20130501-15:45:54.209 56=YYY 45=8 58=Tag appears more than once 371=448 372=AE 373=13 10=210
my config file looks like this:
[DEFAULT]
ConnectionType=initiator
HeartBtInt=30
ReconnectInterval=10
SocketReuseAddress=Y
FileStorePath=D:\Store
FileLogPath=D:\Log
[SESSION]
BeginString=FIXT.1.1
SenderCompID=XXX
TargetCompID=YYY
DefaultApplVerId = FIX.5.0SP1
UseDataDictionary=Y
AppDataDictionary=D:\Interface\FIX50SP1.xml
StartDay=sunday
StartTime=20:55:00
EndTime=06:05:00
EndDay=saturday
SocketConnectHost=1.1.1.1
SocketConnectPort=8443
Any help would be greatly appreciated! Thank you.
Often this happens because there is a field in a repeating group that is not specified in the DataDictionary. The parser sees the field and assumes the repeating group has ended. It continues parsing fields as if they are not part of a group. If it sees a duplicate field in this context, the parser will report an error.
You may clone and modify FIX Data Dictionary (D:\Interface\FIX50SP1.xml) to fit your needs, if you need to process "invalid" messages. Or you may disable message validation.
Related
I have recently started seeing an error when i simply run
SELECT * FROM region-eu.INFORMATION_SCHEMA.JOBS_BY_PROJECT
Field name dml_stats does not exist in STRUCT<...
Excluding field dml_stats doesn't seem to work either.
Anyone else spotted this?
Cheers.
There are already 2 issues reported in the public issue tracker.
https://issuetracker.google.com/issues/193159258
https://issuetracker.google.com/issues/193437397
This seems to be a bug on the Google side. Hopefully, they will fix it soon.
I'm getting a really error message from Lucee:
‹Ä½írÛH’(úûöS`5'ÚîØ–D|zÚž (ÊR[”´$m¹gcc"!-à dyïØw¸¿nÄ9/·Or3 I PdÖÌ9Û–å¬Bf}äweþðë¿œ^g¿ßŒ”e¼ò”›O'—CåàðøøVŸÎN•/ç³ñ¥¢u”Yhû‘»o{ÇÇ£«å`Çë_ŽŸŸŸžõ£ |8žMŽ¿â\...
The whole page is nothing but this gibberish.
I'm completely at a loss. Has anyone ran across something like this before?
I'm trying to handle UDH data for multipart messages and I want to use an appropriate error code when there's a problem with the multipart fields. There are several error codes for bad TLV's, but I don't see anything except generic failure messages that I could use for UDH. Is that the best I can do here, or is there a more direct message I could use?
Your observation is correct - there is not a single error defined for UDH.
If you want, you can use error codes from reserved section and assign specific meaning to one/some of them. I think the "Reserved for SMSC vendor specific errors" (0x400-0x4FF) can be a good candidate.
Actually it looks to me like ESME_RX_R_APPN = 0x66 "ESME receiver reject message error." is the best error code to throw here.
It should correspond to a permanent error for this message only and let further messages be processed.
Using the Java SDK I am creating a load job for just a single record with a fairly complicated schema. When monitoring the status of the load job, it takes a surprisingly long time (but perhaps this is due to working out the schema), but then says:
11:21:06.975 [main] INFO xxx.GoogleBigQuery - Job status (21694ms) create_scans_1384744805079_172221126: DONE
11:24:50.618 [main] ERROR xxx.GoogleBigQuery - Job create_scans_1384744805079_172221126 caused error (invalid) with message
Too many errors encountered. Limit is: 0.
11:24:50.810 [main] ERROR xxx.GoogleBigQuery - {
"message" : "Too many errors encountered. Limit is: 0.",
"reason" : "invalid"
?}
BTW - how do I tell the job that it can have more than zero errors using Java?
This load job does not appear in the list of recent jobs in the console, and as far as I can see, none of the Java objects contains any more details about the actual errors encountered. So how can I pro-grammatically find out what is going wrong? All I can find is:
if (err != null) {
log.error("Job {} caused error ({}) with message\n{}", jobID, err.getReason(), err.getMessage());
try {
log.error(err.toPrettyString());
}
...
In general I am having a difficult time finding good documentation for some of these things and am working it out by trial and error and short snippets of code found on here and older groups. If there is a better source of information than the getting started guides, then I would appreciate any pointers to that information. The Javadoc does not really help and I cannot find any complete examples of loading, querying, testing for errors, cataloging errors and so on.
This job is submitted via a NEWLINE_DELIMITIED_JSON record, supplied to the job via:
InputStream dummy = getClass().getResourceAsStream("/googlebigquery/xxx.record");
final InputStreamContent jsonIn = new InputStreamContent("application/octet-stream", dummy);
createTableJob = bigQuery.jobs().insert(projectId, loadJob, jsonIn).execute();
My authentication and so on seems to work correctly as separate Java code to list the projects, and the datasets in the project all works correctly. So I just need help in working what the actual error is - does it not like the schema (I have records nested within records for instance), or does it think that there is an error in the data I am submitting.
Thanks in advance for any help. The job number cited above is an actual failed load job if that helps any Google staffers who might read this.
It sounds like you have a couple of questions, so I'll try to address them all.
First, the way to get the status of the job that failed is to call jobs().get(jobId), which returns a job object that has an errorResult object that has the error that caused the job to fail (e.g. "too many errors"). The errorStream list is a lost of all of the errors on the job, which should tell you which lines hit errors.
Note if you have the job id, it may be easier to use bq to lookup the job -- you can run bq show <job_id> to get the job error information. If you add the --format=prettyjson it will print out all of the information in the job.
A hint you also might want to consider is to supply your own job id when you create the job -- then even if there is an error starting the job (i.e. the insert() call fails, perhaps due to a network error) you can look up the job to see what actually happened.
To tell BigQuery that some errors are allowed during import, you can use the maxBadResults setting in the load job. See https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/java/latest/com/google/api/services/bigquery/model/JobConfigurationLoad.html#getMaxBadRecords().
My fix engine keeps rejecting messages and I was hoping someone could help me figure out why... I'm receiving the following sample message:
8=FIXT.1.1 9=518 35=AE 34=4 1128=8 49=XXXXXXX 56=YYYYYYY 52=20130322-17:58:37 552=1 54=1 37=Z00097H4ON 11=NOREF 826=0 78=1 79=NOT SPECIFIED 80=100000.000000 5967=129776.520000 453=5 448=BCART6 452=3 447=D 448=BARX 452=1 447=D 448=BARX 452=16 447=D 448=bcart6 452=11 447=D 448=ABCDEFGHI 452=12 447=D 571=6611540 150=F 17=Z00097H4ON 32=100000.000000 38=100000.000000 15=EUR 1056=129776.520000 31=1.2977652 194=1.298120 195=-3.5480 64=20130409 63=W2 60=20130322-17:26:50 75=20130322 1057=Y 460=4 167=FOR 65=OR 55=EUR/USD 10=121
8=FIXT.1.1 9=124 35=3 34=4 49=XXXXXXX 52=20130322-17:58:37.917 56=YYYYYYY 45=4 58=Tag appears more than once 371=448 372=AE 373=13 10=216
But as you can see it's being rejected by the quickfix engine. I am using the 5.0sp1 data dictionary and have configured it in my config file:
[DEFAULT]
ConnectionType=initiator
HeartBtInt=30
ReconnectInterval=10
SocketReuseAddress=Y
FileStorePath=D:\XXX\Interface\ReutersStore
FileLogPath=D:\XXX\Interface\ReutersLog
[SESSION]
BeginString = FIXT.1.1
SenderCompID = XXXXX
TargetCompID= YYYYY
DefaultApplVerId = FIX.5.0
UseDataDictionary=Y
AppDataDictionary=FIX50SP1.xml
StartDay=sunday
StartTime=20:55:00
EndTime=06:05:00
EndDay=saturday
SocketConnectHost= A.B.C.D
SocketConnectPort= 123
Does anyone have any idea why the Engine would be rejecting this message? I know that quickfix is normally able to handle messages with repeating groups, is it a config thing? Any help would be greatly appreciated!
Your message seems to be in order. Try putting this in your config file.
ValidateFieldsOutOfOrder=N
Quickfix by default puts that as Y and the underlying structure storing the tab and field values is unable to see the count before. 453 > 448.
As a sidenote always check these fields. They should point you to the source of the problem.
58=Tag appears more than once
371=448
Maybe it's a shot in the dark, but I had a similar a problem when using a 5.0sp2 dictionary.
I resolved using an updated version of the quickfix library compiled from the library SVN repository. If I remember correctly this was the bug.
It seems that the quickfix library has not been updated since a long time, and for newer version of fix I suggest you to use the "trunk" of the repo.
I had the same problem and i resolved it by tweaking my DataDictionary like the following in message AE TradeCaptureReport