I am having issues with NSB reverting to the Xml Serializer for no apparent reason which obvisouly causes an error when trying to deserialize the message. Below is an excerpt from the log, as you can see some come through from the same source as Json and then all of a sudden XML!
2018-06-05 08:51:36.924 INFO Stockly.Services.Messaging.Common.Extensions.MessageBodyWriter Serialized Message Body:
2018-06-05 08:51:37.026 INFO Stockly.Services.Messaging.Common.Extensions.MessageBodyWriter {"Exchange":"Binance","Symbol":"SYSBTC","MinutesPerCandle":15}
2018-06-05 08:51:38.505 INFO Stockly.Services.QuoteService.QuoteHandler Received quote request for SYSBTC and retrived 1000 candles
2018-06-05 08:51:38.725 INFO Stockly.Services.Messaging.Common.Extensions.MessageBodyWriter Serialized Message Body:
2018-06-05 08:51:38.728 INFO Stockly.Services.Messaging.Common.Extensions.MessageBodyWriter {"Exchange":"Binance","Symbol":"DASHBTC","MinutesPerCandle":15}
2018-06-05 08:51:39.194 INFO Stockly.Services.QuoteService.QuoteHandler Received quote request for DASHBTC and retrived 1000 candles
2018-06-05 08:51:39.228 INFO Stockly.Services.Messaging.Common.Extensions.MessageBodyWriter Serialized Message Body:
2018-06-05 08:51:39.230 INFO Stockly.Services.Messaging.Common.Extensions.MessageBodyWriter {"Exchange":"Binance","Symbol":"ENJBTC","MinutesPerCandle":15}
2018-06-05 08:51:39.715 INFO Stockly.Services.QuoteService.QuoteHandler Received quote request for ENJBTC and retrived 1000 candles
2018-06-05 08:51:39.774 INFO Stockly.Services.Messaging.Common.Extensions.MessageBodyWriter Serialized Message Body:
2018-06-05 08:51:39.777 INFO Stockly.Services.Messaging.Common.Extensions.MessageBodyWriter BinanceRPXBTC15
2018-06-05 08:51:39.824 ERROR NServiceBus.RecoverabilityExecutor Moving message 'f095da8a-e830-4873-8fa0-a8f200719739' to the error queue 'error' because processing failed due to an exception:
NServiceBus.MessageDeserializationException: An error occurred while attempting to extract logical messages from incoming physical message f095da8a-e830-4873-8fa0-a8f200719739 ---> Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
at Newtonsoft.Json.JsonTextReader.ParseValue()
Here is the configuration.
public static EndpointConfiguration ConfigureSerialization(this EndpointConfiguration endpointConfiguration)
{
var serialization = endpointConfiguration.UseSerialization<NewtonsoftSerializer>();
serialization.ContentTypeKey("NewtonsoftJson");
var settings = new JsonSerializerSettings
{
TypeNameHandling = TypeNameHandling.Auto,
Converters =
{
new IsoDateTimeConverter
{
DateTimeStyles = DateTimeStyles.RoundtripKind
}
}
};
serialization.Settings(settings);
endpointConfiguration.RegisterMessageMutator(new MessageBodyWriter());
var externalNewtonsoftJson = endpointConfiguration.AddDeserializer<NewtonsoftSerializer>();
externalNewtonsoftJson.ContentTypeKey("NewtonsoftJson");
return endpointConfiguration;
}
Any idea why this might be happening and how I can stop this from occurring?
P.S. at the moment I am using the LearningTransport.
This turned out to be some left over messages from previous test runs that still used xml as the serialization format. Clearing the input queue solved the issue.
More details: https://github.com/Particular/NServiceBus/issues/5181#issuecomment-396141572
Related
POST request without body parameter gives a JSON parse error. Mentioned the error below.
API Call
api.post('notification/seen')
Error:
{"message":{"name":"SyntaxError","msg":"Unexpected token n in JSON at position 0","stack":"SyntaxError: Unexpected token n in JSON at position 0\n at JSON.parse ()\n at createStrictSyntaxError (/app/node_modules/body-parser/lib/types/json.js:158:10)\n at parse (/app/node_modules/body-parser/lib/types/json.js:83:15)\n at /app/node_modules/body-parser/lib/read.js:121:18\n at invokeCallback (/app/node_modules/raw-body/index.js:224:16)\n at done (/app/node_modules/raw-body/index.js:213:7)\n at IncomingMessage.onEnd (/app/node_modules/raw-body/index.js:273:7)\n at IncomingMessage.emit (events.js:327:22)\n at IncomingMessage.EventEmitter.emit (domain.js:486:12)\n at endReadableNT (_stream_readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21)"},"level":"error","timestamp":"2021-09-28T06:29:19.577Z"}
Swagger API structure
/notification/seen:
post:
tags:
- Notification
responses:
204:
description: Update Seen Notification Success
401:
description: Access Token Missing/Expired
500:
description: Something went wrong!
If same API I call with adding dummy body as below with changing the Swagger API structure I am not getting the error.
API Call
api.post('notification/seen',{id:"data"})
Swagger API structure
/notification/seen:
post:
tags:
- Notification
parameters:
- in: body
name: body
schema:
type: object
responses:
204:
description: Update Seen Notification Success
401:
description: Access Token Missing/Expired
500:
description: Something went wrong!
And also if I change POST to GET request then also I am not getting the error.
How to fix the error with the POST request without a body parameter?
This question already has an answer here:
How to upload CSV file as a post request in Karate 0.9.0 version?
(1 answer)
Closed 2 years ago.
I have an file upload endpoint (/document) in a controller defined as follows:
#RestController
public class FileUploadController {
#Autowired
private PersonCSVReaderService personCSVReaderService;
#PostMapping(value = "/document", consumes= {MediaType.MULTIPART_FORM_DATA_VALUE})
public void handleFileUpload3(#RequestPart("file") MultipartFile file, #RequestPart("metadata") DocumentMetadata metadata) {
System.out.println(String.format("uploading file %s of %s bytes", file.getOriginalFilename(), file.getSize()));
personCSVReaderService.readPersonCSV(file, metadata);
}
}
I can test this endpoint using Advanced Rest Client (ARC) or Postman by defining the "file" part referencing the people.csv file and a text part specifying some sample metadata JSON.
Everything works fine and I get a 200 status back with the people.csv file contents being written to the console output by the service method:
uploading file people.csv of 256 bytes
{Address=1, City=2, Date of Birth=6, Name=0, Phone Number=5, State=3, Zipcode=4}
Person{name='John Brown', address='123 Main St.', city='Scottsdale', state='AZ', zipcode='85259', phoneNumber='555-1212', dateOfBirth='1965-01-01'}
Person{name='Jan Black', address='456 University Dr.', city='Atlanta', state='GA', zipcode='30306', phoneNumber='800-1111', dateOfBirth='1971-02-02'}
Person{name='Mary White', address='789 Possum Rd.', city='Nashville', state='TN', zipcode='37204', phoneNumber='888-2222', dateOfBirth='1980-03-03'}
Now, I want to run this as an automated Karate test. I have specified a MockConfig :
#Configuration
#EnableAutoConfiguration
#PropertySource("classpath:application.properties")
public class MockConfig {
// Services ...
#Bean
public PersonCSVReaderService personCSVReaderService() {
return new PersonCSVReaderService();
}
// Controllers ...
#Bean
public FileUploadController fileUploadController() {
return new FileUploadController();
}
}
I also have a MockSpringMvcServlet in the classpath and my karate-config.js is :
function fn() {
var env = karate.env; // get system property 'karate.env'
if (!env) {
env = 'dev';
}
karate.log('karate.env system property was:', env);
var config = {
env: env,
myVarName: 'someValue',
baseUrl: 'http://localhost:8080'
}
if (env == 'dev') {
var Factory = Java.type('MockSpringMvcServlet');
karate.configure('httpClientInstance', Factory.getMock());
//var result = karate.callSingle('classpath:demo/headers/common-noheaders.feature', config);
} else if (env == 'stg') {
// customize
} else if (env == 'prod') {
// customize
}
return config;
}
Other karate tests run ok using the mock servlet.
However, when I try this test to test the /document endpoint:
Feature: file upload end-point
Background:
* url baseUrl
* configure lowerCaseResponseHeaders = true
Scenario: upload file
Given path '/document'
And header Content-Type = 'multipart/form-data'
And multipart file file = { read: 'people.csv', filename: 'people.csv', contentType: 'text/csv' }
And multipart field metadata = { name: "joe", description: "stuff" }
When method post
Then status 200
I get this error:
16:14:42.674 [main] INFO com.intuit.karate - karate.env system property was: dev
16:14:42.718 [main] INFO o.s.mock.web.MockServletContext - Initializing Spring DispatcherServlet ''
16:14:42.719 [main] INFO o.s.web.servlet.DispatcherServlet - Initializing Servlet ''
16:14:43.668 [main] INFO o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$a4c7d08f] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
16:14:43.910 [main] INFO o.h.validator.internal.util.Version - HV000001: Hibernate Validator 6.0.14.Final
16:14:44.483 [main] INFO o.s.s.c.ThreadPoolTaskExecutor - Initializing ExecutorService 'applicationTaskExecutor'
16:14:44.968 [main] INFO o.s.b.a.e.web.EndpointLinksResolver - Exposing 2 endpoint(s) beneath base path '/actuator'
16:14:45.006 [main] INFO o.s.web.servlet.DispatcherServlet - Completed initialization in 2287 ms
16:14:45.066 [main] INFO c.i.k.mock.servlet.MockHttpClient - making mock http client request: POST - http://localhost:8080/document
16:14:45.085 [main] DEBUG c.i.k.mock.servlet.MockHttpClient -
1 > POST http://localhost:8080/document
1 > Content-Type: multipart/form-data
16:14:45.095 [main] ERROR com.intuit.karate - http request failed: null
file-upload.feature:13 - null
HTML report: (paste into browser to view) | Karate version: 0.9.2
I can only assume that the arguments did not conform to what my endpoint was expecting - I never entered the endpoint in debug mode.
I tried this:
And multipart file file = read('people.csv')
And multipart field metadata = { name: "joe", description: "stuff" }
But that was a non-starter as well.
What am I doing wrong? The people.csv is in the same folder as fileupload.feature, so I am assuming it is finding the file. I also looked at upload.feature file in the Karate demo project given here:
Karate demo project upload.feature
But I could not make it work. Any help appreciated. Thanks in advance.
The Postman request looks like this:
EDIT UPDATE:
I could not get the suggested answer to work.
Here is the feature file:
Feature: file upload
Background:
* url baseUrl
* configure lowerCaseResponseHeaders = true
Scenario: upload file
Given path '/document'
And header Content-Type = 'multipart/form-data'
* def temp = karate.readAsString('people.csv')
* print temp
And multipart file file = { value: '#(temp)', filename: 'people.csv', contentType: 'text/csv' }
And multipart field metadata = { value: {name: 'joe', description: 'stuff'}, contentType: 'application/json' }
When method post
Then status 200
And here is the console output from running that test:
09:27:22.051 [main] INFO com.intuit.karate - found scenario at line: 7 - ^upload file$
09:27:22.156 [main] INFO com.intuit.karate - karate.env system property was: dev
09:27:22.190 [main] INFO o.s.mock.web.MockServletContext - Initializing Spring DispatcherServlet ''
09:27:22.190 [main] INFO o.s.web.servlet.DispatcherServlet - Initializing Servlet ''
09:27:23.084 [main] INFO o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$a4c7d08f] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
09:27:23.327 [main] INFO o.h.validator.internal.util.Version - HV000001: Hibernate Validator 6.0.14.Final
09:27:23.896 [main] INFO o.s.s.c.ThreadPoolTaskExecutor - Initializing ExecutorService 'applicationTaskExecutor'
09:27:24.381 [main] INFO o.s.b.a.e.web.EndpointLinksResolver - Exposing 2 endpoint(s) beneath base path '/actuator'
09:27:24.418 [main] INFO o.s.web.servlet.DispatcherServlet - Completed initialization in 2228 ms
09:27:24.435 [main] INFO com.intuit.karate - [print] Name,Address,City,State,Zipcode,Phone Number,Date of Birth
John Brown,123 Main St.,Scottsdale,AZ,85259,555-1212,1965-01-01
Jan Black,456 University Dr.,Atlanta,GA,30306,800-1111,1971-02-02
Mary White,789 Possum Rd.,Nashville,TN,37204,888-2222,1980-03-03
09:27:24.482 [main] INFO c.i.k.mock.servlet.MockHttpClient - making mock http client request: POST - http://localhost:8080/document
09:27:24.500 [main] DEBUG c.i.k.mock.servlet.MockHttpClient -
1 > POST http://localhost:8080/document
1 > Content-Type: multipart/form-data
09:27:24.510 [main] ERROR com.intuit.karate - http request failed: null
file-upload.feature:14 - null
HTML report: (paste into browser to view) | Karate version: 0.9.2
Note: people.csv file reads successfully and prints in console.
Refer to this part of the docs: https://github.com/intuit/karate#read-file-as-string
So make this change:
* def temp = karate.readAsString('people.csv')
And multipart file file = { value: '#(temp)', filename: 'people.csv', contentType: 'text/csv' }
EDIT: my bad, also refer: https://github.com/intuit/karate#multipart-file
Feature: upload csv
Background: And def admin = read('classpath:com/project/data/adminLogin.json')
Scenario:
Given url baseUrl
And header Authorization = admin.token
And multipart file residentDetails = { read:'classpath:com/project/data/ResidentApp_Details.csv', filename: 'ResidentApp_Details.csv' }
When method POST
Then status 200
Note: Add only one extra line i.e And multipart file residentDetails = { read:'classpath:com/project/data/ResidentApp_Details.csv', filename: 'ResidentApp_Details.csv' }
Well, I've started a simple project with freediameter library. what I need to achieve in my project is to create a client diameter application that could send some CCR request to a diameter server.
for this matter, I tried to create a new extension for freediameter daemon.
So, the first thing I did was to setup the peer diameter server in config file:
ConnectPeer = "vm-pc.my.domain" { No_TLS; ConnectTo = "192.168.56.2"; Port=3868; };
and then initialize a request message in ta_entry function, in message body i specified the Destination-Host and Destination-Realm but still I receive this error:
01/10/17,01:55:24.980611 ERROR Routing error: 'No remaining suitable candidate to route the message to' for the following message:
01/10/17,01:55:24.980620 ERROR 'Credit-Control-Request'
01/10/17,01:55:24.980628 ERROR Version: 0x01
01/10/17,01:55:24.980635 ERROR Length: 20
01/10/17,01:55:24.980642 ERROR Flags: 0xC0 (RP--)
01/10/17,01:55:24.980649 ERROR Command Code: 272
01/10/17,01:55:24.980656 ERROR ApplicationId: 4
01/10/17,01:55:24.980664 ERROR Hop-by-Hop Identifier: 0x00000000
01/10/17,01:55:24.980671 ERROR End-to-End Identifier: 0xDCA05EF4
01/10/17,01:55:24.980678 ERROR {internal data}: src:(nil)(0) rwb:0x0 rt:0 cb:0x80523df30,0x0(0x803397da0) qry:0x0 asso:0 sess:0x0
01/10/17,01:55:24.981562 ERROR AVP: 'Session-Id'(263) l=8 f=-M val="hadi-pc.my.domain;1484000714;6"
01/10/17,01:55:24.981569 ERROR AVP: 'Origin-Host'(264) l=8 f=-M val="hadi-pc.my.domain"
01/10/17,01:55:24.981577 ERROR AVP: 'Origin-Realm'(296) l=8 f=-M val="my.domain"
01/10/17,01:55:24.981584 ERROR AVP: 'Destination-Host'(293) l=8 f=-M val="vm-pc.my.domain"
01/10/17,01:55:24.981591 ERROR AVP: 'Destination-Realm'(283) l=8 f=-M val="my.domain"
01/10/17,01:55:24.981599 ERROR AVP: 'Auth-Application-Id'(258) l=12 f=-M val=4 (0x4)
01/10/17,01:55:24.981606 ERROR AVP: 'CC-Request-Type'(416) l=12 f=-M val='EVENT_REQUEST' (4 (0x4))
01/10/17,01:55:24.981613 ERROR AVP: 'CC-Request-Number'(415) l=12 f=-M val=2 (0x2)
Can anybody help me on this ?
ps: as I see in the logs the CER/CEA are normal.
What Origin-Realm was signaled in the Capability-Exchange-Answer message when the connection was established?
The Diameter request routing process (as described in https://www.rfc-editor.org/rfc/rfc6733#section-6.1) relies on the realm, so if the Origin-Realm returned by the peer is not "my.domain", freeDiameter will not route to this peer, even if the Origin-Host matches.
In my application, after I get a message from rabbitmq, I push the message to a client, and wait for its ACK message, if the client doesn't reply with a ACK after some time, I requeue the message in rabbitmq with basic.reject with requeue being true.
This works fine for the first requeue operation, but after I requeue the same message for the second time, the channel is closed abruptly. From the server log, I get this error:
{amqp_error,precondition_failed,"unknown delivery tag 2",'basic.reject'}
I gather this is because the message has been removed from the queue. Why is this happening?
I had similar error
Error: Channel closed by server: 406 (PRECONDITION-FAILED) with message "PRECONDITION_FAILED - unknown delivery tag 1"
at Channel.C.accept (C:\Owlet\shopify_email_server\node_modules\amqplib\lib\channel.js:422:17)
at Connection.mainAccept [as accept] (C:\Owlet\shopify_email_server\node_modules\amqplib\lib\connection.js:64:33)
at Socket.go (C:\Owlet\shopify_email_server\node_modules\amqplib\lib\connection.js:478:48)
at Socket.emit (node:events:390:28)
at Socket.emit (node:domain:475:12)
at emitReadable_ (node:internal/streams/readable:578:12)
at processTicksAndRejections (node:internal/process/task_queues:82:21) {
code: 406,
classId: 60,
methodId: 80
My solution was to assert channel with enabled option durable
channel.assertQueue(queueName, { durable: true });
The below code shows how I am setting header and message type to AMQP message.
MessageProperties properties = new MessageProperties();
properties.setHeader("KEY", "HOUSE");
properties.setContentType(MessageProperties.CONTENT_TYPE_JSON);
Message message = new Message("1234567;Branch A;SALES;3000.50;Pending approval".getBytes(), properties);
rabbitTemplate.sendAndReceive("", QUEUE_NAME, message);
After sending the message in the queue, the message is received by Transformer.
#Transformer(inputChannel = "inboundChannel", outputChannel = "toutboundChannel")
public Property buildProperty(Message<String> property){
LOGGER.info("message received :: HEADERS: {}, PAYLOAD :{}", property.getHeaders(), property.getPayload());
....
}
In the logs, the header "KEY: HOUSE" is missing and even the message type is not JSON and "text/plain" instead.
LOGS:
[SimpleAsyncTaskExecutor-1] INFO com.demo.maven.spring.integration.endpoint.TransformerRequestBuilder - message received :: HEADERS: {amqp_receivedRoutingKey=mobile.queue, amqp_deliveryTag=2, amqp_replyTo=amq.rabbitmq.reply-to.g2dkABByYWJiaXRAbG9jYWxob3N0AAAW9QAAAAAD.tTIFOS2gsM7qIlGYaybfrg==, amqp_deliveryMode=PERSISTENT, amqp_redelivered=true, id=399dda4f-4ba1-7cf4-2310-03dbfbac82b6, contentType=text/plain, timestamp=1421649922840}, PAYLOAD :1234567;Branch A;SALES;3000.50;Pending approval
MessagePropertiesBuilder class is for that.
By default Spring Integration AMQP Inbound Endpoint (AmqpInboundChannelAdapter and AmqpInboundGateway) maps only standard AMQP headers. That's is a default behaviour of DefaultAmqpHeaderMapper. To accept any user-specofic headers you should inject AmqpHeaderMapper (setHeaderMapper) to that inbound endpoint with an option setRequestHeaderNames("*"). Or provide full list of names of desired custom headers.
Re. contentType=text/plain: I think something between AMQP Inbound Endpoint and that #Transformer(inputChannel = "inboundChannel" overrides the received from AMQP contentType header. Because RabbitTemplate doesn't do that, if you send Message not any other Object. Please, share DEBUG logs for the org.springframework.integration category for the message receiver. Of course we need that part of logs, when you receive message till that #Transformer
This will work, you have to build the messageproperties correctly.
MessageProperties properties = new MessageProperties();
properties.builder()
.contentType(MediaType.APPLICATION_JSON)
//headers here
.headers(Map<String, Object>)
.build();