Java NIO ByteBuffer : read the message size on head before read the complete message - size

I'm making a java NIO server which receive messages, each message have its size on the head of the message, that why I'm reading first into a buffer which have default size(44), then get the complete size from this buffer, and then create a new buffer which is supposed to get the rest of the message(body) and then use System.arrayCopy() to make an array with the complete message. this operations are working good, the problem is that the second buffer(body of the message)have the size but does not contain the right data.
plz here is my code if you something wrong :
public void getMessageByMessageSize(SelectionKey key) {
socket = (SocketChannel) key.channel();
int nBytes = 0;
byte[] message = null;
try {
nBytes = socket.read(headBuffer);
if (nBytes < 0) {
try {
key.channel().close();
key.cancel();
return;
} catch (IOException e) {
e.printStackTrace();
}
}
//size of the message body
int corpMessageSize = MessageUtils.getMessageSize(headBuffer)
- HEADER_SIZE;
ByteBuffer corpsBuffer = ByteBuffer.allocate(corpMessageSize);
headBuffer.flip();
nBytes += socket.read(corpsBuffer);
corpsBuffer.flip();
byte[] corp=corpsBuffer.array();
message = new byte[nBytes];
System.arraycopy(headBuffer.array(), 0, message, 0, HEADER_SIZE);
System.arraycopy(corpsBuffer.array(), 0, message, HEADER_SIZE,
nBytes - HEADER_SIZE);
System.out.println(nBytes);
headBuffer.clear();
corpsBuffer.clear();
} catch (IOException e) {
e.printStackTrace();
try {
key.channel().close();
key.cancel();
return;
} catch (IOException ex) {
e.printStackTrace();
}
}
this.worker.verifyConnection(this,message, key);
//this.worker.processData(this, socket, message, nBytes);
}
i have a simple client which send create a byte message, make its size in the head, and then send it.
thanks

nBytes += socket.read(corpsBuffer);
You are assuming that this reads the entire message and fills the buffer. Nothing in TCP/IP guarantees that. You have to loop. If you're in non-blocking mode, you have to re-select if you get a zero length read.

Related

netty udp can not receive or send msg

I am using netty UDP as the IOT server of our company's equipment.
But every few days, udpserver can no longer send and receive messages, and our equipment can't connect to the server.
Sometimes it'll take a while.
Sometimes it still doesn't work after a while, so you need to restart the server.
I don't know why.
I guess is it because:
here bytebuf = packet copy().content();
the packet is too long ,cause the port 'cheat death'(can not receive or send msg),
How to limit the size of the packet?
Here is my code:
#Override
public void channelRead0(ChannelHandlerContext channelHandlerContext, DatagramPacket packet) {
ByteBuf buf = packet.copy().content();
// 大于1000 个字节不再处理 packet too long!
if (buf.capacity() > 1000) {
log.warn("收到过长的数据包 此包不再处理 packet too big ,discard");
return;
}
try {
byte[] b = new byte[buf.readableBytes()];
buf.readBytes(b);
// do sth....
}
catch (Exception e) {
log.error(e.getMessage(), e);
}
finally {
buf.release();
}
}
}

JPOS NACChannel Unable to Parse Incoming Message with Header

I am using NACChannel of JPOS v2.1.0. I am using GenericPackager for packing my message. I am able to successfully send message to my ISO application. But while receiving incoming message my NACChannel.receive() throws parsing error.
My incoming message has a custom Header. I suspect this is causing the parsing error. So my questions are:
How can I handle this ISOMsg header in my incoming response?
Is there any way I can disable the Parsing Step and Receive the byte[] response from the channel?
My JPOS client code I am using:
public class BuildISOMessage {
public static void main(String[] args) throws IOException, ISOException {
createISOMessage();
}
public static byte[] createISOMessage() throws ISOException {
String HOST = "localhost";
int PORT = 40021;
// Create Packager based on XML that contain DE type
ISOBasePackager packager = new GenericPackager("800_fields.xml");
NACChannel testchannel = new NACChannel();
testchannel.setHost(HOST, PORT);
ISOMsg isoMsg = build200ISOMessage(packager);
// print the DE list
logISOMsg(isoMsg);
// Get and print the output result
byte[] data = isoMsg.pack();
try {
testchannel.setPackager(isoMsg.getPackager());
testchannel.setHeader(("ISO01" + String.format("%04d", data.length)).getBytes());
testchannel.connect();
testchannel.send(isoMsg);
if (testchannel.isConnected()) {
ISOMsg response = testchannel.receive();
}
testchannel.disconnect();
} catch (Exception e) {
e.printStackTrace();
}
return data;
}
private static ISOMsg build200ISOMessage(ISOBasePackager packager)
throws ISOException {
// Create ISO Message
ISOMsg isoMsg = new ISOMsg();
isoMsg.setPackager(packager);
isoMsg.setMTI("200");
isoMsg.set(2, "4622871000060891");
isoMsg.set(3, "300000");
isoMsg.set(4, "100");
isoMsg.set(7, "1026043633");
isoMsg.set(11, "999901");
isoMsg.set(12, "113633");
isoMsg.set(13, "1026");
isoMsg.set(15, "1116");
isoMsg.set(16, "1116");
isoMsg.set(18, "6011");
isoMsg.set(22, "21");
isoMsg.set(32, "0000004");
isoMsg.set(33, "0000004");
isoMsg.set(35, "4622871000060891=22082211963393100000");
isoMsg.set(37, "829911364035");
isoMsg.set(43, "TBNKTAS2B065B999P999300501000050 TH");
isoMsg.set(48, "00000000040000000002000000013000000000005000000000007000TYRIONLANNISER ARYA STARK000000003334000000000202 00000000000000000000");
isoMsg.set(49, "764");
isoMsg.set(52, "FFFFFFFFFFFFFFFF");
isoMsg.set(62, "221000000000");
return isoMsg;
}
private static void logISOMsg(ISOMsg msg) {
System.out.println("----ISO MESSAGE-----");
try {
System.out.println(" MTI : " + msg.getMTI());
for (int i = 1; i <= msg.getMaxField(); i++) {
if (msg.hasField(i)) {
System.out.println(" Field-" + i + " : "
+ msg.getString(i));
}
}
} catch (ISOException e) {
e.printStackTrace();
} finally {
System.out.println("--------------------");
}
}
}
Thanks all for your help.
While looking for an answer for debug purpose, I created a subclass of NACChannel and put some debug statements. By doing that I realized the issue was with my field definition and nothing to do with JPOS framework.
I was setting a header(e.g. ISO010200 where message length is 200) of length '9' by the below code.
testchannel.setHeader(("ISO01" + String.format("%04d", data.length)).getBytes());
My response also had a similar header of length '9'.
So the NACChanel receive() method was able to extract the 9 digit Header correctly.
But failed to parse the response message, my field definition was not correct.
Once that was fixed, JPOS was able to parse the response message coorectly.

How to move messages from one queue to another in RabbitMQ

In RabbitMQ,I have a failure queue, in which I have all the failed messages from different Queues. Now I want to give the functionality of 'Retry', so that administrator can again move the failed messages to their respective queue. The idea is something like that:
Above diagram is structure of my failure queue. After click on Retry link, message should move into original queue i.e. queue1, queue2 etc.
If you are looking for a Java code to do this, then you have to simply consume the messages you want to move and publish those messages to the required queue. Just look up on the Tutorials page of rabbitmq if you are unfamiliar with basic consuming and publishing operations.
It's not straight forward consume and publish. RabbitMQ is not designed in that way. it takes into consideration that exchange and queue both could be temporary and can be deleted. This is embedded in the channel to close the connection after single publish.
Assumptions:
- You have a durable queue and exchange for destination ( to send to)
- You have a durable queue for target ( to take from )
Here is the code to do so:
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.QueueingConsumer;
import org.apache.commons.lang.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.rabbit.connection.CachingConnectionFactory;
public object shovelMessage(
String exchange,
String targetQueue,
String destinationQueue,
String host,
Integer port,
String user,
String pass,
int count) throws IOException, TimeoutException, InterruptedException {
if(StringUtils.isEmpty(exchange) || StringUtils.isEmpty(targetQueue) || StringUtils.isEmpty(destinationQueue)) {
return null;
}
CachingConnectionFactory factory = new CachingConnectionFactory();
factory.setHost(StringUtils.isEmpty(host)?internalHost.split(":")[0]:host);
factory.setPort(port>0 ? port: Integer.parseInt(internalPort.split(":")[1]));
factory.setUsername(StringUtils.isEmpty(user)? this.user: user);
factory.setPassword(StringUtils.isEmpty(pass)? this.pass: pass);
Channel tgtChannel = null;
try {
org.springframework.amqp.rabbit.connection.Connection connection = factory.createConnection();
tgtChannel = connection.createChannel(false);
tgtChannel.queueDeclarePassive(targetQueue);
QueueingConsumer consumer = new QueueingConsumer(tgtChannel);
tgtChannel.basicQos(1);
tgtChannel.basicConsume(targetQueue, false, consumer);
for (int i = 0; i < count; i++) {
QueueingConsumer.Delivery msg = consumer.nextDelivery(500);
if(msg == null) {
// if no message found, break from the loop.
break;
}
//Send it to destination Queue
// This repetition is required as channel looses the connection with
//queue after single publish and start throwing queue or exchange not
//found connection.
Channel destChannel = connection.createChannel(false);
try {
destChannel.queueDeclarePassive(destinationQueue);
SerializerMessageConverter serializerMessageConverter = new SerializerMessageConverter();
Message message = new Message(msg.getBody(), new MessageProperties());
Object o = serializerMessageConverter.fromMessage(message);
// for some reason msg.getBody() writes byte array which is read as a byte array // on the consumer end due to which this double conversion.
destChannel.basicPublish(exchange, destinationQueue, null, serializerMessageConverter.toMessage(o, new MessageProperties()).getBody());
tgtChannel.basicAck(msg.getEnvelope().getDeliveryTag(), false);
} catch (Exception ex) {
// Send Nack if not able to publish so that retry is attempted
tgtChannel.basicNack(msg.getEnvelope().getDeliveryTag(), true, true);
log.error("Exception while producing message ", ex);
} finally {
try {
destChannel.close();
} catch (Exception e) {
log.error("Exception while closing destination channel ", e);
}
}
}
} catch (Exception ex) {
log.error("Exception while creating consumer ", ex);
} finally {
try {
tgtChannel.close();
} catch (Exception e) {
log.error("Exception while closing destination channel ", e);
}
}
return null;
}
To requeue a message you can use the receiveAndReply method. The following code will move all messages from the dlq-queue to the queue-queue:
do {
val movedToQueue = rabbitTemplate.receiveAndReply<String, String>(dlq, { it }, "", queue)
} while (movedToQueue)
In the code example above, dlq is the source queue, { it } is the identity function (you could transform the message here), "" is the default exchange and queue is the destination queue.
I also have implemented something like that, so I can move messages from a dlq back to processing. Link: https://github.com/kestraa/rabbit-move-messages
Here is a more generic tool for some administrative/supporting tasks, the management-ui is not capable of.
Link: https://github.com/bkrieger1991/rabbitcli
It also allows you to fetch/move/dump messages from queues even with a filter on message-content or message-headers :)

Send many byte array from server to UWP as a stream

I have a WCF service that updates the byte array from camera continuous like this:
private void ImageGrabbedCamera(object sender, ImageGrabbedEventArgs e)
{
try
{
if (e.GrabResult.GrabSucceeded)
{
//This variable is updated continuous from the camera
result = e.GrabResult.Clone();
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
And I call this method from the UWP client app to get the byte array from the server
public Stream GetStreamCamera()
{
MemoryStream ms;
if (result != null)
{
ms = new MemoryStream(ObjectToByteArray(result.PixelData as byte[]));
ms.Position = 0;
return ms;
}
else
{
return new MemoryStream();
}
}
In client app, I call while(true) to GetStreamCamera() method to get the frame but it not OK because the capacity is very big and it's not only 1 camera, it about 10 cameras with resolution (1280 * 960). So do we have any protocol that UWP support to stream the image from the server to UWP client?
I don't want to call while(true) to get 1 frame/call anymore.

How can I do parallel in one session?

My Download function is below. It's in Controller Class:
public void Download(string fileId)
{
// **************************************************
//MAKE FILEPATH
string filePath = makeFilePath(fileId);
string outFileName = _info.FileName;
System.IO.Stream iStream = null;
// Create buffer for reading [intBufferSize] bytes from file
int intBufferSize = 10 * 1024;
byte[] buffer = new System.Byte[intBufferSize];
// Length of the file That Really Has Been Read From The Stream and Total bytes to read
int length; long dataToRead;
// **************************************************
if (System.IO.File.Exists(filePath))
{
try
{
// Open the file.
iStream = new System.IO.FileStream(
path: filePath,
mode: System.IO.FileMode.Open,
access: System.IO.FileAccess.Read,
share: System.IO.FileShare.Read);
// Total bytes to read:
dataToRead = iStream.Length;
// **************************************************
Response.Clear();
// Setting the unknown [ContentType]
// will display the saving dialog for the user
Response.ContentType = "application/octet-stream";
// With setting the file name,
// in the saving dialog, user will see
// the [outFileName] name instead of [download]!
Response.AddHeader("Content-Disposition", "attachment; filename=" + outFileName);
// Notify user (client) the total file length
Response.AddHeader("Content-Length", iStream.Length.ToString());
// **************************************************
// Read the bytes.
while (dataToRead > 0)
{
// Verify that the client is connected.
if (Response.IsClientConnected)
{
// Read the data and put it in the buffer.
length = iStream.Read(buffer: buffer, offset: 0, count: intBufferSize);
// Write the data from buffer to the current output stream.
Response.OutputStream.Write(buffer: buffer, offset: 0, count: length);
// Flush (Send) the data to output
// (Don't buffer in server's RAM!)
Response.Flush();
buffer = new Byte[intBufferSize];
dataToRead = dataToRead - length;
}
else
{
//prevent infinite loop if user disconnects
dataToRead = -1;
}
}
}
catch (Exception ex)
{
throw new ApplicationException(ex.Message);
}
finally
{
if (iStream != null)
{
//Close the file.
iStream.Close();
iStream = null;
}
Response.Close();
}
}
}
}
Everything work ok, but when I'm downloading a file, I cannot take other actions in my controller until the download process is finished.
How can I solve that?
I think the problem is that server is too busy to process file. And the Response gate of server is busy, too, so that, when client send a new request, it'll be pending.
I've solved my problem by putting the attribute [SessionState(SessionStateBehavior.ReadOnly)] on the controller.
It worked fine.