Recently I found a problem with printing out large quantities of text through Jsch channel.getInputStream().
using this code:
outputStream_from_outputChannel = channel.getInputStream();
br = new BufferedReader(new InputStreamReader(outputStream_from_outputChannel));
while ( (line = br.readLine()) != null ){
System.out.println(line);
numOfLines++;
}
...sometimes (most of the time) I will get results back, but sometimes it will stop producing the printout without returning, meaning the printing will stop in the middle and the thread will hang. What do you think that might be due to? So far I found out that is not the jsch session that is timout-ing. Its timeout is set to 0. Has anyone encountered this before?
You need to check first if the stream is ready\available:
byte[] tmp = new byte[1024];
while (outputStream_from_outputChannel.available() > 0) {
int i = is.read(tmp, 0, 1024);
if (i < 0)
break;
someStrBuff.append(new String(tmp, 0, i));
}
Sometimes reading from channel input stream hangs if remote server stops responding (due to reasons such as network connectivity, overload etc) but streams readline method blocks forever. Solution to the problem is to set server alive interval using :
session.setServerAliveInterval(int milliseconds);
It will send piggyback packets in order to check connectivity after specified time interval. And readline will return if no response is received from server when checking for connectivity.
Related
I have simple TCP/IP client code in Kotlin below.
This code works.
The client opens the socket and loops forever, first sending a message to the server, and then waiting forever for a response form the server.
I know this code isn’t great, looping forever, keeping the socket connection open etc., but it is just for test purposes right now.
fun tcp_client() {
thread {
val client1 = Socket(SERVER_IP_ADDRESS, SERVER_IP_PORT)
val output1 = PrintWriter(client1.getOutputStream(), true)
val input1 = BufferedReader(InputStreamReader(client1.inputStream))
while (true) {
output1.println(str_user_text)
str_rcvd_data = input1.readLine()
}
}
client1.close()
}
The line:
str_rcvd_data = input1.readLine()
waits forever for a server response.
My question: Is it possible to modify this code so that the client does NOT wait forvever for a server response? Something like this:
If (server data received) {
// process the data
} else {
// do something else for now and check again in a short while
}
Thanks in advance for any suggestions
Garrett
I eventually worked this out - I am not sure how 'correct' this solution is, but it works for me:
Connecting to the server....
My old code would hang if it couldn't connect, because the call to Socket() with the IP address and Port is a Blocking call - i.e.e wait forever:
val client1 = Socket(SERVER_IP_ADDRESS, SERVER_IP_PORT)
So I replaced the code with this:
try {
client1 = Socket()
client1.connect(InetSocketAddress(SERVER_IP_ADDRESS, SERVER_IP_PORT), 3000)
output1 = DataOutputStream (client1.getOutputStream())
input1 = DataInputStream (client1.getInputStream())
} catch (ex : Exception) {
// do something
} finally {
// do something
}
This isn't perfect, but it works.
For reading the data, my old code called readline() which is blocking:
str_rcvd_data = input1.readLine()
Now, my code first checks if there is any data and then grabs each byte
iRxDataAvailable = input1.available()
while (iRxDataAvailable > 0)
{
iRxDataAvailable--
// Take a copy of the received byte
byRcvdByte = input1.readByte()
// Do something with this data byte...
}
Finally, to send data to the server, the data is placed in a byte array, and then:
output1.write(byArray)
I'm writing a serial communication wrapper class in Objective-C. To list all serial available modems and setup the connection I'm using pretty much the same code as used in this example project by Apple.
I could read and write the ways apple does it. But I want to implement a loop on a second thread and write to the stream if a NSString *writeString longer 0 and read after write if bytes are available.
I got writing working quite straight forward. I just used the write function declared in unistd.h.
Reading will not work. Whenever I call read(), the function hangs and my loop does not proceed.
Here is the code used in my loop:
- (void)runInCOMLoop {
do {
// write
} while (bytesWritten < strlen([_writeString UTF8String]));
NSMutableString *readString = [NSMutableString string];
ssize_t bytesRead = 0;
ssize_t readB = 0;
char buffer[256];
do {
readB = read(_fileDescriptor, &buffer, sizeof(buffer));
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this function hangs
bytesRead += readB;
if (readB == -1 {
// error
}
else if (readB > 0) {
if(buffer[bytesRead - 1] == '\r' ]] buffer[bytesRead - 1] == '\n') {
break;
}
[readString appendString:[NSString stringWithUTF8String:buffer]];
}
} while (readB > 0);
What am I doing wrong here?
read() will block if there is nothing to read. Apple probably has their own of doing things, but you can use select() to see if there is anything to read on _fileDescriptor. Google around for examples on how to use select.
Here's one link on StackOverflow:
Can someone give me an example of how select() is alerted to an fd becoming "ready"
This excerpt from the select man is pertains:
To effect a poll, the timeout argument should be
non-nil, pointing to a zero-valued timeval structure. Timeout is not
changed by select(), and may be reused on subsequent calls, however it is
good style to re-initialize it before each invocation of select().
You can set the non-blocking flag (O_NONBLOCK) on the file descriptor using fcntl() to keep read() from waiting for data, but if you do that, you have to continuously poll looking for data, which is obviously bad from a CPU usage standpoint. As Charlie Burns' answer explains, the best solution is to use select() which will allow your program to efficiently wait until there is some data to be read on the port's file descriptor. Here's some example code taken from my own Objective-C serial port class, ORSSerialPort (slightly modified):
fd_set localReadFDSet;
FD_ZERO(&localReadFDSet);
FD_SET(self.fileDescriptor, &localReadFDSet);
timeout.tv_sec = 0;
timeout.tv_usec = 100000; // Check to see if port closed every 100ms
result = select(localPortFD+1, &localReadFDSet, NULL, NULL, &timeout);
if (!self.isOpen) break; // Port closed while select call was waiting
if (result < 0) {
// Handle error
}
if (result == 0 || !FD_ISSET(localPortFD, &localReadFDSet)) continue;
// Data is available
char buf[1024];
long lengthRead = read(localPortFD, buf, sizeof(buf));
NSData *readData = nil;
if (lengthRead>0) readData = [NSData dataWithBytes:buf length:lengthRead];
Note that select() indicates that data is available by returning. So, your program will sit suspended at the select() call while no data is available. The program is not hung, that's how it's supposed to work. If you need to do other things while select() is waiting, you should put the select() call on a different queue/thread from the other work you need to do. ORSSerialPort does this.
Is there a way to receive multiple message using a single synchronous call ?
When I know that there are N messages( N could be a small value less than 10) in the queue, then I should be able to do something like channel.basic_get(String queue, boolean autoAck , int numberofMsg ). I don't want to make multiple requests to the server .
RabbitMQ's basic.get doesn't support multiple messages unfortunately as seen in the docs. The preferred method to retrieve multiple messages is to use basic.consume which will push the messages to the client avoiding multiple round trips. acks are asynchronous so your client won't be waiting for the server to respond. basic.consume also has the benefit of allowing RabbitMQ to redeliver the message if the client disconnects, something that basic.get cannot do. This can be turned off as well setting no-ack to true.
Setting basic.qos prefetch-count will set the number of messages to push to the client at any time. If there isn't a message waiting on the client side (which would return immediately) client libraries tend to block with an optional timeout.
You can use a QueueingConsumer implementation of Consumer interface which allows you to retrieve several messages in a single request.
QueueingConsumer queueingConsumer = new QueueingConsumer(channel);
channel.basicConsume(plugin.getQueueName(), false, queueingConsumer);
for(int i = 0; i < 10; i++){
QueueingConsumer.Delivery delivery = queueingConsumer.nextDelivery(100);//read timeout in ms
if(delivery == null){
break;
}
}
Not an elegant solution and does not solve making multiple calls but you can use the MessageCount method. For example:
bool noAck = false;
var messageCount = channel.MessageCount("hello");
BasicGetResult result = null;
if (messageCount == 0)
{
// No messages available
}
else
{
while (messageCount > 0)
{
result = channel.BasicGet("hello", noAck);
var message = Encoding.UTF8.GetString(result.Body);
//process message .....
messageCount = channel.MessageCount("hello");
}
First declare instance of QueueingBasicConsumer() wich wraps the model.
From the model execute model.BasicConsume(QueueName, false, consumer)
Then implement a loop that will loop around messages from the queue which will then processing
Next line - consumer.Queue.Dequeue() method - waiting for the message to be received from the queue.
Then convert byte array to a string and display it.
Model.BasicAck() - release message from the queue to receive next message
And then on the server side can start waiting for the next message to come through:
public string GetMessagesByQueue(string QueueName)
{
var consumer = new QueueingBasicConsumer(_model);
_model.BasicConsume(QueueName, false, consumer);
string message = string.Empty;
while (Enabled)
{
//Get next message
var deliveryArgs = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
//Serialize message
message = Encoding.Default.GetString(deliveryArgs.Body);
_model.BasicAck(deliveryArgs.DeliveryTag, false);
}
return message;
}
If I'm connected to RabbitMQ and listening for events using an EventingBasicConsumer, how can I tell if I've been disconnected from the server?
I know there is a Shutdown event, but it doesn't fire if I unplug my network cable to simulate a failure.
I've also tried the ModelShutdown event, and CallbackException on the model but none seem to work.
EDIT-----
The one I marked as the answer is correct, but it was only part of the solution for me. There is also HeartBeat functionality built into RabbitMQ. The server specifies it in the configuration file. It defaults to 10 minutes but of course you can change that.
The client can also request a different interval for the heartbeat by setting the RequestedHeartbeat value on the ConnectionFactory instance.
I'm guessing that you're using the C# library? (but even so I think the others have a similar event).
You can do the following:
public class MyRabbitConsumer
{
private IConnection connection;
public void Connect()
{
connection = CreateAndOpenConnection();
connection.ConnectionShutdown += connection_ConnectionShutdown;
}
public IConnection CreateAndOpenConnection() { ... }
private void connection_ConnectionShutdown(IConnection connection, ShutdownEventArgs reason)
{
}
}
This is an example of it, but the marked answer is what lead me to this.
var factory = new ConnectionFactory
{
HostName = "MY_HOST_NAME",
UserName = "USERNAME",
Password = "PASSWORD",
RequestedHeartbeat = 30
};
using (var connection = factory.CreateConnection())
{
connection.ConnectionShutdown += (o, e) =>
{
//handle disconnect
};
using (var model = connection.CreateModel())
{
model.ExchangeDeclare(EXCHANGE_NAME, "topic");
var queueName = model.QueueDeclare();
model.QueueBind(queueName, EXCHANGE_NAME, "#");
var consumer = new QueueingBasicConsumer(model);
model.BasicConsume(queueName, true, consumer);
while (!stop)
{
BasicDeliverEventArgs args;
consumer.Queue.Dequeue(5000, out args);
if (stop) return;
if (args == null) continue;
if (args.Body.Length == 0) continue;
Task.Factory.StartNew(() =>
{
//Do work here on different thread then this one
}, TaskCreationOptions.PreferFairness);
}
}
}
A few things to note about this.
I'm using # for the topic. This grabs everything. Usually you want to limit by a topic.
I'm setting a variable called "stop" to determine when the process should end. You'll notice the loop runs forever until that variable is true.
The Dequeue waits 5 seconds then leaves without getting data if there is no new message. This is to ensure we listen for that stop variable and actually quit at some point. Change the value to your liking.
When a message comes in I spawn the handling code on a new thread. The current thread is being reserved for just listening to the rabbitmq messages and if a handler takes too long to process I don't want it slowing down the other messages. You may or may not need this depending on your implementation. Be careful however writing the code to handle the messages. If it takes a minute to run and your getting messages at sub-second times you will run out of memory or at least into severe performance issues.
I am finding the doc for CFStreamCreatePairWithSocketToCFHost confusing:
Specifically, its not clear to me how the function can set the readStream pointer to null on error.
as far as I understand, the pointer is passed by value - so the function can only change the objected pointed to by the pointer.
Right now I can't figure out how to detect connection errors.
Relevant doc snippet:
Creates readable and writable streams connected to a given CFHost object.
void CFStreamCreatePairWithSocketToCFHost (
CFAllocatorRef alloc,
CFHostRef host,
SInt32 port,
CFReadStreamRef *readStream,
CFWriteStreamRef *writeStream
);
readStream
Upon return, contains a CFReadStream object connected to the host host on port port, or NULL if there is a failure during creation. If you pass NULL, the function will not create a readable stream. Ownership follows the Create Rule.
This is my connecting code, it goes all the way to NSLog(#"Connected") even when the server is down.
NSLog(#"Attempting to (re)connect to %#:%d", m_host, m_port);
while(TRUE)
{
CFHostRef host = CFHostCreateWithName(kCFAllocatorDefault, (CFStringRef)m_host);
if (!host)
{
NSLog(#"Error resolving host %#", m_host);
[NSThread sleepForTimeInterval:5.0];
continue;
}
CFStreamCreatePairWithSocketToCFHost(kCFAllocatorDefault, host , m_port, &m_in, &m_out);
CFRelease(host);
if (!m_in)
{
NSLog(#"Error");
}
CFStreamClientContext context = {0, self,nil,nil,nil};
if (CFReadStreamSetClient(m_in, kCFStreamEventHasBytesAvailable | kCFStreamEventErrorOccurred | kCFStreamEventEndEncountered, networkReadEvent, &context))
{
CFReadStreamScheduleWithRunLoop(m_in, CFRunLoopGetCurrent(),kCFRunLoopCommonModes);
}
if (CFWriteStreamSetClient(m_out, kCFStreamEventErrorOccurred | kCFStreamEventEndEncountered, networkWriteEvent, &context))
{
CFWriteStreamScheduleWithRunLoop(m_out, CFRunLoopGetCurrent(),kCFRunLoopCommonModes);
}
BOOL success = CFReadStreamOpen(m_in);
CFErrorRef error = CFReadStreamCopyError(m_in);
if (!success || (error && CFErrorGetCode(error) != 0))
{
NSLog(#"Connect error %s : %d", CFErrorGetDomain(error), CFErrorGetCode(error));
[NSThread sleepForTimeInterval:5.0];
}
else
{
NSLog(#"Connected");
break;
}
}
From the "CFNetwork Programming Guide":
Opening a stream can be a lengthy process, so the CFReadStreamOpen and CFWriteStreamOpen functions avoid blocking by returning TRUE to
indicate that the process of opening the stream has begun. To check
the status of the open, call the functions CFReadStreamGetStatus and
CFWriteStreamGetStatus, which returnkCFStreamStatusOpening if the open
is still in progress, kCFStreamStatusOpen if the open is complete,
orkCFStreamStatusErrorOccurred if the open has completed but failed.
In most cases, it doesn’t matter whether the open is complete because
the CFStream functions that read and write will block until the stream
is open.
Also check out the kCFStreamEventOpenCompleted,
(http://developer.apple.com/library/ios/#documentation/CoreFoundation/Reference/CFStreamConstants/Reference/reference.html)
: a stream event that reports the successful completion of the opening
process. So to conclude, after calling CFReadStreamOpen (or Write),
which will probably succeed, register to listen to the "OpenCompleted"
event to identify a "real" success.
Surely after you call CFStreamCreatePairWithSocketToCFHost() just test readstream to see if it's NULL?
As you're passing in the memory location of the readstream pointer, the function can easily set that to whatever value it chooses (either a reference to a created object, or alternatively NULL).
Edit
I've tried your code, and I agree, it's very confusing. It appears that the CFReadStreamRef is readily created and opened, even for a nonsense host (I literally used "nonsense"). I don't believe this function will return NULL pointers for an unreachable host.
I suppose this makes sense, in as far as until one tries to open the stream, whether it will work or not is unknown.
So, the readStream param is a pointer to the CFReadStreamRef and, as such, can definitely be set to NULL by the function. &foo means "address of foo" and if you have the address you can set the value.
My reading of the documentation for CFStreamCreatePairWithSocketToCFHost is that they will be set to NULL on failure, but that failure is not about connection failure, but other kinds of failure (memory, etc). So not likely you'll get an error there.
Looks to me like the issue is that CFReadStreamOpen can return immediately with true when it can open the stream in the background and so this code is not really opening the stream or testing that it's been opened, merely queuing it for opening). From the documentation for CFReadStreamOpen:
" If the stream can open in the background without blocking, this function always returns true."
So I think you will need to follow the rest of the instructions for CFReadStreamOpen and schedule the stream on a run loop, or perhaps poll (though obviously polling in a tight loop isn't likely to work).
In the documentation for CFReadStreamOpen we see:
Opening a stream causes it to reserve all the system resources it requires. If the stream can open in the background without blocking, this function always returns true.
I suspect that the stream is opening in the background, and thus you are saying "Connected" before it actually opens. You've already scheduled the stream with a runloop, so if you let the run loop run, you'll probably get a callback with the event type set to kCFStreamEventErrorOccurred, and from there you can process the error appropriately.