I know this question has already been asked, but the answer given to that person was to use read(), until a EOF. I applied this answer and my server still sends "Connection reset". For those that are not familiar with the problem, I'm trying to connect a GPS that is install in my car to my server. It works with SMS and GPRS commands, so the idea is to connect the device succesfully. Here's the code:
HERE'S MY CLIENT SOCKET MANAGER
package tcp.clientservermodel;
import java.net.*;
import java.io.*;
public class ClientSocketManager extends Thread{
Socket client_socket_;
InputStream input_;
OutputStream output_;
ClientSocketCaller caller_;
public ClientSocketManager( ClientSocketCaller caller,Socket incoming_socket){
this.caller_=caller;
this.client_socket_=incoming_socket;
this.start();
}
public void run(){
try{
input_=this.client_socket_.getInputStream();
output_=this.client_socket_.getOutputStream();
int incoming_char=0;
StringBuffer incoming_message=new StringBuffer();
while((incoming_char=input_.read())!=-1){
// System.out.println(incoming_message.toString());
if(incoming_char=='\n'){ ProcessMessage(this. client_socket_.getInetAddress().toString()+": "+incoming_message.toString());
incoming_message=new StringBuffer();
}else{
incoming_message.append((char)incoming_char);
}
}
}catch(Exception err){
}
}
public boolean SendMessage(String message){
boolean success=true;
try{
this.output_.write(("123456G").getBytes());
}catch(Exception err){
success=false;
}
return success;
}
public void ProcessMessage(String message){
System.out.println(message);
caller_.MessageHasBeenRecieved(message);
}
}
SERVER SOCKET MANAGER:
package tcp.clientservermodel;
import java.net.*;
import java.io.*;
import java.util.*;
public class ServerSocketManager extends Thread
implements ClientSocketCaller{
int port_;
ServerSocket server_socket_;
public boolean is_enabled_=true;
Vector client_sockets_vector_=new Vector();
public ServerSocketManager(int port){
this.port_=port;
this.start();
}
public void run(){
try{
this.server_socket_=new ServerSocket(port_);
while(is_enabled_){
Socket client_socket=this.server_socket_.accept();
System.out.println("Connection Succesful "+client_socket.getInetAddress().toString());
this.client_sockets_vector_.add(new ClientSocketManager(this,client_socket));
}
}catch(Exception err){
System.out.println(err +" 3");
}
}
public void MessageHasBeenRecieved(String message) {
for(int i=0;i<this.client_sockets_vector_.size();i++){
ClientSocketManager current=(ClientSocketManager)this.client_sockets_vector_.get(i);
if(current!=null){
current.SendMessage(message);
}
}
}
}
HERE'S IS THE CLIENT SOCKET CALLER
package tcp.clientservermodel;
public interface ClientSocketCaller {
public void MessageHasBeenRecieved(String message);
}
The error is located in this line
(incoming_char=input_.read())!=-1
and output is
java.net.SocketException connection reset
PLEASE I NEED HELP!
I solved the problem...I used Wireshark to see the packages between the server and the GPS. So I notice that the size of the "win" parameter in the Wireshark were different between both streams. From the GPS => win = 10880, and from the server => win = 8019. I just change the win element of the server and worked!!!!
Related
I am having an issue with keystore, truststore, and LDAP credentials for Google.
I am using GraalVM 22.2 on JDK17 for the test.
I created a test class to be able to log into Google LDAP. Once, I figure out how to do it in the test, then I can move it into our application.
I downloaded my Google LDAP credential and key. I then followed the instructions from Google to create a KeyStore:
https://support.google.com/a/answer/9089736?hl=en#other-java-applications
According to Google's instructions, you just add the following to your System properties:
-Djavax.net.ssl.keyStore=[File Path to KeyStore] -Djavax.net.ssl.keyStorePassword=[KeyStore Pass]
-- OR --
System.setProperty("javax.net.ssl.keyStore", "[File Path to KeyStore]");
System.setProperty("javax.net.ssl.keyStorePassword", "[KeyStore Pass]");
The problem with this is we will have a KeyStore with a password that corresponds with a specific LDAP server. This is due to the fact that our application will be able to connect to multiple LDAP servers.
After doing some research, I saw you can create an SSLSocketFactory and set the property "java.naming.ldap.factory.socket". From there, you can add the KeyStore to use via the TrustManager, or KeyManager.
My problem is, when trying to connect to the Google LDAP server; the SSLSocketFactory is completely ignored. How do I know? None of the debugging statements in the SSLSocketFactory, TrustManger, or KeyManager are printed out. (I added a KeyStoreManager for debugging purposes.)
I even tried to import the keystore into cacerts with no luck. In fact, I had multiple problems importing because "keytool" kept complaining about the JKS vs PKCS12 format.
The only way I can connect successfully is to add the 2 System properties. I know I am missing something simple that will explain why it is not working correctly. Anyone got any ideas?
Here is my test class:
import java.io.Console;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.InetAddress;
import java.net.Socket;
import java.security.KeyStore;
import java.security.Principal;
import java.security.PrivateKey;
import java.security.SecureRandom;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.Properties;
import javax.naming.Context;
import javax.naming.directory.InitialDirContext;
import javax.net.ssl.KeyManager;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSocketFactory;
import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.X509KeyManager;
import javax.net.ssl.X509TrustManager;
public class LDAPIT {
public static final String JAVA_NAMING_LDAP_FACTORY_SOCKET = "java.naming.ldap.factory.socket";
public static void main(final String[] args) {
final LDAPIT test = new LDAPIT();
try {
test.run();
} catch (final Exception e) {
throw new RuntimeException(e);
}
}
public void run() throws Exception {
final Console cnsl = System.console();
if (cnsl == null) {
throw new RuntimeException("Console is null");
}
final String url = cnsl.readLine("Enter URL: ");
final String user = cnsl.readLine("Enter username : ");
final String password = String.valueOf(cnsl.readPassword("Enter password : "));
System.out.println("Start Trying to Connect to LDAP: " + url);
System.out.println("With User: " + user);
System.out.println("Creating LDAP Properties");
final Properties props = new Properties();
props.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
props.put(Context.PROVIDER_URL, url);
props.put(Context.SECURITY_PRINCIPAL, user);
props.put(Context.SECURITY_AUTHENTICATION, "simple");
props.put(Context.SECURITY_PROTOCOL, "ssl");
props.put(JAVA_NAMING_LDAP_FACTORY_SOCKET, MySSLSocketFactory.class.getName());
System.out.println("LDAP Properties without password: " + props);
props.put(Context.SECURITY_CREDENTIALS, password);
// System.setProperty("javax.net.ssl.keyStore", "[File Path to KeyStore]");
// System.setProperty("javax.net.ssl.keyStorePassword", "[KeyStore Pass]");
try {
System.out.println("Connecting to LDAP Server");
final InitialDirContext idc = new InitialDirContext(props);
System.out.println("Successfully connected to LDAP");
idc.close();
} catch (final Exception e) {
e.printStackTrace();
}
System.out.println("End Trying to Connect to LDAP");
}
public static class MySSLSocketFactory extends SSLSocketFactory {
private SSLSocketFactory socketFactory;
public MySSLSocketFactory() {
try {
System.out.println("Creating Key Manager");
final MyKeyManager keyManager = new MyKeyManager();
final KeyManager[] keyManagers = {keyManager};
System.out.println("Creating Trust Manager");
final MyTrustManager trustManager = new MyTrustManager();
final TrustManager[] trustManagers = {trustManager};
System.out.println("Creating Socket Context");
final SSLContext ctx = SSLContext.getInstance("TLS");
System.out.println("Initializing Socket Context");
ctx.init(keyManagers, trustManagers, new SecureRandom());
System.out.println("Creating Socket Factory");
socketFactory = ctx.getSocketFactory();
} catch (final Exception ex) {
ex.printStackTrace();
}
}
#Override
public String[] getDefaultCipherSuites() {
return socketFactory.getDefaultCipherSuites();
}
#Override
public String[] getSupportedCipherSuites() {
return socketFactory.getSupportedCipherSuites();
}
#Override
public Socket createSocket(Socket socket, String string, int i, boolean bln) throws IOException {
System.out.println("Creating Socket with: Socket, String, int, Boolean");
return socketFactory.createSocket(socket, string, i, bln);
}
#Override
public Socket createSocket(final Socket s, final InputStream consumed, final boolean autoClose) throws IOException {
System.out.println("Creating Socket with: Socket, InputStream, Boolean");
return super.createSocket(s, consumed, autoClose);
}
#Override
public Socket createSocket() throws IOException {
System.out.println("Creating Socket without parameters");
return super.createSocket();
}
#Override
public Socket createSocket(String string, int i) throws IOException {
System.out.println("Creating Socket with: String ,int");
return socketFactory.createSocket(string, i);
}
#Override
public Socket createSocket(String string, int i, InetAddress ia, int i1) throws IOException {
System.out.println("Creating Socket with: String, int, InetAddress, int");
return socketFactory.createSocket(string, i, ia, i1);
}
#Override
public Socket createSocket(InetAddress ia, int i) throws IOException {
System.out.println("Creating Socket with: InetAddress, int");
return socketFactory.createSocket(ia, i);
}
#Override
public Socket createSocket(InetAddress ia, int i, InetAddress ia1, int i1) throws IOException {
System.out.println("Creating Socket with: InetAddress, InetAddress, int");
return socketFactory.createSocket(ia, i, ia1, i1);
}
}
public static class MyKeyManager implements X509KeyManager {
#Override
public String[] getClientAliases(final String keyType, final Principal[] issuers) {
System.out.println("Getting Client Aliases: String, Principal[]");
return new String[0];
}
#Override
public String chooseClientAlias(final String[] keyType, final Principal[] issuers, final Socket socket) {
System.out.println("Choosing Client Alias: String[], Principal[], Socket");
return null;
}
#Override
public String[] getServerAliases(final String keyType, final Principal[] issuers) {
System.out.println("Getting Server Aliases: String, Principal[]");
return new String[0];
}
#Override
public String chooseServerAlias(final String keyType, final Principal[] issuers, final Socket socket) {
System.out.println("Choosing Server Aliases: String, Principal[], Socket");
return null;
}
#Override
public X509Certificate[] getCertificateChain(final String alias) {
System.out.println("Getting Certificate Chain: String");
return new X509Certificate[0];
}
#Override
public PrivateKey getPrivateKey(final String alias) {
System.out.println("Getting Private Key: String");
return null;
}
}
public static class MyTrustManager implements X509TrustManager {
private X509TrustManager gTrustManager;
public MyTrustManager() {
System.out.println("Creating My Trust Manager");
initTrustManager();
System.out.println("Created My Trust Manager");
}
#Override
public void checkClientTrusted(final X509Certificate[] chain, final String authType) throws CertificateException {
if (gTrustManager != null) {
gTrustManager.checkClientTrusted(chain, authType);
}
}
#Override
public void checkServerTrusted(final X509Certificate[] chain, final String authType) throws CertificateException {
if (gTrustManager != null) {
gTrustManager.checkServerTrusted(chain, authType);
}
}
#Override
public X509Certificate[] getAcceptedIssuers() {
return gTrustManager.getAcceptedIssuers();
}
private void initTrustManager() {
try {
final String keystoreType = KeyStore.getDefaultType();
System.out.println("Loading Keystore Type: " + keystoreType);
KeyStore keyStore = KeyStore.getInstance(keystoreType);
System.out.println("Loading CA Certs");
loadCACerts(keyStore);
System.out.println("Creating SSL Context");
System.out.println("Creating Trust Manager");
gTrustManager = getTrustManager(keyStore);
} catch (final Exception e) {
e.printStackTrace();
}
}
private void loadCACerts(final KeyStore keyStore) throws Exception {
final File keyStoreFile = new File("[File Path to KeyStore]]");
System.out.println("Loading Keystore: " + keyStoreFile.getAbsolutePath());
try (final FileInputStream keyStoreInputStream = new FileInputStream(keyStoreFile)) {
keyStore.load(keyStoreInputStream, "[KeyStore Pass]".toCharArray());
}
System.out.println("Keystore Loaded");
}
private X509TrustManager getTrustManager(final KeyStore k) throws Exception {
final String defaultAlgorithm = TrustManagerFactory.getDefaultAlgorithm();
System.out.println("Loading Trust Manager Algorithm: " + defaultAlgorithm);
final TrustManagerFactory tmf = TrustManagerFactory.getInstance(defaultAlgorithm);
System.out.println("Initializing Trust Manager Factory");
tmf.init(k);
System.out.println("Getting Trust Managers");
final TrustManager[] tms = tmf.getTrustManagers();
int arraySize = 0;
if (tms != null) {
arraySize = tms.length;
}
System.out.println("Iterating through Trust Managers: " + arraySize);
for (final TrustManager tm : tms) {
System.out.println("Trust Manager: " + tm.getClass().getName());
if (tm instanceof final X509TrustManager x509TrustManager) {
System.out.println("Returning X509TrustManager");
return x509TrustManager;
}
}
System.out.println("Could not find correct Trust Manager, returning Null");
return null;
}
}
}
Wouldn't you know it...after I ask the question I figure it out. The answer was you MUST implement the 'getDefault()' method in your custom SSLSocketFactory. I knew it was something simple.
https://docs.oracle.com/javase/jndi/tutorial/ldap/security/ssl.html
Look at Using Custom Sockets.
I am writing an Application which communicates with an external REST-API using Apache HttpClient.
The tasks it performs include some lengthy PUT-Operations and my application should automatically retry to continue the operation if it has failed, because someone tripped over the network cable or the like.
I'd like to write a test for this behaviour and my idea is to open a mock connection which will consume X bytes and then throw an IOException. To be able to do so I'll inject the HTTPClient into my system and thus I can inject a preconfigured HTTPClient into my system for testing which will show the desired behaviour. The HTTPClient has so many abstractions, factories and so on that I'll believe that this is probably possible, but I tend to get totally lost in all the depths.
I sucessfully put of to actually submit that question until I eventually managed to get it working myself (took me long enough) :-).
Pretty much EYERYTHING is somehow abstracted away in this library and like EVERYTHING can be configured. Except Sockets. The only thing which seems to lack any abstraction in apache http client. It took me forever to try to get the lib work just on streams and not on Sockets until I finally gave up and implemented my own "sockets".
But then it was quite simple: In the HttpClientBuilder a connectionManager can be set which instantiates sockets and connects them. You can implement your own connection manager and return a socket with an overriden getOutputStream/getInputStream method and then you can work with those streams that you created. Don't forget to also override the connect method, because we don't want to create any network activity here.
Here is an example TestNG-Test which demonstrates the behavior and on which you can surely built upon.
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.net.Socket;
import org.apache.http.HttpHost;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.conn.BasicHttpClientConnectionManager;
import org.apache.http.protocol.HttpContext;
import org.testng.annotations.Test;
public class InterruptedConnectionTest {
private static final InputStream EMPTY_INPUT_STREAM = new InputStream() {
#Override
public int read() throws IOException {
return -1;
}
};
private static final ConnectionSocketFactory FAILURE_SOCKET_FACTORY = new ConnectionSocketFactory() {
#Override
public Socket createSocket(HttpContext context) throws IOException {
final InputStream in = EMPTY_INPUT_STREAM;
final OutputStream out = new FailureOutputStream(10);
return new Socket() {
#Override
public InputStream getInputStream() throws IOException {
return in;
}
#Override
public OutputStream getOutputStream() throws IOException {
return out;
}
};
}
#Override
public Socket connectSocket(int connectTimeout, Socket sock,
HttpHost host, InetSocketAddress remoteAddress,
InetSocketAddress localAddress, HttpContext context)
throws IOException {
return sock; // do nothing
}
};
#Test(expectedExceptions = MockIOException.class)
public void executingRequest_throwsException() throws Exception {
HttpClient httpClient = HttpClientBuilder
.create()
.setConnectionManager(new BasicHttpClientConnectionManager(RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", FAILURE_SOCKET_FACTORY)
.build()))
.build();
httpClient.execute(new HttpGet("http://localhost/some/path"));
}
private static class FailureOutputStream extends OutputStream {
private int bytesRead = 0;
private final int failByte;
private FailureOutputStream(int failByte) {
super();
this.failByte = failByte;
}
#Override
public void write(int b) throws IOException {
++bytesRead;
if (bytesRead >= failByte) {
throw new MockIOException("Mock error after having having read <" + bytesRead + "> bytes");
}
}
}
private static class MockIOException extends IOException {
public MockIOException(String message) {
super(message);
}
}
}
I am trying to integrate the Apache FTP server into my application. I have followed the instructions given here but have run into some problems. Currently I am able to run the server and connect to it from my browser but can not log in. I have tried admin/admin and anonymous/*, but the login fails every time. In the apache-ftpserver-1.0.6 source code I had downloaded, the files associated with the user manager are located in res/conf, although when I try to match that file path in my own program I get an error that says "invalid resource directory name" and am unable to build. I also tried including the files users.properties and ftpd-typical.xml in the main directly and can run, but again cannot log in. It seems like my project does not realize these files are present.
Does anyone have experience with Apache FTP Server that could tell me the correct way to include these files so that I can log in to my server?
Thanks!
P.S. I don't think it should make any difference, but I am developing this program for Android.
In the following code I am crating admin user and non-admin user, setting restrictions of reading, writing and restricting throttling and upload rate limit and imposing download rate limiting.
Added a listener to listen user login and logout download start and download finish events.
import java.io.File;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import org.apache.ftpserver.FtpServer;
import org.apache.ftpserver.FtpServerFactory;
import org.apache.ftpserver.ftplet.Authority;
import org.apache.ftpserver.ftplet.FileSystemFactory;
import org.apache.ftpserver.ftplet.FtpException;
import org.apache.ftpserver.ftplet.Ftplet;
import org.apache.ftpserver.ftplet.UserManager;
import org.apache.ftpserver.ftpletcontainer.impl.DefaultFtpletContainer;
import org.apache.ftpserver.listener.ListenerFactory;
import org.apache.ftpserver.usermanager.PropertiesUserManagerFactory;
import org.apache.ftpserver.usermanager.SaltedPasswordEncryptor;
import org.apache.ftpserver.usermanager.impl.BaseUser;
import org.apache.ftpserver.usermanager.impl.ConcurrentLoginPermission;
import org.apache.ftpserver.usermanager.impl.TransferRatePermission;
import org.apache.ftpserver.usermanager.impl.WritePermission;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class SFTPServer {
// ===========================================================
// Constants
// ===========================================================
private final int FTP_PORT = 2221;
private final String DEFAULT_LISTENER = "default";
// private final Logger LOG = LoggerFactory.getLogger(SFTPServer.class);
private static final List<Authority> ADMIN_AUTHORITIES;
private static final int BYTES_PER_KB = 1024;
private static final String DEFAULT_USER_DIR = "C:\\upload";
public final static int MAX_CONCURRENT_LOGINS = 1;
public final static int MAX_CONCURRENT_LOGINS_PER_IP = 1;
// ===========================================================
// Fields
// ===========================================================
private static FtpServer mFTPServer;
private static UserManager mUserManager;
private static FtpServerFactory mFTPServerFactory;
private ListenerFactory mListenerFactor;
// ===========================================================
// Constructors
// ===========================================================
static {
// Admin Authorities
ADMIN_AUTHORITIES = new ArrayList<Authority>();
ADMIN_AUTHORITIES.add(new WritePermission());
ADMIN_AUTHORITIES.add(new ConcurrentLoginPermission(MAX_CONCURRENT_LOGINS, MAX_CONCURRENT_LOGINS_PER_IP));
ADMIN_AUTHORITIES.add(new TransferRatePermission(Integer.MAX_VALUE, Integer.MAX_VALUE));
}
// ===========================================================
// Getter & Setter
// ===========================================================
// ===========================================================
// Methods for/from SuperClass/Interfaces
// ===========================================================
// ===========================================================
// Methods
// ===========================================================
public void init() throws FtpException {
mFTPServerFactory = new FtpServerFactory();
mListenerFactor = new ListenerFactory();
mListenerFactor.setPort(FTP_PORT);
mFTPServerFactory.addListener(DEFAULT_LISTENER, mListenerFactor.createListener());
mFTPServerFactory.getFtplets().put(FTPLetImpl.class.getName(), new FTPLetImpl());
PropertiesUserManagerFactory userManagerFactory = new PropertiesUserManagerFactory();
userManagerFactory.setFile(new File("ftpusers.properties"));
userManagerFactory.setPasswordEncryptor(new SaltedPasswordEncryptor());
mUserManager = userManagerFactory.createUserManager();
mFTPServerFactory.setUserManager(mUserManager);
this.createAdminUser();
SFTPServer.addUser("admin1", "admin1", 20, 20);
mFTPServer = mFTPServerFactory.createServer();
mFTPServer.start();
}
private UserManager createAdminUser() throws FtpException {
UserManager userManager = mFTPServerFactory.getUserManager();
String adminName = userManager.getAdminName();
if (!userManager.doesExist(adminName)) {
// LOG.info((new
// StringBuilder()).append("Creating user : ").append(adminName).toString());
BaseUser adminUser = new BaseUser();
adminUser.setName(adminName);
adminUser.setPassword(adminName);
adminUser.setEnabled(true);
adminUser.setAuthorities(ADMIN_AUTHORITIES);
adminUser.setHomeDirectory(DEFAULT_USER_DIR);
adminUser.setMaxIdleTime(0);
userManager.save(adminUser);
}
return userManager;
}
public static void addUser(String username, String password, int uploadRateKB, int downloadRateKB) throws FtpException {
BaseUser user = new BaseUser();
user.setName(username);
user.setPassword(password);
user.setHomeDirectory(DEFAULT_USER_DIR);
user.setEnabled(true);
List<Authority> list = new ArrayList<Authority>();
list.add(new TransferRatePermission(downloadRateKB * BYTES_PER_KB, uploadRateKB * BYTES_PER_KB)); // 20KB
list.add(new ConcurrentLoginPermission(MAX_CONCURRENT_LOGINS, MAX_CONCURRENT_LOGINS_PER_IP));
user.setAuthorities(list);
mFTPServerFactory.getUserManager().save(user);
}
public static void restartFTP() throws FtpException {
if (mFTPServer != null) {
mFTPServer.stop();
try {
Thread.sleep(1000 * 3);
} catch (InterruptedException e) {
}
mFTPServer.start();
}
}
public static void stopFTP() throws FtpException {
if (mFTPServer != null) {
mFTPServer.stop();
}
}
public static void pauseFTP() throws FtpException {
if (mFTPServer != null) {
mFTPServer.suspend();
}
}
public static void resumeFTP() throws FtpException {
if (mFTPServer != null) {
mFTPServer.resume();
}
}
public static void main(String... are) {
try {
new SFTPServer().init();
} catch (FtpException e) {
e.printStackTrace();
}
}
// ===========================================================
// Inner and Anonymous Classes
// ===========================================================
}
FTPLET Listener
import java.io.IOException;
import org.apache.ftpserver.ftplet.DefaultFtplet;
import org.apache.ftpserver.ftplet.FtpException;
import org.apache.ftpserver.ftplet.FtpRequest;
import org.apache.ftpserver.ftplet.FtpSession;
import org.apache.ftpserver.ftplet.FtpletResult;
public class FTPLetImpl extends DefaultFtplet {
#Override
public FtpletResult onLogin(FtpSession session, FtpRequest request) throws FtpException, IOException {
System.out.println(session.getUser().getName() + " Logged in");
return super.onLogin(session, request);
}
#Override
public FtpletResult onDisconnect(FtpSession session) throws FtpException, IOException {
System.out.println(session.getUser().getName() + " Disconnected");
return super.onDisconnect(session);
}
#Override
public FtpletResult onDownloadStart(FtpSession session, FtpRequest request) throws FtpException, IOException {
System.out.println(session.getUser().getName() + " Started Downloading File " + request.getArgument());
return super.onDownloadStart(session, request);
}
#Override
public FtpletResult onDownloadEnd(FtpSession session, FtpRequest request) throws FtpException, IOException {
System.out.println("Finished Downloading " + request.getArgument());
return super.onDownloadEnd(session, request);
}
}
The setup
Windows 7 Professionnal
Eclipse Juno
Java jre7
Netty 4.0.0 Beta2
I have a netty server running on another machine. Then I have a program running on my machine, which is made to simulate many clients communicating with the server concurrently. In order to do that, I have a thread pool implemented with java.util.concurrent.ExecutorService . Each client creates a thread and submit it to the ExecutorService. Just before it ends, that thread creates another one with the same code. The submited code does those steps :
connect to server by sending a handshake (netty bootstrap A and channel A)
get the token from the handshake response
connect to server (netty bootstrap B and channel B)
send one request to server
receive the response
close the connection
create another thread with the same code
The problem
I sometimes get a NullPointerException in NettySocketCommunication.sendMessage(), on channel.write(byteBuf) when sending a request to the server.
01728 16:25:23.870 [nioEventLoopGroup-3804-2] ERROR
c.f.s.virtualuser.VirtualUser - java.lang.RuntimeException:
java.lang.NullPointerException at
c.f.s.virtualuser.VirtualUser.processMessageStep(VirtualUser.java:324)
at
c.f.s.virtualuser.VirtualUser.processNextStep(VirtualUser.java:252)
at
c.f.s.virtualuser.VirtualUser.onChannelConnected(VirtualUser.java:395)
at
c.f.s.c.m.handler.ClientSocketBasedHandler.channelActive(ClientSocketBasedHandler.java:95)
at
io.netty.channel.DefaultChannelHandlerContext.invokeChannelActive(DefaultChannelHandlerContext.java:774)
at
io.netty.channel.DefaultChannelHandlerContext.fireChannelActive(DefaultChannelHandlerContext.java:760)
at
io.netty.channel.ChannelStateHandlerAdapter.channelActive(ChannelStateHandlerAdapter.java:58)
at
io.netty.channel.DefaultChannelHandlerContext.invokeChannelActive(DefaultChannelHandlerContext.java:774)
at
io.netty.channel.DefaultChannelHandlerContext.fireChannelActive(DefaultChannelHandlerContext.java:760)
at
io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:884)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:223)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:417)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:365)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:302) at
io.netty.channel.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110)
at java.lang.Thread.run(Unknown Source)
Caused by:
java.lang.NullPointerException: null at
c.f.s.c.m.NettySocketCommunication.sendMessage(NettySocketCommunication.java:109)
at
c.f.s.virtualuser.VirtualUser.processMessageStep(VirtualUser.java:317)
... 15 common frames omitted
The code
I removed some logging and comments to make the code blocks shorter. I also have the VirtualUser.java class (not shown here) that implements both IVirtualUserCommunication and IVirtualUserMessages interfaces.
[AbstractVirtualUserCommunication.java]
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelHandler;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundMessageHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.http.DefaultLastHttpContent;
import io.netty.handler.codec.http.HttpRequestEncoder;
import io.netty.handler.codec.http.HttpResponseDecoder;
import io.netty.util.CharsetUtil;
import [...].Amf3;
import [...].Properties;
import [...].LogFactory;
import [...].Logger;
import [...].AbstractRequest;
import [...].IRouterMessage;
import [...].RouterMessage;
import [...].FrimaHandshake;
import [...].IVirtualUserCommunication;
public abstract class AbstractVirtualUserCommunication implements IVirtualUserCommunication
{
protected static Logger Log = LogFactory.getInstance().getLogger(AbstractVirtualUserCommunication.class);
protected String CONFIG_APPLICATION = "target.server.application";
protected String application;
protected String CONFIG_VERSION = "target.server.version";
protected String version;
protected String CONFIG_HANDSHAKE_PORT = "netty.handshake.port";
protected final int defaultHandshakePort = 80;
// The following variables are used by both HTTP and SOCKET communication
protected Bootstrap bootstrapHandshake; // Netty bootstrap used only for handshake
protected Channel channelHandshake; // Netty channel used only for handshake
protected String token; // The token received through handshake process
// Host & port are set in the connect() method
protected String host;
protected int port;
protected Bootstrap bootstrap; // Netty bootstrap used for communication
protected Channel channel; // Netty channel used for communication
/** Connect to the server to get the token */
public void sendHandshake(String host)
{
// Get properties, with default values if they are not specified
this.application = Properties.getString(CONFIG_APPLICATION, "snowstorm");
this.version = Properties.getString(CONFIG_VERSION, "0.0.1");
int handshakePort = Properties.getInt(CONFIG_HANDSHAKE_PORT, defaultHandshakePort);
bootstrapHandshake = new Bootstrap();
try
{
bootstrapHandshake.group(new NioEventLoopGroup());
bootstrapHandshake.channel(NioSocketChannel.class);
bootstrapHandshake.handler(new HandShakeInitializer(/* this */));
// Connect and listen on handshake host/port
channelHandshake = bootstrapHandshake.connect(host, handshakePort).sync().channel();
channelHandshake.closeFuture().sync();
}
catch (InterruptedException e)
{
Log.error(e);
}
finally
{
bootstrapHandshake.shutdown();
}
}
/** Method called after completion of the handshake (the token has been set). */
protected abstract void afterHandshake();
/** Connect to the target server for stress test script execution. */
protected void connect(ChannelHandler handler)
{
bootstrap = new Bootstrap();
try
{
// Initialize the pipeline
bootstrap.group(new NioEventLoopGroup());
bootstrap.channel(NioSocketChannel.class);
bootstrap.handler(handler);
// Connect and listen on host/port
channel = bootstrap.connect(host, port).sync().channel();
if (channel == null)
{
Log.error("PROBLEM : The channel is null in the afterHandshake() method");
}
channel.closeFuture().sync();
}
catch (InterruptedException e)
{
Log.error(e);
}
finally
{
bootstrap.shutdown();
}
}
/** Create a RouterMessage with the specified request. */
protected IRouterMessage buildMessage(AbstractRequest request)
{
RouterMessage routerMessage = new RouterMessage();
routerMessage.bytes = Amf3.serialize(request);
routerMessage.token = this.token;
routerMessage.application = this.application;
routerMessage.version = this.version;
return routerMessage;
}
#Override
public void disconnect()
{
// TODO Is it dangerous to not call channel.close() ??
if (channel != null)
{
channel.close().awaitUninterruptibly();
}
else
{
Log.error("PROBLEM : The channel is null when calling the disconnect() method");
}
bootstrap.shutdown();
}
#Override
public boolean isConnected()
{
if (channel == null)
{
return false;
}
return channel.isActive();
}
private class HandShakeInitializer extends ChannelInitializer<SocketChannel>
{
public HandShakeInitializer()
{
super();
}
#Override
protected void initChannel(SocketChannel socketChannel) throws Exception
{
socketChannel.pipeline().addLast("encoder", new HttpRequestEncoder());
socketChannel.pipeline().addLast("decoder", new HttpResponseDecoder());
socketChannel.pipeline().addLast("handler", new HandShakeHandler(/* communication */));
}
}
private class HandShakeHandler extends ChannelInboundMessageHandlerAdapter<Object>
{
public HandShakeHandler()
{
super();
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception
{
super.channelActive(ctx);
ctx.write(FrimaHandshake.create(null, version, application));
ctx.flush();
}
#Override
public void messageReceived(ChannelHandlerContext ctx, Object msg) throws Exception
{
if (msg instanceof DefaultLastHttpContent)
{
DefaultLastHttpContent defaultLastHttpContent = (DefaultLastHttpContent) msg;
String content = defaultLastHttpContent.data().toString(CharsetUtil.UTF_8);
// Format = token~publicDNS and we only need the token here
token = content;// .split("~")[0];
Log.debug("Starting a bot with token " + token);
afterHandshake();
}
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception
{
Log.error(cause);
ctx.close();
}
}
}
[NettySocketCommunication.java]
import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelFuture;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioSocketChannel;
import [...].AbstractRequest;
import [...].IRouterMessage;
import [...].Serializer;
import [...].ClientSocketBasedHandler;
import [...].ClientSocketBasedInitializer;
import [...].IVirtualUserMessages;
public class NettySocketCommunication extends AbstractVirtualUserCommunication
{
private ClientSocketBasedHandler handler;
private ChannelFuture testChannelFuture;
public NettySocketCommunication()
{
super();
Log.setLevelToInfo();
this.handler = new ClientSocketBasedHandler();
}
#Override
public void setVirtualUser(IVirtualUserMessages virtualUser)
{
this.handler.setVirtualUser(virtualUser);
}
#Override
public void connect(String host, int port)
{
this.host = host;
this.port = port;
// Get the token from the server through the handshake process
sendHandshake(host);
}
#Override
public boolean connectTest(String host, int port)
{
boolean connectSuccess = false;
bootstrap = new Bootstrap();
// Initialize the pipeline
bootstrap.group(new NioEventLoopGroup());
bootstrap.channel(NioSocketChannel.class);
bootstrap.handler(new ClientSocketBasedInitializer(new ClientSocketBasedHandler()));
// Listen on host/port (connect a channel)
testChannelFuture = bootstrap.connect(host, port);
testChannelFuture.awaitUninterruptibly();
if (testChannelFuture.isSuccess())
{
connectSuccess = true;
}
testChannelFuture.channel().close().awaitUninterruptibly();
bootstrap.shutdown();
return connectSuccess;
}
#Override
protected void afterHandshake()
{
super.connect(new ClientSocketBasedInitializer(handler));
}
#Override
public void sendMessage(AbstractRequest request)
{
IRouterMessage routerMessage = buildMessage(request);
ByteBuf byteBuf = Serializer.encode(routerMessage, true);
// Send message
channel.write(byteBuf);
channel.flush();
}
}
[ClientSocketBasedHandler.java]
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundMessageHandlerAdapter;
import java.util.ArrayList;
import java.util.List;
import [...].Amf3;
import [...].LogFactory;
import [...].Logger;
import [...].IMessage;
import [...].IRouterMessage;
import [...].IVirtualUserMessages;
public class ClientSocketBasedHandler extends ChannelInboundMessageHandlerAdapter<IRouterMessage>
{
protected static Logger Log = LogFactory.getInstance().getLogger(ClientSocketBasedHandler.class);
private IVirtualUserMessages virtualUser;
public ClientSocketBasedHandler()
{
super();
Log.setLevelToInfo();
}
public void setVirtualUser(IVirtualUserMessages virtualUser)
{
this.virtualUser = virtualUser;
}
#Override
public void messageReceived(ChannelHandlerContext ctx, IRouterMessage routerMessage) throws Exception
{
List<IMessage> messages = deserializeMessages(routerMessage.getBytes());
for (IMessage message : messages)
{
Log.debug("Received socket : " + message);
if (virtualUser == null)
{
throw new RuntimeException("Must call the setVirtualUser() method before receiving messages");
}
virtualUser.onManticoreMessageReceived(message);
}
}
protected List<IMessage> deserializeMessages(byte[] bytes)
{
Object probablyMessages = Amf3.deserialize(bytes);
List<IMessage> messages = null;
// List of Messages
if (probablyMessages instanceof ArrayList)
{
messages = (List<IMessage>) probablyMessages;
}
// Single Message
else if (probablyMessages instanceof IMessage)
{
messages = new ArrayList<IMessage>(1);
messages.add((IMessage) probablyMessages);
}
// Probably Pollution
else
{
Log.error("Cannot deserialize message '{}'", probablyMessages.toString());
}
return messages;
}
#Override
public void channelActive(ChannelHandlerContext ctx)
{
if (virtualUser != null)
{
virtualUser.onChannelConnected();
}
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception
{
Log.error(cause);
ctx.close();
}
}
The search
I looked at netty channels related questions on stack overflow, but couldn't find anything relevant to my case.
Links
http://netty.io/4.0/api/io/netty/channel/Channel.html
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html
In our organisation, we implemented our own protocol over UDP and TCP to let external devices connected to the Internet exchange messages with a server that we developed using Netty (indeed!).
For testing purpose, we would like to connect those devices directly to our computers through USB/serial interface (we did not choose the serial communication library yet). We would also like to deploy/port the embedded software we developed for our devices on our computer to simulate the devices and to connect directly to our server using a named pipe for example (IPC).
In the Architecture Overview documentation of Netty, you claim that we could use Netty as well for such serial communication:
"Also, you are even able to take advantage of new transports which aren't yet written (such as serial port communication transport), again by replacing just a couple lines of constructor calls. Moreover, you can write your own transport by extending the core API."
Is anyone somewhere already developed such implementation in Netty or does someone else plan to do such implementation? I am also wondering if Netty is really well-suited for that since the Channel interface and many other ones use a SocketAddress to bind/connect to a peer?
Thank you for your suggestions, advices!
I wonder if you may be able to use the new iostream package for that. All you need here is an InputStream and Outputstream. See [1]
[1] https://github.com/netty/netty/tree/master/transport/src/main/java/io/netty/channel/iostream
It is possible to implement such a solutions. I have not meet problems with binding with SocketAddress.
I’m posting my implementation of USB connection with Netty.
Serial communication is quite simillar, I'm not posting it for brevity. However I am happy to add it as well if anyone needs it.
Here is base class for connection. A ChannelHandler shall be implemented according to communication needs.
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelFutureListener;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelOutboundHandlerAdapter;
import io.netty.channel.ChannelPromise;
import java.net.SocketAddress;
public abstract class ConnectorImpl {
protected ChannelHandler handler;
protected Bootstrap bootstrap;
protected ChannelFuture channelFuture;
public ChannelFuture connect() throws Exception {
if (!isConnected()) {
channelFuture = bootstrap.connect(getSocketAddress()).sync();
}
return channelFuture.channel().closeFuture();
}
public boolean isConnected() {
try {
return channelFuture.channel().isOpen();
} catch (NullPointerException ex) {
return false;
}
}
public void close() {
if (!isConnected()) {
return;
}
try {
channelFuture.channel().close().sync();
} catch (InterruptedException e) {
}
}
protected ChannelOutboundHandlerAdapter createOutgoingErrorHandler() {
return new ChannelOutboundHandlerAdapter() {
#Override
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) {
final ChannelFutureListener channelFutureListener = future -> {
if (!future.isSuccess()) {
future.channel().close();
}
};
promise.addListener(channelFutureListener);
ctx.write(msg, promise);
}
};
}
public abstract SocketAddress getSocketAddress();
}
An extensions of that connector for needed type of connection together with Channel implementations is needed.
USB connector:
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.handler.timeout.ReadTimeoutHandler;
import javax.usb.UsbDevice;
import java.net.SocketAddress;
import java.util.concurrent.TimeUnit;
public class UsbConnectorImpl extends ConnectorImpl {
private static final int READ_TIMEOUT = 60;
private final UsbDevice usbDevice;
public UsbConnectorImpl(UsbChannelHandler handler, UsbDevice usbDevice) {
this.handler = handler;
this.usbDevice = usbDevice;
this.bootstrap = new Bootstrap()
.channel(getChannelClass())
.group(getLoop())
.handler(getChannelInitializer());
}
public EventLoopGroup getLoop() {
return new NioEventLoopGroup(1);
}
Class<UsbAsyncChannel> getChannelClass() {
return UsbAsyncChannel.class;
}
ChannelInitializer<Channel> getChannelInitializer() {
return new ChannelInitializer<Channel>() {
#Override
public void initChannel(#SuppressWarnings("NullableProblems") Channel ch) {
ch.pipeline()
.addLast("Generic encoder", new RequestEncoder())
.addLast("Decoder", new ResponseDecoder())
.addLast("Read timeout handler", new ReadTimeoutHandler(READ_TIMEOUT, TimeUnit.SECONDS))
.addLast("Outgoing Error Handler", createOutgoingErrorHandler())
.addLast("Card Reader handler", handler);
}
};
}
public SocketAddress getSocketAddress() {
return new UsbDeviceAddress(usbDevice);
}
}
USB Channel:
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelFutureListener;
import io.netty.channel.FileRegion;
import io.netty.channel.nio.AbstractNioByteChannel;
import org.usb4java.LibUsb;
import javax.usb.UsbConfiguration;
import javax.usb.UsbDevice;
import javax.usb.UsbEndpoint;
import javax.usb.UsbInterface;
import javax.usb.UsbPipe;
import javax.usb.event.UsbPipeDataEvent;
import javax.usb.event.UsbPipeErrorEvent;
import javax.usb.event.UsbPipeListener;
import java.io.IOException;
import java.net.SocketAddress;
import java.nio.channels.Pipe;
import java.util.concurrent.ScheduledFuture;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
public abstract class UsbChannel extends AbstractNioByteChannel {
protected static final byte INTERFACE_BULK_PIPES = (byte) 1;
private static final AtomicInteger READ_TASK_COUNTER = new AtomicInteger();
private final UsbChannelConfig config;
protected UsbPipe outPipe = null;
protected UsbPipe inPipe = null;
private UsbDevice usbDevice;
private UsbDeviceAddress deviceAddress;
private UsbInterface usbInterface;
public UsbChannel() throws IOException {
super(null, Pipe.open().source());
config = new UsbChannelConfig(this);
}
#Override
public UsbChannelConfig config() {
return config;
}
#Override
public boolean isActive() {
return usbDevice != null;
}
#Override
protected ChannelFuture shutdownInput() {
try {
doClose();
} catch (Exception e) {
pipeline().fireExceptionCaught(e);
}
return null;
}
protected abstract ReadTask createReadTask();
protected void invokeRead() {
ReadTask task = createReadTask();
task.scheduledFuture = eventLoop().schedule(task, 0, TimeUnit.MILLISECONDS);
}
#Override
protected AbstractNioUnsafe newUnsafe() {
return new UsbUnsafe();
}
#Override
protected long doWriteFileRegion(FileRegion region) throws Exception {
throw new UnsupportedOperationException();
}
#Override
protected int doReadBytes(ByteBuf buf) throws Exception {
return 0;
}
#Override
protected Pipe.SourceChannel javaChannel() {
return (Pipe.SourceChannel) super.javaChannel();
}
#Override
protected boolean doConnect(SocketAddress remoteAddress, SocketAddress localAddress) throws Exception {
UsbDeviceAddress remote = (UsbDeviceAddress) remoteAddress;
usbDevice = remote.value();
UsbConfiguration configuration = usbDevice.getActiveUsbConfiguration();
usbInterface = configuration.getUsbInterface(INTERFACE_BULK_PIPES);
usbInterface = usbInterface.getActiveSetting();
usbInterface.claim();
for (int i = 0; i < usbInterface.getUsbEndpoints().size(); i++) {
UsbEndpoint endpoint = (UsbEndpoint) usbInterface.getUsbEndpoints().get(i);
UsbPipe usbPipe = endpoint.getUsbPipe();
if (endpoint.getDirection() == LibUsb.ENDPOINT_IN) {
inPipe = usbPipe;
inPipe.open();
} else if (endpoint.getDirection() == LibUsb.ENDPOINT_OUT) {
outPipe = usbPipe;
outPipe.open();
}
if (inPipe != null && outPipe != null) {
break;
}
}
outPipe.addUsbPipeListener(new UsbPipeListener() {
#Override
public void errorEventOccurred(UsbPipeErrorEvent event) {
pipeline().fireExceptionCaught(event.getUsbException());
}
#Override
public void dataEventOccurred(UsbPipeDataEvent event) {
invokeRead();
}
});
inPipe.addUsbPipeListener(new UsbPipeListener() {
#Override
public void errorEventOccurred(UsbPipeErrorEvent event) {
pipeline().fireExceptionCaught(event.getUsbException());
}
#Override
public void dataEventOccurred(UsbPipeDataEvent event) {
pipeline().fireChannelRead(Unpooled.wrappedBuffer(event.getData(), 0, event.getData().length));
}
});
deviceAddress = remote;
return true;
}
#Override
protected void doFinishConnect() throws Exception {
}
#Override
public UsbDeviceAddress localAddress() {
return (UsbDeviceAddress) super.localAddress();
}
#Override
public UsbDeviceAddress remoteAddress() {
return (UsbDeviceAddress) super.remoteAddress();
}
#Override
protected UsbDeviceAddress localAddress0() {
return deviceAddress;
}
#Override
protected UsbDeviceAddress remoteAddress0() {
return deviceAddress;
}
#Override
protected void doBind(SocketAddress localAddress) throws Exception {
throw new UnsupportedOperationException();
}
#Override
protected void doDisconnect() throws Exception {
doClose();
}
#Override
protected void doClose() throws Exception {
try {
super.doClose();
javaChannel().close();
} finally {
if (inPipe != null) {
inPipe.close();
inPipe = null;
}
if (outPipe != null) {
outPipe.close();
outPipe = null;
}
if (usbInterface != null) {
usbInterface.release();
usbInterface = null;
}
if (usbDevice != null) {
usbDevice = null;
}
}
}
protected abstract static class ReadTask implements Runnable, ChannelFutureListener {
protected final int id;
protected ScheduledFuture<?> scheduledFuture;
public ReadTask() {
this.id = READ_TASK_COUNTER.incrementAndGet();
}
}
private final class UsbUnsafe extends AbstractNioUnsafe {
#Override
public void read() {
}
}
}