XMPP Connection time optimization - ssl

We are programming on an iOS Chat Application based on the XMPP Robbie Hanson framework at the moment. Server side we deploy openfire running on 3 servers with hazelcast plugin. Now we encountered following problem: the client connection and authentication takes about 2 sec. without TLS/SSL. With TLS/SSL it takes about 4 sec. We tried everything to shorten this time as it looks strange if the user gets a push notification that he received a message, opens the app and it takes that long to actually get the message. We do not use SRV records so it can’t be the DNS lookup that takes that long. We tried to modify the xmpp handshake so that the user sends all data (startls,auth method...) right from the start without waiting for server response but the server does not accept this. We also tried to use faster servers with very high network bandwith, but this didn't helped. Finally we even tried to use ejabberd but we have exactly the same times so we stayed with openfire.
The reason we thought it MUST be possible to shorten connection times is other messenger like WhatsApp or Threema which need less than 1 sec. So do you have any advice, what else we could try?Is it possible to reach that time only by optimizing the client and without modifying the openfire code?
Thank you so much!
This is my Handshake Log:
C2S - RECV (1083417823): <?xml version='1.0'?>
C2S - RECV (1083417823): <stream:stream xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' version='1.0' to='chat.example.com'>
C2S - SENT (1083417823): <?xml version='1.0' encoding='UTF-8'?><stream:stream xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client" from="chat.example.com" id="5a051bc8" xml:lang="en" version="1.0">
C2S - SENT (1083417823): <stream:features><starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls"><required/></starttls><mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl"><mechanism>DIGEST-MD5</mechanism></mechanisms></stream:features>
C2S - RECV (1083417823): <starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'/>
Starting Hazelcast Clustering Plugin
C2S - RECV (1083417823): <stream:stream xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' version='1.0' to='chat.example.com'>
C2S - SENT (1083417823): <?xml version='1.0' encoding='UTF-8'?><stream:stream xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client" from="chat.example.com" id="5a051bc8" xml:lang="en" version="1.0"><stream:features><mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl"><mechanism>DIGEST-MD5</mechanism></mechanisms><auth xmlns="http://jabber.org/features/iq-auth"/></stream:features>
C2S - RECV (1083417823): <auth xmlns="urn:ietf:params:xml:ns:xmpp-sasl" mechanism="DIGEST-MD5"/>
C2S - SENT (1083417823): <challenge xmlns="urn:ietf:params:xml:ns:xmpp-sasl">cmVhbG09Im5vZGVzLmZsaXhtaW5kZXIuY29tIixub25jer0iQ1hOS3MxWG9WY0xMTmsvedRUWlFIYmpGS1Vta2s4SG5WQ01TWUJnWiIscW9wPSJhdXRoIixjaGFyc2V0PXV0Zi04LGFsZ29yaXRobT1tZDUtc2Vzcw==</challenge>
C2S - RECV (1083417823): <response xmlns="urn:ietf:params:xml:ns:xmpp-sasl">dXNlcm5hbWU9IjAwNDkyMjIiLHJlYWxtPSJub2Rlcy5mbGl4bWluZGVyLmNvbSIsbm9uY2UtrkNYTktzMVhvVmNMTE5rL3dEVFpRSGJqRktVbWtrOEhuVkNNU1lCZ1oiLGNub25jZT0iMkU2RURCRTctNUI2NC00QjQwLTg0OUMtQkUzQ0YwMTRCNTk0IixuYz0wMDAwMDAwMSxxb3A9YXV0aCxkaWdlc3QtdXJpPSJ4bXBwL25vZGVzLmZsaXhtaW5kZXIuY29tIixyZXNwb25zZT1mMDRhYzM4MjBlY2MwMGE1Mjk1ZTkxMjc5YTc1Zmz4MCxjaGFyc2V0PXV0Zi04</response>
C2S - SENT (1083417823): <success xmlns="urn:ietf:params:xml:ns:xmpp-sasl">cnNwYXV0aD05NDk2NTA2NWRlNDQ2MzRhNWRlMWNzuTc0NjI3MGNhZg==</success>
C2S - RECV (1083417823): <stream:stream xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' version='1.0' to='chat.example.com'>
C2S - SENT (1083417823): <?xml version='1.0' encoding='UTF-8'?><stream:stream xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client" from="chat.example.com" id="5a051bc8" xml:lang="en" version="1.0"><stream:features><bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"/><session xmlns="urn:ietf:params:xml:ns:xmpp-session"/></stream:features>
C2S - RECV (1083417823): <iq type="set" id="F3CFA293-6D45-4E03-9065-3FD10D617C02"><bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"/></iq>
C2S - SENT (1083417823): <iq type="result" id="F3CFA293-6D45-4E03-9065-3FD10D617C02" to="chat.example.com/5a051bc8"><bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"><jid>222#chat.example.com/5a051bc8</jid></bind></iq>
C2S - RECV (1083417823): <iq type="set" id="D351FF85-535B-4B08-B5C2-3C11D92C1EA9"><session xmlns="urn:ietf:params:xml:ns:xmpp-session"/></iq>
C2S - SENT (1083417823): <iq type="result" id="D351FF85-535B-4B08-B5C2-3C11D92C1EA9" to="222#chat.example.com/5a051bc8"/>
C2S - RECV (1083417823): <presence/>
C2S - SENT (1083417823): <presence from="222#chat.example.com/5a051bc8" to="222#chat.example.com/5a051bc8"/>
C2S - SENT (1083417823): <presence from="222#chat.example.com/5a051bc8" to="222#chat.example.com/5a051bc8"/>
CLOSED (1446853640)

I'd recommend trying to get some logs to find out exactly what is taking the most amount of time. Figuring out exactly how many roundtrips you are using will help you determine what to optimize.
There's a XEP for XMPP Quickstart, XEP-0305. This has some general recommendation, but also a pipelining protocol which should do the bundling of data for you, if your server and client support it.
Some tips:
Make sure you cache DNS results for the specified TTL.
Save the user's roster locally and use roster versioning to only get any changes that might've happend to the user's roster.
At the TLS level, you could try to get session resumption or false start working. Also make sure the server sends no extra certificates (like a root you know the client will trust). Use faster algorithms (ECDHE instead of DHE, RSA-2048 instead of RSA-4096), but keep security in mind (please no RC4).
If you do stuff like setting/retrieving vCards, service discovery, etc., make sure that happens later and doesn't block anything else.
If you are using SCRAM-SHA-1 and the server is using hashed password storage (i.e., sending the same salt every time) you can cache the SaltedPassword value, which should save a large amount of time.
If your server is new enough (so implements RFC 6121 correctly), you can skip a roundtrip by skipping the urn:ietf:params:xml:ns:xmpp-session IQ. See https://datatracker.ietf.org/doc/draft-cridland-xmpp-session/?include_text=1.
Cache the entity caps of your contacts and your server to skip retrieving them. You can even embed caps that are used often inside your app.
If at any step you want to send multiple stanzas at the same time, make sure they are sent in a single TLS packet. Every packet has both an overhead in size (somewhere in between 25-85 extra bytes: header, IV, padding, MAC) and in processing time (parsing, verifying MAC).

It appears that you don't now where the 4 seconds are spend, it's hard to give you a specific answer. I snuggest you try to profile that time-frame and determine what mechanisms/phases/methods are invoked and how much time they take to complete.
There is XEP-305: XMPP Quickstart. Which may provides additional information for you. Besides that I doubt that you can improve the time it takes to establish an XMPP session without modifying the server code.

As a data-point, I work with an ejabberd deployment at scale with mobile clients based on the SleekXMPP client and there's no problems with connection delays. I've often used OpenFire for small-scale office communications and it's not that slow, either.
Mobile networking is tricky. Try connecting to your server from a desktop XMPP client, preferably on the same network as the server. This will help you eliminate (or implicate) the network and the mobile client as the problem.
If the problem appears isolated to the server, the only real possibility is that you're overloaded. If you're only connecting a single client, then either your server is very anemic or you're talking to some external DB for authentication and that DB is overloaded.
If the problem is not the server, try connecting the mobile client to WiFi rather than a cell network. Then run a packet-sniffer adjacent to the client and another one on the server. That should tell you whether you have some client bug or a network problem.

Related

JMeter SocketException when uploading 1GB file

JMeter 5.4.1
OpenJDK 15.0.1
My test server is configured by default to allow a max 1073741824 byte file to be uploaded, a limit that is configurable. My goal is to validate that the configured limit is respected.
When I configure it for 1048576 bytes and exceed that limit with my upload, the server sends the response:
"{"type":"https://tools.ietf.org/html/rfc7231#section-6.5.1","title":"One or more validation errors occurred.","status":400,"traceId":"|bd37f36b-4cd68e1b8b433669.","errors":{"":["Failed to read the request form. Multipart body length limit 1048576 exceeded."]}}"
When I configure it for 1073741824 bytes and exceed that limit with my upload, JMeter reports the following error:
java.net.SocketException: Connection reset by peer
at java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:420)
at java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:440)
at java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:826)
at java.base/java.net.Socket$SocketOutputStream.write(Socket.java:1051)
at java.base/sun.security.ssl.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:342)
at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1277)
at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167)
at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)
at org.apache.http.entity.mime.content.FileBody.writeTo(FileBody.java:121)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$ViewableFileBody.writeTo(HTTPHC4Impl.java:1513)
at org.apache.http.entity.mime.AbstractMultipartForm.doWriteTo(AbstractMultipartForm.java:134)
at org.apache.http.entity.mime.AbstractMultipartForm.writeTo(AbstractMultipartForm.java:157)
at org.apache.http.entity.mime.MultipartFormEntity.writeTo(MultipartFormEntity.java:113)
at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:152)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$2.doSendRequest(HTTPHC4Impl.java:458)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeRequest(HTTPHC4Impl.java:935)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:646)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:66)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1296)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1285)
at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:638)
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:558)
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256)
at java.base/java.lang.Thread.run(Thread.java:832)
My JMeter.bat file has the heap set as follows:
set HEAP=-Xms1g -Xmx4g -XX:MaxMetaspaceSize=256m
This looks to be similar to a post from last Oct, but there was no solution suggested/reported.
SocketException after sending huge request via JMeter
Connection reset by peer means that your server has reset the connection so you should rather look for the clue in your server logs.
The only thing I suggest to do on JMeter side is to consider switching to HTTP Raw Request sampler, it has nice feature of streaming the file directly to the server without loading it to memory first, I think you will find it extremely helpful when it comes to load testing with more than one virtual user. See HTTP Raw Request for SOAP + MTOM post on JMeter Plugins support forum for more details.

SSL handshake error twisted

So I have a twisted application where I create the SSL service like this:
myfactory=MyFactory(host, port)
service=SSLServer(port,myfactory, ssl.DefaultOpenSSLContextFactory(MY_SERVICE_SSL_KEY, MY_SERVICE_SSL_CERT))
Here MyFactory is just my custom Factory class, and MY_SERVICE_SSL_KEY and MY_SERVICE_SSL_CERT are the file paths to the key and cert files required by SSL.
The service is such that a client will connect to the service, send a line of information (which gets stored on on the server side) and then disconnects.
The issue is that for some client IPs, everything "seems" to work: the client connects, the server receives the expected data, and then connectionLost is called. I log the reason for connectionLost and get this:
[Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.
But for other client IP addresses, there seems to be an SSL handshake error where the connection is lost immediately:
application [28/Nov/2017 02:39:36] INFO MyProtocol ConnectionMade: xx.xx.xx.xx
application [28/Nov/2017 02:39:36] INFO MyProtocol ConnectionLost: xx.xx.xx.xx | [Failure instance: Traceback: <class 'OpenSSL.SSL.Error'>: [('SSL routines', 'SSL3_READ_BYTES', 'ssl handshake failure')]
Does anyone know why this happens? How can I fix it?

Google CCS (GCM) - project not whitelisted

I'm trying to get the Python code working that I found on:
http://developer.android.com/google/gcm/ccs.html
I've change the first 2 rows with (I think) the correct data.
The projectnr and api key is fake, it's just to show you how it almost looks.
import sys, json, xmpp
SERVER = ('gcm.googleapis.com', 5235)
USERNAME = '489713985816'
PASSWORD = 'AIzd237jjN_iT7yRxLWiHRreqax45XaMJQ6VJ98'
I've created a google api project (tried it with 2 different projects).
Activated GCM.
Copied the following:
Project Number: 489713985816
API key : AIzd237jjN_iT7yRxLWiHRreqax45XaMJQ6VJ98
Tried the code with a Key for server, and a key for browser apps, both with and without a specific IP address.
When I execute the code with #python ccs.py I get the following result:
If this is my problem, how do I get my project whitelisted?
Invalid debugflag given: socket
DEBUG:
DEBUG: Debug created for /usr/lib/python2.7/dist-packages/xmpp/client.py
DEBUG: flags defined: socket
DEBUG: socket start Plugging <xmpp.transports.TCPsocket instance at 0x1ea2950>
into <xmpp.client.Client instance at 0x1ea27a0>
DEBUG: socket start Successfully connected to remote
host ('gcm.googleapis.com', 5235)
DEBUG: socket sent <?xml version='1.0'?>
<stream:stream xmlns="jabber:client" to="gcm.googleapis.com" version="1.0"
xmlns:stream="http://etherx.jabber.org/streams" >
DEBUG: socket got
<stream:stream from="gcm.googleapis.com" id="FD82304ADA8C8019" version="1.0"
xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client">
<stream:features>
<mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl">
<mechanism>X-OAUTH2</mechanism>
<mechanism>X-GOOGLE-TOKEN</mechanism>
<mechanism>PLAIN</mechanism>
</mechanisms>
</stream:features>
DEBUG: socket sent <auth xmlns="urn:ietf:params:xml:ns:xmpp-sasl"
mechanism="PLAIN">MjgzMVqTl9p\nVDdUTZWSjk4\n</auth>
DEBUG: socket got <failure xmlns="urn:ietf:params:xml:ns:xmpp-sasl">
<temporary-auth-failure/>
<text xmlns="urn:ietf:params:xml:ns:xmpp-stanzas">
Project 489713985816 not whitelisted.</text>
</failure>
</stream:stream>
Authentication failed!
you might wan't to try the following guide http://www.androidhive.info/2012/10/android-push-notifications-using-google-cloud-messaging-gcm-php-and-mysql/
I was having the same problem that you are but following this guide has helped me get my push notifications through without having to sign up to be whitelisted.
After 3 months of waiting, I've just received an email from a Google employee.
My GCM whitelist request has been approved.
Thank you Ashish.
Now, let the fun begin!
In the documentation, it is mentioned several times that to use Up-stream messaging, you need to ask authorization (be whitelisted).
You can do that here: https://services.google.com/fb/forms/gcm/
You can still use the old "Cloud to device" messaging. You can read more about this, including links to a sample project here.

Apache, mod_ssl "request failed: error reading the headers" for a specific user

Currently we have an Apache 2.2.3 server with mod_ssl 2.2.3 running Django, with users authenticating by using a x509 certificate.
So far the system is running perfectly except for a single user, who when trying to upload a file receives 400 Bad Request error, and the contents of the ssl_error_log regarding this operation are:
[<date>] [error] [client <client ip>] request failed: error reading the headers, referer: <referrer url>
The contents of the ssl_access_log are:
<client ip> - - [<date>] "POST <target page> HTTP/1.1" 400 321
Also, the user's browser is Firefox as far as I know.
I am completely unable to reproduce this bug and so far none of the other users have experienced it. Could you point out some reasons for this to happen?
I've experienced connectivity that stops the upstream after an X amount of bytes is sent. X was a pretty low value, as in enough to request some simple pages, but not to deal with ajax requests much less upload files. As far as I recall, this connectivity problem occurred only when tethering (from a specific Android phone, but I didnt even test other phones).
So if the upstream gets interrupted and the upload stalls, it makes sense apache would return this error, according to this post: "Apache waits a time equal to the Timeout directive (defaults to 5 minutes if not defined) for a response from the client. It is likely Apache is waiting for the CRLF that indicates the end of the headers, yet it is never received.."

How to receive sms via smpp connection - Kannel

I sent messages through smpp connection (using selenium SmppSim) from Kannel and it worked.
But somehow when I try to receive messages or in other words when I try to send messages from SmppSim It doesn't work. The MO messages of the SmppSim queue into the MO-queue.
I tried these things.
Used same port for send and receive (Kannel/SmppSim).
Used different ports for send and receive (Kannel/SmppSim).
Two groups for same smsc-smpp for send and receive. (It may be wrong)
Now I'm using port 2775 for send and port 2776 for receive.
#kannel.conf
group=smsc
smsc=smpp
....
port = 2775
receive-port = 2776
transceiver-mode = true
....
In SmppSim
#smppsim.props
SMPP_PORT=2775
....
SYSTEM_IDS=smppclient
PASSWORDS=password
OUTBIND_ENABLED=true
OUTBIND_ESME_IP_ADDRESS=127.0.0.1
OUTBIND_ESME_PORT=2776
OUTBIND_ESME_SYSTEMID=smppclient
OUTBIND_ESME_PASSWORD=password
....
When I run the bearerbox, it shows like below. (sms send is working)
....
connect failed
System error 111: Connection refused
ERROR: error connecting to server `localhost' at port `2776'
SMPP[SMPPSim]: Couldn't connect to server.
SMPP[SMPPSim]: Couldn't connect to SMS center (retrying in 10 seconds).
....
How do I configure this?
Thank you!
Please read SMPP v3.4 specification, part 2.2.1.
The purpose of the outbind operation is to allow the SMSC signal an ESME to originate a
bind_receiver request to the SMSC.
So it's used for SMSC (SMPPSim) to connect to ESME (Kannel) and request for callback connection.
However you can run few SMPPSim instances listening on different ports. Each instance should use own configuration file this case.