JMeter: Non HTTP response message: Connection to URL refused - ssl

Question

I'm using JMeter to test our backend services by using OS Samplers. I'm using CURL in the OS samplers to generate the load of a 4 step process.
POST the certificate to receive the token
POST the token to receive the session
GET session info
POST renew session
The issue is that I'm facing is JMeter reports much higher levels of response time than the service logs. We need to identify where the extra time (+125 ms with 1 concurrent user) is coming from during the transaction execution. The test environment is all on the same VLAN without firewalls or proxy servers between the two client and target servers. The median latency between the two servers is .3 ms with the average being 1.2 ms (with a small sample size). Speaking to the dev team they state that the service logs don't log the very first moment when request is received but can't see how it could be more than just a few ms difference. Data from a few tests which increases the throughput and the overhead roughly remains constant would be consistent with that assumption.
So we focusing on seeing if JMeter is causing the extra overhead, at this point. One assumption is that JMeter begins the transaction time when it begins to generate the CURL request and the packaging of the request is included in this timing. So we want to remove the CURL OS Sampler from the test and replace it with a HTTP Sampler.
When converting the JMeter OS Sampler CURL request to HTTP Sampler HTTPS request we're running into an error JMeter: Non HTTP response message: Connection to URL refused. As stated above we first post the certificate, post the token and then steps 3 and 4. The HTTP Sampler is failing on the 2nd step when posting the acquired token from the first step. We've verified that the acquired token is good by continuing on error and processing the 2nd step original CURL POST request. So there are 2 things to here. 1. the error message says it never completes the handshake so it doesn't get to the point of processing the message. 2. the following CURL request using the same information completes the handshake and correctly processes the transaction.
Making the conversion boils down to a question of "Why would sending a OS Sampler CURL command complete and a HTTP Sampler fail to complete a handshake?
OS Sampler CURL command is configured as:
curl -k -d "" -v -H "{token}" {URL}
HTTP Sampler is configured as:
IP: {URL}
PORT: {Port#}
Implementation: HttpClient4
Protocol: HTTPS
Method: POST
Path: {path}
Use KeepAlive: Check
Header Manager: {token}

Answer 1

There are two separate questions in your post. Regarding the first:
How are you measuring latency between the servers? If you're using ping, you're measuring the round trip time for 1 send and receive. A HTTP POST is usually more than that including TCP back and forth to handshake and then sending out content - which again depending on size can be split across several packets - HTTP responses usually are larger than requests. There is also a possibility of latency being a tad bit higher for larger payload packets compared to a simple ping.
This might not account for the whole difference you're seeing (Like you've noted, some of it comes from the delay in launching curl), but is still something that contributes to increased overall latency. You should use a network analyzer of some sort, at the very least a sniffer like WireShark to understand the chattiness or number-of back and forth turns for each HTTP step you're using.