I'm trying to setup JMeter in a distributed mode.
I have a server running on an ec2 intance, and I want the master to run on my local computer.
I had to jump through some hopes to get RMI working correctly on the server but was solved with setting the "java.rmi.server.hostname" to the IP of the ec2 instance.
The next (and hopefully last) problem is the server communicating back to the master.
The problem is that because I am doing this from an internal network, the master is sending its local/internal ip address (192.168.1.XXX) when it should be sending back the IP of my external connection (92.XXX.XXX.XXX).
I can see this in the jmeter-server.log:
ERROR - jmeter.samplers.RemoteListenerWrapper: testStarted(host) java.rmi.ConnectException: Connection refused to host: 192.168.1.50; nested exception is:
That host IP is wrong. It should be the 92.XXX.XXX.XX address. I assume this is because in the master logs I see the following:
2012/07/29 20:45:25 INFO - jmeter.JMeter: IP: 192.168.1.50 Name: XXXXXX.local FullName: 192.168.1.50
And this IP is sent to the server during RMI setup.
So I think I have two options:
Tell the master to send the external IP
Tell the server to connect on the external IP of the master.
But I can't see where to set these commands.
Any help would be useful.
For the benefit of future readers, don't take no for an answer. It is possible! Plus you can keep your firewall in place.
In this case, I did everything over port 4000.
How to connect a JMeter client and server for distributed testing with Amazon EC2 instance and local dev machine across different networks.
Setup:
JMeter 2.13 Client: local dev computer (different network)
JMeter 2.13 Server: Amazon EC2 instance
I configured distributed client / server JMeter connectivity as follows:
1. Added a port forwarding rule on my firewall/router:
Port: 4000
Destination: JMeter client private IP address on the LAN.
2. Configured the "Security Group" settings on the EC2 instance:
Type: Allow: Inbound
Port: 4000
Source: JMeter client public IP address (my dev computer/network public IP)
Update: If you already have SSH connectivity, you could use an SSH tunnel for the connection, that will avoid needing to add the firewall rules.
$ ssh -i ~/.ssh/54-179-XXX-XXX.pem ServerAliveInterval=60 -R 4000:localhost:4000 jmeter#54.179.XXX.XXX
3. Configured client $JMETER_HOME/bin/jmeter.properties file RMI section:
note only the non-default values that I changed are included here:
#---------------------------------------------------------------------------
# Remote hosts and RMI configuration
#---------------------------------------------------------------------------
# Remote Hosts - comma delimited
# Add EC2 JMeter server public IP address:Port combo
remote_hosts=127.0.0.1,54.179.XXX.XXX:4000
# RMI port to be used by the server (must start rmiregistry with same port)
server_port=4000
# Parameter that controls the RMI port used by the RemoteSampleListenerImpl (The Controler)
# Default value is 0 which means port is randomly assigned
# You may need to open Firewall port on the Controller machine
client.rmi.localport=4000
# To change the default port (1099) used to access the server:
server.rmi.port=4000
# To use a specific port for the JMeter server engine, define
# the following property before starting the server:
server.rmi.localport=4000
4. Configured remote server $JMETER_HOME/bin/jmeter.properties file RMI section as follows:
#---------------------------------------------------------------------------
# Remote hosts and RMI configuration
#---------------------------------------------------------------------------
# RMI port to be used by the server (must start rmiregistry with same port)
server_port=4000
# Parameter that controls the RMI port used by the RemoteSampleListenerImpl (The Controler)
# Default value is 0 which means port is randomly assigned
# You may need to open Firewall port on the Controller machine
client.rmi.localport=4000
# To use a specific port for the JMeter server engine, define
# the following property before starting the server:
server.rmi.localport=4000
5. Started the JMeter server/slave with:
jmeter-server -Djava.rmi.server.hostname=54.179.XXX.XXX
where 54.179.XXX.XXX is the public IP address of the EC2 server
6. Started the JMeter client/master with:
jmeter -Djava.rmi.server.hostname=121.73.XXX.XXX
where 121.73.XXX.XXX is the public IP address of my client computer.
7. Ran a JMeter test suite.
JMeter GUI log output
Success!
I had a similar problem: the JMeter server tried to connect to the wrong address for sending the results of the test (it tried to connect to localhost).
I solved this by setting the following parameter when starting the JMeter master:
-Djava.rmi.server.hostname=xx.xx.xx.xx
It looks as though this wont work Distributed JMeter Testing explains the requirements for load testing in a distributed environment. Number 2 and 3 are particular to your use case I believe.
The firewalls on the systems are turned off.
All the clients are on the same subnet.
The server is in the same subnet, if 192.x.x.x or 10.x.x.x ip addresses are used.
Make sure JMeter can access the server.
Make sure you use the same version of JMeter on all the systems. Mixing versions may not work correctly.
Might be very late in the game but still. Im running this with jmeter 5.3.
So to get it work by setting up the slaves in aws and the controller on your local machine.
Make sure your slave has the proper localports and hostname. The hostname on the slave should be the ec2 instance public dns.
Make sure AWS has proper security policies.
For the controller (which is your local machine) make sure you run with the parameter '-Djava.rmi.server.hostname='. You can get the ip by googling "my public ip address". Definately not those 192.xxx.xxx.x or 172.xx.xxx.
Then you have to configure your modem to port forward your machine that is used to be your controller. The port can be obtained when from the slave log (the ones that has the FINE: RMI RenewClean....., yeah you have to set the log to verbose). OR set DMZ and put your controller machine. Dangerous, but convinient just for the testing time, don't forget to off it after that
Then it should work.
Related
While trying to access to instances via NoVNC console from other nodes such as storage and compute connection at 127.0.0.1:6080 cannot be established but everything works well on the controller node.
Still looking for the missing configuration
this IP '127.0.0.1' is a localhost IP, which means that this IP can resolve from the server on that server and another server, 127.0.0.1 point to a different destination.
you should run noVNC on some IP from the network and you can configure this at /etc/kolla/nova-novncproxy/nova.conf
It's not that much of a question, rather a confirmation that what I did is right or not and if it is safe or not.
Until now what I have found googling around is that you cannot run rtorrent through a proxy. You can either put the http request through a proxy, or tsocks, in both cases either the actual transfers are done directly or not done at all. Therefore until now the only proposed viable solution is a VPN which I wanted to avoid.
What I did was use an http proxy for the http part and a port forwarding for the actual download part. For example, lets assume the following:
192.168.1.10 --> Local machine with the actual rtorrent
remote.machine.com --> The remote machine used as a proxy
Procedure:
I created 2 ssh tunnels
ssh -N -D 9090 user#remote.machine.com
ssh -R 9091:localhost:9091 user#remote.machine.com
From the local machine I installed polipo as the html proxy and configured it to use a socks proxy in the remote.machine.com.
I edited the following lines in /etc/polipo/config so that I can get the socks proxy.
socksParentProxy = "localhost:9090"
socksProxyType = socks5
I also changed the html proxy port for extra security, again in /etc/polipo/config
proxyPort = 9080
On the local machine I changed the ~/.rtorrent.rc as following:
#Proxy of the http requests through polipo
http_proxy=localhost:9080
# The ip address reported to the tracker.
#Really important, in order to get connections for downloads
ip = remote.machine.com
# The ip address the listening socket and outgoing connections is
# bound to.
bind = 192.168.1.10
# Port range to use for listening.
port_range = 9091-9091
# Start opening ports at a random position within the port range.
port_random = no
The system seems to work. I connect to the trackers and I have up and down traffic. So the questions are:
Am I safe that all the traffic concerning rtorrent is done through the remote.machine.com?
Did I miss something?
Are there any problems or concerns regarding this method?
As far as I see, you have covered inbound connections, as well as outgoing HTTP traffic, but any outbound peer-to-peer connections will be created directly, not through any tunnel. Currently, rtorrent does not appear to support passing outbound P2P connections through a tunnel or proxy of any kind, so in order to handle these, you'll need some other mechanism.
You mentioned tsocks and that it does not work – not even in addition to the rtorrent configuration you have set up above? (Although with tsocks you should be able to drop the HTTP proxy part.)
If that fails, there are alternatives to tsocks mentioned on the tsocks project page. A slightly more involved alternative would be to create a new loopback interface (lo:1 with IP 127.0.0.2), bind your rtorrent to that one and use something like sshuttle to direct all traffic originating on that interface through an SSH tunnel. Unfortunately, sshuttle doesn't let you restrict its operation to a specific interface at the moment, though, so you'd have to fiddle with the iptables rules it creates to make them match your needs. I assume a patch adding this feature to sshuttle would be welcome.
As a side note, you can create multiple port forwards and SOCKS proxies in a single SSH connection, like this:
ssh -N -D 9090 -R 0.0.0.0:9091:localhost:9091 myself#my.example.com
I'm trying to get Apache working on a GCE instance.
Following GCE's Quickstart guide, I did the following:
Created instance "my-instance" in "my-project" (CentOS image)
Installed httpd, verified it's running
Added the following firewall rule:
gcutil addfirewall http2 --description="Incoming http allowed." --allowed="tcp:http"
and did the same for HTTPS and ICMP
Verified through gce gui that these rules were added to default network
I can ping my instance's IP address but I can't get an HTTP response. I've tried through the browser, from a curl command - no dice. And it works fine when on localhost so I know Apache is returning the index.html page.
When I use curl from a remote host, the error is:
curl: (7) Failed connect to (instance ip addr):80; Connection refused
Thoughts?
I did some experiments to replicate this. In short, I believe HTTP port 80 may be blocked by iptables firewall rules on the local Centos instance. This appears to be the default behavior.
I have a GCE firewall rule setup to allow port 80 traffic to all instances. I created a centos based image via the Cloud Console (which is indeed using the v1 API). Logged in via SSH and started a web server on port 80. I was not able to hit the web server from my laptop. However I was also not able to hit it from another instance in my project. This lead me to suspect a firewall local to the instance rather than Compute Engine's firewall.
I ran this command (which drops the default reject of all ports for testing - this is unsafe to do for machines which are directly exposed to the internet):
$ sudo iptables -D INPUT -j REJECT --reject-with icmp-host-prohibited
After running that, I was able to hit my webserver from both another instance and my laptop. Note that this change is lost after restarting the instance. I don't know the correct procedure for changing the default firewall rules on Centos.
Please try a similar experiment on your instances, especially try to hit the web server from another Compute Engine instance, since service level firewalls do not block traffic between instances on the same network.
I have Machine A where stunnel is running. I am sending data on port of Machine A from Machine B but stunnel is not listening this port when request comes from different machine. It works fine when I send request on same port from local machine.
Stunnel setting on Machine A is like this:
[MyConn]
client = yes
accept = 127.0.0.1:3500
connect=67.68.90.73:5510
Machine B
Application is sending data on port 3500 of Machine A:
SocketConnectHost=MachineA
SocketConnectPort=3500
But I don't see any request coming in stunnel from Machine B, I have also checked that when I run tcplistener on Machine A, request is coming from Machine B. It seems issues with stunnel application. Please suggest is there any configuration or setting needed in stunnel.
127.0.0.1 is the local machine. You have restricted connections which can be accepted to the local machine only.
You can run stunnel in inetd mode (inetd starts stunnel up when a connection attempt arrives), or the preferred daemon mode. Refer to the stunnel HOWTO for more details. Here's what you do for daemon mode.
First add the following line to /etc/services:
myconn 3500/tcp # name the service
Stunnel configuration file needs at least the section name and accept option. For example:
[myconn service]
accept = myconn
...
I have recently starred out with EC2. Currently I am using the Free Tier to test and learn about it. However as I am behind a proxy that allows only connections at port 80 and 443, I am unable to connect the EC2 instance. Is there a way to get past this ?
So far I've guess that running sslh on the EC2 instance, as described here might help. But I am not sure if this behavior should remain persistent once the instance is terminated and re-started (as I am using Free Tier). Is there a way I can achieve persistence in terms of settings and installed resources like sslh (and many others) while using the Free Tier ?
Thanks in advance.
Once when behind a firewall that only allowed outgoing communication on ports such as 80, I just ran an sshd on the server on a different port. You won't be able to set this up while behind the firewall, you'll have to go somewhere else, ssh in, and reconfigure ssh.
Instead of running sshd on a non-standard port, you could also just have something redirect traffic from some other port to port 22.
If your ec2 instance isn't running a web server, you can use port 80 or 443 for the sshd. If you're not using https, then use 443.
You say they only allow outgoing traffic to remote ports 80 and 443, but often times ports above 1024 are also unblocked.
Make sure you've also correctly configured your security groups on the ec2 instance, since it has a firewall as well. You'll have to make sure it's configured to allow incoming traffic on the port supplying the sshd from your IP address. This can be done through the aws management console.
Here there's is a neat solution. I haven't tried it. The idea is to pass a script to boot the instance with ssh bind to port 80.
Goto instances
at the top of the list of your running instances you should see "instance action"
In that menu you should see "connect"
Select "connect from your browser using Java ssh client"
note, you need Java to be installed.