How to configure RabbitMQ Cluster on Windows - rabbitmq

Question

Please, I need your kind help.
I am struggling since last week trying to configure RabbitMQ cluster on windows servers. I reinstalled RabbitMQ 3.7.8 and ErlangOTP 21-1 for more than 15 times trying to figure out the issue and fix it, but unfortunately, all of my attempts failed.
My environment:
Three virtual machines with OS: windows server 2012
Environment Variable setting:
• ERLANG_HOME = C:\Program Files\erl10.1
• path: C:\Program Files\Handle
• %RABBITMQ_BASE% = c:\RabbitMQ
• %RABBITMQ_CONFIG_FILE% = %RABBITMQ_BASE%
• RABBITMQ_NODENAME = masternode
• Server names: RabbitMQ01 “masternode” , RabbitMQ02, RabbitMQ03
• Adding IP & hots name “masternode” in the host file for name resolution (C:\Windows\System32\drivers\etc\hosts)
• Setting RabbitMQ configuration file “advanced.config” in c:\users\USERNAME\AppData\Roaming\RabbitMQ” to enable users to login to RabbitMQ management remotely by adding the following setting
[
{rabbit,
[
{tcp_listeners, [5672]},
{loopback_users, []},
{num_tcp_acceptors, 100}
]
}
].
Installing steps: (for each server)
1. After setting the environment variables, I restarted the machine. Then
2. Install ErlangOTP 21-1
3. Install RabbitMQ 3.7.8 server
4. Enable plugins - rabbitmq-plugins.bat enable rabbitmq_management
After completing the installation of the above servers, I start configuring the cluster
First, I verified that masternode is reachable from RabbitMQ02, 02 by pinging Rabbit#masternode and it’s successful.
Stopped RabbitMQ server and App by using the following command
rabbitmq-service stop
rabbitmqctl stop_app
Copy .erlang.cookie from masternode “path=c:\windows\system32\config\ systemprofile” to RabbitMQ02 “path= c:\windows\system32\config\ systemprofile” and “path=c:\users\USERNAME”.
Start RabbitMQ service & App by executing the below commands
rabbitmqctl start_app
rabbitmq-service start
Stop App again by executing this command “rabbitmqctl stop_app”
I start joining RabbitMQ02 to masternode
rabbitmqctl join_cluster rabbit#masternode (small case)
Once I reach this point, I’m getting the following error on RabbitMQ02.
C:\Program Files\RabbitMQ Server\rabbitmq_server-3.7.8\sbin>rabbitmqctl join_cluster rabbit#masternode
Error: unable to perform an operation on node 'rabbit#rhrabbitmq02'. Please see
diagnostics information and suggestions below.
Most common reasons for this are:
Target node is unreachable (e.g. due to hostname resolution, TCP connection o
r firewall issues)
CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang
cookie not matching that of the server)
Target node is not running
In addition to the diagnostics info below:
See the CLI, clustering and networking guides on http://rabbitmq.com/document
ation.html to learn more
Consult server logs on node rabbit#rhrabbitmq02
DIAGNOSTICS
attempted to contact: [rabbit#rhrabbitmq02]
rabbit#rhrabbitmq02:
* connected to epmd (port 4369) on rhrabbitmq02
* epmd reports node 'rabbit' uses port 25672 for inter-node and CLI tool traff
ic
* TCP connection succeeded but Erlang distribution failed
Authentication failed (rejected by the remote node), please check the Erlang
cookie
Current node details:
* node name: rabbitmqcli41#rhrabbitmq02
* effective user's home directory: C:\Users\admin
* Erlang cookie hash: pndBksvQokXyiSqBdnne/g==
masternode Status:
C:\Program Files\RabbitMQ Server\rabbitmq_server-3.7.8\sbin>rabbitmqctl status
Status of node masternode#rhrabbitmq01 ...
[{pid,4008},
{running_applications,
[{rabbitmq_management,"RabbitMQ Management Console","3.7.8"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","3.7.8"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.7.8"},
{rabbit,"RabbitMQ","3.7.8"},
{amqp_client,"RabbitMQ AMQP Client","3.7.8"},
{rabbit_common,
"Modules shared by rabbitmq-server and rabbitmq-erlang-client",
"3.7.8"},
{os_mon,"CPO CXC 138 46","2.4.6"},
{cowboy,"Small, fast, modern HTTP server.","2.2.2"},
{mnesia,"MNESIA CXC 138 12","4.15.5"},
{cowlib,"Support library for manipulating Web protocols.","2.1.0"},
{lager,"Erlang logging framework","3.6.3"},
{syslog,"An RFC 3164 and RFC 5424 compliant logging framework.","3.4.3"},
{ranch_proxy_protocol,"Ranch Proxy Protocol Transport","1.5.0"},
{jsx,"a streaming, evented json parsing toolkit","2.8.2"},
{ranch,"Socket acceptor pool for TCP protocols.","1.5.0"},
{ssl,"Erlang/OTP SSL application","9.0.2"},
{public_key,"Public key infrastructure","1.6.2"},
{asn1,"The Erlang ASN1 compiler version 5.0.7","5.0.7"},
{inets,"INETS CXC 138 49","7.0.2"},
{xmerl,"XML parser","1.3.18"},
{recon,"Diagnostic tools for production use","2.3.2"},
{crypto,"CRYPTO","4.3.3"},
{goldrush,"Erlang event stream processor","0.1.9"},
{compiler,"ERTS CXC 138 10","7.2.5"},
{syntax_tools,"Syntax tools","2.1.6"},
{sasl,"SASL CXC 138 11","3.2.1"},
{stdlib,"ERTS CXC 138 10","3.6"},
{kernel,"ERTS CXC 138 10","6.1"}]},
{os,{win32,nt}},
{erlang_version,
"Erlang/OTP 21 [erts-10.1] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:64
]\n"},
{memory,
[{connection_readers,0},
{connection_writers,0},
{connection_channels,0},
{connection_other,29212},
{queue_procs,0},
{queue_slave_procs,0},
{plugins,1192744},
{other_proc,26188552},
{metrics,195524},
{mgmt_db,337648},
{mnesia,75088},
{other_ets,2748464},
{binary,181696},
{msg_index,27840},
{code,27573597},
{atom,1131721},
{other_system,9846202},
{allocated_unused,9377056},
{reserved_unallocated,0},
{strategy,rss},
{total,[{erlang,69528288},{rss,78905344},{allocated,78905344}]}]},
{alarms,[]},
{listeners,
[{clustering,25672,"::"},
{amqp,5672,"::"},
{amqp,5672,"0.0.0.0"},
{http,15672,"::"},
{http,15672,"0.0.0.0"}]},
{vm_memory_calculation_strategy,rss},
{vm_memory_high_watermark,0.4},
{vm_memory_limit,3435785420},
{disk_free_limit,50000000},
{disk_free,86038073344},
{file_descriptors,
[{total_limit,8092},
{total_used,2},
{sockets_limit,7280},
{sockets_used,0}]},
{processes,[{limit,1048576},{used,399}]},
{run_queue,1},
{uptime,6087},
{kernel,{net_ticktime,60}}]

Answer 1

Problem solution: setting RABBITMQ_NODENAME incorrectly “masternode” instead of “rabbit#masternode”, Error in command execution sequence which crucial thing to perform (when joining the cluster - stop_app first then service stop), and restarting the service after changing the configuration instead of reinstalling the rabbitmq service.
I will be happy to share all of the steps details if needed for installing, configuring rabbitmq, and configuring the cluster on windows based on my setup scenario. Best regards.