Masstransit cannot access host machine RabbitMQ from a docker container - rabbitmq

I created a simple .net core console application with docker support. Following
Masstransit code fails to connect to RabbitMQ instance on host machine. But similar implementation using RabitMq.Client is able to connect to host machine RabbitMQ instance.
Masstransit throws
MassTransit.RabbitMqTransport.RabbitMqConnectionException: Connect
failed: ctas#192.168.0.9:5672/ --->
RabbitMQ.Client.Exceptions.BrokerUnreachableException:
host machine ip : 192.168.0.9
using Masstransit
string rabbitMqUri = "rabbitmq://192.168.0.9/";
string userName = "ctas";
string password = "ctas#123";
string assetServiceQueue = "hello";
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(rabbitMqUri), hst =>
{
hst.Username(userName);
hst.Password(password);
});
cfg.ReceiveEndpoint(host,
assetServiceQueue, e =>
{
e.Consumer<AddNewAssetReceivedConsumer>();
});
});
bus.Start();
Console.WriteLine("Service Running.... Press enter to exit");
Console.ReadLine();
bus.Stop();
Using RabbitMQ Client
public static void Main()
{
var factory = new ConnectionFactory();
factory.UserName = "ctas";
factory.Password = "ctas#123";
factory.VirtualHost = "watcherindustry";
factory.HostName = "192.168.0.9";
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: "hello",
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
};
channel.BasicConsume(queue: "hello",
autoAck: true,
consumer: consumer);
Console.WriteLine(" Press [enter] to exit.");
Console.ReadLine();
}
}
Docker file
FROM microsoft/dotnet:1.1-runtime
ARG source
WORKDIR /app
COPY ${source:-obj/Docker/publish} .
ENTRYPOINT ["dotnet", "TestClient.dll"]

I created an example, and was able to connect my host, using the preview package from masstransit.
Start rabbitmq in docker and expose ports on the host
docker run -d -p 5672:5672 -p 15672:15672 --hostname my-rabbit --name some-rabbit rabbitmq:3-management
Build and run console app.
docker build -t dotnetapp .
docker run -d -e RABBITMQ_URI=rabbitmq://guest:guest#172.17.0.2:5672 --name some-dotnetapp dotnetapp
To verify your receiving messages run
docker logs some-dotnetapp --follow
you should see the following output
Application is starting...
Connecting to rabbitmq://guest:guest#172.17.0.2:5672
Received: Hello, World [08/12/2017 04:35:53]
Received: Hello, World [08/12/2017 04:35:58]
Received: Hello, World [08/12/2017 04:36:03]
Received: Hello, World [08/12/2017 04:36:08]
Received: Hello, World [08/12/2017 04:36:13]
...
Notes:
172.17.0.2 was my-rabbit container ip address but you can replace it with your machine ip address
http://localhost:15672 is the rabbitmq management console log in with guest as username and password.
Lastly portainer.io is a very useful application to visually view you local docker environment.

Thanks for the response. I managed to resolve this issue. My findings are as follows.
to connect to a rabbitmq instance on another docker container, they have to be moved/connected to the same network. To do this
create a newtork
docker network create -d bridge my_bridge
connect both app and rabbitmq containers to same network
docker network connect my_bridge <container name>
For masstransit uri use rabbitmq container IP on that network or container name
To connect rabbitmq instance of host machine from a app on docker container.
masstransit uri should include machine name( I tried IP, that did not work)

Try using virtual host in MassTransit configuration too, not sure why you decided to omit it.
var host = cfg.Host("192.168.0.9", "watcherindustry", hst =>
{
hst.Username(userName);
hst.Password(password);
});

Look at Alexey Zimarev comment to your question, if your rabbit runs on a container then it should be on your docker-compese file and then use that entry in your endpoint definition to connect to rabbit because docker creates an internal network on which you are agnostic from source code...
rabbitmq:
container_name: "rabbitmq-yournode01"
hostname: rabbit
image: rabbitmq:3.6.6-management
environment:
- RABBITMQ_DEFAULT_USER=yourusergoeshere
- RABBITMQ_DEFAULT_PASS=yourpasswordgoeshere
- RABBITMQ_DEFAULT_VHOST=vhost
volumes:
- rabbit-volume:/var/lib/rabbitmq
ports:
- "5672:5672"
- "15672:15672"
In your app settings you should have something lie:
"ConnectionString": "host=rabbitmq:5672;virtualHost=vhost;username=yourusergoeshere;password=yourpasswordgoeshere;timeout=0;prefetchcount=1",
And if you'd use EasyNEtQ you could do:
_bus = RabbitHutch.CreateBus(_connectionString); // The one above
I hope it helps,
Juan

Related

Terraform - Failed to set up SSH tunneling for host

Hell, I am trying to deploy rke k8s with terraform, but I am not able to connect to the desired host via ssh:
time="2022-02-28T11:17:38+01:00" level=warning msg="Failed to set up SSH tunneling for host [poc-k8s.my-domain.com]: Can't retrieve Docker Info: error during connect: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info\": Unable to access node with address [poc-k8s.my-domain.com:22] using SSH. Please check if you are able to SSH to the node using the specified SSH Private Key and if you have configured the correct SSH username. Error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain"
and this is the .tf file I am using:
terraform {
required_providers {
rke = {
source = "rancher/rke"
version = "1.3.0"
}
}
}
provider "rke" {
log_file = "rke_debug.log"
}
resource "rke_cluster" "cluster" {
nodes {
address = "poc-k8s.my-domain.com"
user = "root"
role = ["controlplane", "worker", "etcd"]
ssh_key = file("~/.ssh/root_key")
}
nodes {
address = "poc-k8s.my-domain.com"
user = "root"
role = ["worker", "etcd"]
ssh_key = file("~/.ssh/root_key")
}
addons_include = [
"https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml",
"https://gist.githubusercontent.com/superseb/499f2caa2637c404af41cfb7e5f4a938/raw/930841ac00653fdff8beca61dab9a20bb8983782/k8s-dashboard-user.yml",
]
}
resource "local_file" "kube_cluster_yaml" {
filename = "~/.kube/kube_config_cluster.yml"
sensitive_content = "rke_cluster.cluster.kube_config_yaml"
}
The key if of course correct and I am able to connect to the desired host:
ssh -i ~/.ssh/root_key root#poc-k8s.my-domain.com
what am I missing here?
[Update]
Cluster resource has delay_on_creation property that can be used
resource "rke_cluster" "cluster" {
delay_on_creation = 180
(...)
}
I'm facing a similar issue. On the second run of terrafor apply it works correctly. In my case the issue is that docker is not up fast enough for RKE provider.
I've found following workaround from citynetwork /
citycloud-examples:
resource "rke_cluster" "cluster" {
(...)
depends_on = [null_resource.wait-for-docker]
}
resource "null_resource" "wait-for-docker" {
provisioner "local-exec" {
command = "sleep 180"
}
depends_on = [
# list of servers docker being installed on
(...)
]
}
It waits for 180s which is not ideal, though.

How to expose RSK node to an external network?

I am having problems exposing my RSK node to an external IP.
My startup command looks as follows:
java \
-cp $HOME/Downloads/rskj-core-3.0.1-IRIS-all.jar \
-Drsk.conf.file=/root/bitcoind-lnd/rsk/rsk.conf \
-Drpc.providers.web.cors=* \
-Drpc.providers.web.ws.enabled=true \
co.rsk.Start \
--regtest
This is my rsk.conf:
rpc {
providers {
web {
cors: "*",
http {
enabled = true
bind_address = "0.0.0.0"
hosts = ["localhost", "0.0.0.0"]
port: 4444
}
}
}
}
API is accessible from localhost, but from external network I get error 400. How do I expose it to external network?
You should add your external IP to hosts. Adding just 0.0.0.0 is not enough to indicate all IPs to be valid. Port forwarding needs to be enabled for the port number that you have configured in rsk.conf, which in this case is the default value of 4444.
rpc {
providers {
web {
cors: “*”,
http {
enabled = true
bind_address = “0.0.0.0"
hosts = [“localhost”, “0.0.0.0", “216.58.208.100”]
port: 4444
}
}
}
}
where 216.58.208.100 is your external IP

How to use `ioredis` to connect to Redis instance (AWS elasticcache) across ssh tunnel with SSL?

This seems to be something about ioredis and its support for TLS. This is all on a mac, Catalina, etc.
I have an elasticcache Redis instance running, inside a VPC. I tunnel to it with ssh,
ssh -L 6379:clustercfg.my-test-redis.amazonaws.com:6379 -N MyEC2
The following doesn't work with node 12.9, ioredis 4.19.4
> const Redis = require("ioredis");
> const redis = new Redis('rediss://127.0.0.1:6379');
[ioredis] Unhandled error event: Error [ERR_TLS_CERT_ALTNAME_INVALID]: Hostname/IP does not match certificate's altnames: IP: 127.0.0.1 is not in the cert's list:
at Object.checkServerIdentity (tls.js:287:12)
<repeated ... many times>
This doesn't work either:
> const Redis = require("ioredis");
> const redis = new Redis('redis://127.0.0.1:6379');
> redis.status
'connect'
> redis.set('fooo','barr').then(console.log).catch(console.error)
Promise { <pending> }
> redis.status
'connect'
Is there a way to let me do this with ioredis? This is just for debugging. If the first form is correct, is there a setting to allow "non-strict" validation of the cert or something?
This works (on a mac)
% openssl s_client -connect localhost:6379
set "fred" "Mary"
+OK
get "fred"
$4
Mary
This works (with redis installed via pip3)
#!/usr/bin/env python3
import redis
r = redis.Redis(host='127.0.0.1', ssl=True, port=6379)
r.set('foo', 'bar')
print(r.get('foo'))
While I wouldn't recommend this for production, you said this was for debugging.
You need to disable the server identity check. You can do that by overriding the function in the configuration with a noop:
const Redis = require("ioredis");
const redis = new Redis('rediss://127.0.0.1:6379', {
tls: {
checkServerIdentity: () => undefined,
}
});

dotnetcore console app : rabbitmq with docker Connection refused 127.0.0.1:5672

rabbit connection from console app :
var factory = new ConnectionFactory()
{
HostName = Environment.GetEnvironmentVariable("RabbitMq/Host"),
UserName = Environment.GetEnvironmentVariable("RabbitMq/Username"),
Password = Environment.GetEnvironmentVariable("RabbitMq/Password")
};
using (var connection = factory.CreateConnection()) // GETTING ERROR HERE
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: "rss",
durable: fa...
I'm getting this error :
Unhandled Exception:
RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the
specified endpoints were reachable --->
RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed
---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException:
Connection refused 127.0.0.1:5672
my docker-compose.yml file :
version: '3'
services:
message.api:
image: message.api
build:
context: ./message_api
dockerfile: Dockerfile
container_name: message.api
environment:
- "RabbitMq/Host=rabbit"
- "RabbitMq/Username=guest"
- "RabbitMq/Password=guest"
depends_on:
- rabbit
rabbit:
image: rabbitmq:3.7.2-management
hostname: rabbit
ports:
- "15672:15672"
- "5672:5672"
rsscomparator:
image: rsscomparator
build:
context: ./rss_comparator_app
dockerfile: Dockerfile
container_name: rsscomparator
environment:
- "RabbitMq/Host=rabbit"
- "RabbitMq/Username=guest"
- "RabbitMq/Password=guest"
depends_on:
- rabbit
I'm using dotnetcore console app. When I use this app in docker I'm getting error. I can reach rabbitmq web browser(http://192.168.99.100:15672) but app can not reach.
You are trying to connect from your container app to your rabbitmq app.
You try to achieve this with 127.0.0.1:5672 in your console app container.
But this is pointing to your localhost inside this container, and not to your localhost on your host.
You are deploying your services using the same docker-compose without specifying network settings which means they are all deployed inside the same docker bridge network. This will allow you to let the containers communicate with each other using their container or service names.
So try to connect to rabbit:5672 instead of 127.0.0.1:5672. This name will be translated to the container IP (172.xx.xx.xx) which means you'll create a private connection between your containers.

Unable to forward remote(ec2) port to the local port using ruby "net/ssh" gem in window environment

I am trying to forward the remote(ec2) port to local port to watch the jobtracker in my local browser which is running on amazon ec2 cluster. But unable to forward the remote port.
Here is my code:
Net::SSH.start('ec2-23-23-4-229.compute-1.amazonaws.com' ,'hadoop' ,:keys =>"mykeypair.pem",:forward_agent => true) do |ssh|
ssh.forward.remote(9033, "localhost",9101,"ec2-23-22-4-249.compute-1.amazonaws.com")
ssh.loop { !ssh.forward.active_remotes.include?([9033, "localhost"]) }
end
According to the original poster:
Found the solution. It might help someone:
ssh=Net::SSH.start(dns, 'hadoop' ,:keys => ["src/#{$parameters_hash['pem_file_name']}"],:forward_agent => true)
ssh.forward.local(port_available, 'localhost', 9100)