How to use `ioredis` to connect to Redis instance (AWS elasticcache) across ssh tunnel with SSL? - ssl

This seems to be something about ioredis and its support for TLS. This is all on a mac, Catalina, etc.
I have an elasticcache Redis instance running, inside a VPC. I tunnel to it with ssh,
ssh -L 6379:clustercfg.my-test-redis.amazonaws.com:6379 -N MyEC2
The following doesn't work with node 12.9, ioredis 4.19.4
> const Redis = require("ioredis");
> const redis = new Redis('rediss://127.0.0.1:6379');
[ioredis] Unhandled error event: Error [ERR_TLS_CERT_ALTNAME_INVALID]: Hostname/IP does not match certificate's altnames: IP: 127.0.0.1 is not in the cert's list:
at Object.checkServerIdentity (tls.js:287:12)
<repeated ... many times>
This doesn't work either:
> const Redis = require("ioredis");
> const redis = new Redis('redis://127.0.0.1:6379');
> redis.status
'connect'
> redis.set('fooo','barr').then(console.log).catch(console.error)
Promise { <pending> }
> redis.status
'connect'
Is there a way to let me do this with ioredis? This is just for debugging. If the first form is correct, is there a setting to allow "non-strict" validation of the cert or something?
This works (on a mac)
% openssl s_client -connect localhost:6379
set "fred" "Mary"
+OK
get "fred"
$4
Mary
This works (with redis installed via pip3)
#!/usr/bin/env python3
import redis
r = redis.Redis(host='127.0.0.1', ssl=True, port=6379)
r.set('foo', 'bar')
print(r.get('foo'))

While I wouldn't recommend this for production, you said this was for debugging.
You need to disable the server identity check. You can do that by overriding the function in the configuration with a noop:
const Redis = require("ioredis");
const redis = new Redis('rediss://127.0.0.1:6379', {
tls: {
checkServerIdentity: () => undefined,
}
});

Related

Encountering "Error: connect ECONNREFUSED 127.0.0.1:6379..." when running Google Cloud Function connecting to a Redis Instance

Im running a function to ingest data to a Redis instance. However, I am encountering an issue where I can't seem to connect to the client.
Here is the part of the code where it connects to the server.
const redis = require('redis');
require('dotenv').config;
const REDISHOST = process.env.REDISHOST;
const REDISPORT = 6379;
const REDISAUTH = process.env.AUTHSTRING;
const client = redis.createClient({
port: REDISPORT,
host: REDISHOST,
password: REDISAUTH
});
await client.connect();
Here is the whole error message:
"Error: connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16)
at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17)"
Any help would be appreciated. Thanks
I have tried using ioredis instead of redis but it shows a different error altogether.
It's trying to connect to localhost and Redis doesn't exist there. IF not given a hostname, this is the default behavior of Redis.
Looking at your code, this is probably because the REDISHOST environment variable isn't defined.

Terraform - Failed to set up SSH tunneling for host

Hell, I am trying to deploy rke k8s with terraform, but I am not able to connect to the desired host via ssh:
time="2022-02-28T11:17:38+01:00" level=warning msg="Failed to set up SSH tunneling for host [poc-k8s.my-domain.com]: Can't retrieve Docker Info: error during connect: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info\": Unable to access node with address [poc-k8s.my-domain.com:22] using SSH. Please check if you are able to SSH to the node using the specified SSH Private Key and if you have configured the correct SSH username. Error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain"
and this is the .tf file I am using:
terraform {
required_providers {
rke = {
source = "rancher/rke"
version = "1.3.0"
}
}
}
provider "rke" {
log_file = "rke_debug.log"
}
resource "rke_cluster" "cluster" {
nodes {
address = "poc-k8s.my-domain.com"
user = "root"
role = ["controlplane", "worker", "etcd"]
ssh_key = file("~/.ssh/root_key")
}
nodes {
address = "poc-k8s.my-domain.com"
user = "root"
role = ["worker", "etcd"]
ssh_key = file("~/.ssh/root_key")
}
addons_include = [
"https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml",
"https://gist.githubusercontent.com/superseb/499f2caa2637c404af41cfb7e5f4a938/raw/930841ac00653fdff8beca61dab9a20bb8983782/k8s-dashboard-user.yml",
]
}
resource "local_file" "kube_cluster_yaml" {
filename = "~/.kube/kube_config_cluster.yml"
sensitive_content = "rke_cluster.cluster.kube_config_yaml"
}
The key if of course correct and I am able to connect to the desired host:
ssh -i ~/.ssh/root_key root#poc-k8s.my-domain.com
what am I missing here?
[Update]
Cluster resource has delay_on_creation property that can be used
resource "rke_cluster" "cluster" {
delay_on_creation = 180
(...)
}
I'm facing a similar issue. On the second run of terrafor apply it works correctly. In my case the issue is that docker is not up fast enough for RKE provider.
I've found following workaround from citynetwork /
citycloud-examples:
resource "rke_cluster" "cluster" {
(...)
depends_on = [null_resource.wait-for-docker]
}
resource "null_resource" "wait-for-docker" {
provisioner "local-exec" {
command = "sleep 180"
}
depends_on = [
# list of servers docker being installed on
(...)
]
}
It waits for 180s which is not ideal, though.

Redis performance with a Digital Ocean managed instance

I'm using nodeJS to connect my a Digital Ocean droplet (Ubuntu 20.04) to a Digital Ocean managed Redis instance. I'm using the ioredis npm library.
Consider the simple trivial code below. This code works perfectly with the public network name albeit taking around 400ms. If I use the private network name the entire script hangs. I've also tried the private IP 10...* but that doesn't work either.
Does anyone have any experience here or insight as to how to connect directly with the VPC? Is there a specific way to use the private network name?
const Redis = require("ioredis");
(async () => {
// Spin up a redis client
const redis = new Redis({
host: "db-redis-**************-0.b.db.ondigitalocean.com",
port: *****,
username: "******",
password: "**********",
tls: {
key: "",
cert: "",
},
});
console.time("Total time to write/read a 10 character string to redis");
// Generate a random string
const generateRandomString = (length = 6) =>
Math.random().toString(20).substr(2, length);
// Save data to the redis server with a TTL of 2 miniutes
redis.set("redisTest", generateRandomString(10), "EX", 120);
// Now read it back
await redis.get("redisTest", function (err, result) {
if (err) {
console.error(err);
} else {
console.log("Data retrieved: ", result);
}
});
// Done
console.log("Done.");
console.timeEnd("Total time to write/read a 10 character string to redis");
})();
If using the private network address hangs during opening the connection, it's likely because your Droplet is not in the same VPC as your Redis database. In your case, it turned out that the Droplet and Redis were in different regions, so moving them to the same region (and ensuring they're in the same VPC within that region) should resolve the issue.

Masstransit cannot access host machine RabbitMQ from a docker container

I created a simple .net core console application with docker support. Following
Masstransit code fails to connect to RabbitMQ instance on host machine. But similar implementation using RabitMq.Client is able to connect to host machine RabbitMQ instance.
Masstransit throws
MassTransit.RabbitMqTransport.RabbitMqConnectionException: Connect
failed: ctas#192.168.0.9:5672/ --->
RabbitMQ.Client.Exceptions.BrokerUnreachableException:
host machine ip : 192.168.0.9
using Masstransit
string rabbitMqUri = "rabbitmq://192.168.0.9/";
string userName = "ctas";
string password = "ctas#123";
string assetServiceQueue = "hello";
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(rabbitMqUri), hst =>
{
hst.Username(userName);
hst.Password(password);
});
cfg.ReceiveEndpoint(host,
assetServiceQueue, e =>
{
e.Consumer<AddNewAssetReceivedConsumer>();
});
});
bus.Start();
Console.WriteLine("Service Running.... Press enter to exit");
Console.ReadLine();
bus.Stop();
Using RabbitMQ Client
public static void Main()
{
var factory = new ConnectionFactory();
factory.UserName = "ctas";
factory.Password = "ctas#123";
factory.VirtualHost = "watcherindustry";
factory.HostName = "192.168.0.9";
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: "hello",
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
};
channel.BasicConsume(queue: "hello",
autoAck: true,
consumer: consumer);
Console.WriteLine(" Press [enter] to exit.");
Console.ReadLine();
}
}
Docker file
FROM microsoft/dotnet:1.1-runtime
ARG source
WORKDIR /app
COPY ${source:-obj/Docker/publish} .
ENTRYPOINT ["dotnet", "TestClient.dll"]
I created an example, and was able to connect my host, using the preview package from masstransit.
Start rabbitmq in docker and expose ports on the host
docker run -d -p 5672:5672 -p 15672:15672 --hostname my-rabbit --name some-rabbit rabbitmq:3-management
Build and run console app.
docker build -t dotnetapp .
docker run -d -e RABBITMQ_URI=rabbitmq://guest:guest#172.17.0.2:5672 --name some-dotnetapp dotnetapp
To verify your receiving messages run
docker logs some-dotnetapp --follow
you should see the following output
Application is starting...
Connecting to rabbitmq://guest:guest#172.17.0.2:5672
Received: Hello, World [08/12/2017 04:35:53]
Received: Hello, World [08/12/2017 04:35:58]
Received: Hello, World [08/12/2017 04:36:03]
Received: Hello, World [08/12/2017 04:36:08]
Received: Hello, World [08/12/2017 04:36:13]
...
Notes:
172.17.0.2 was my-rabbit container ip address but you can replace it with your machine ip address
http://localhost:15672 is the rabbitmq management console log in with guest as username and password.
Lastly portainer.io is a very useful application to visually view you local docker environment.
Thanks for the response. I managed to resolve this issue. My findings are as follows.
to connect to a rabbitmq instance on another docker container, they have to be moved/connected to the same network. To do this
create a newtork
docker network create -d bridge my_bridge
connect both app and rabbitmq containers to same network
docker network connect my_bridge <container name>
For masstransit uri use rabbitmq container IP on that network or container name
To connect rabbitmq instance of host machine from a app on docker container.
masstransit uri should include machine name( I tried IP, that did not work)
Try using virtual host in MassTransit configuration too, not sure why you decided to omit it.
var host = cfg.Host("192.168.0.9", "watcherindustry", hst =>
{
hst.Username(userName);
hst.Password(password);
});
Look at Alexey Zimarev comment to your question, if your rabbit runs on a container then it should be on your docker-compese file and then use that entry in your endpoint definition to connect to rabbit because docker creates an internal network on which you are agnostic from source code...
rabbitmq:
container_name: "rabbitmq-yournode01"
hostname: rabbit
image: rabbitmq:3.6.6-management
environment:
- RABBITMQ_DEFAULT_USER=yourusergoeshere
- RABBITMQ_DEFAULT_PASS=yourpasswordgoeshere
- RABBITMQ_DEFAULT_VHOST=vhost
volumes:
- rabbit-volume:/var/lib/rabbitmq
ports:
- "5672:5672"
- "15672:15672"
In your app settings you should have something lie:
"ConnectionString": "host=rabbitmq:5672;virtualHost=vhost;username=yourusergoeshere;password=yourpasswordgoeshere;timeout=0;prefetchcount=1",
And if you'd use EasyNEtQ you could do:
_bus = RabbitHutch.CreateBus(_connectionString); // The one above
I hope it helps,
Juan

Cannot have file provisioner working with Terraform on DigitalOcean

I try to use Terraform to create a DigitalOcean node on which consul is installed.
I'm using the following .tf file but it hangs up and do not copy the consul .zip file onto the droplet.
I got the following error message after a couple of minutes:
ssh: handshake failed: ssh: unable to authenticate, attempted methods
[none publickey], no supported methods remain
The droplets are correctly created though. I can login on command line with the key I specified (thus not specifying password). I'm guessing the connection part might be faulty but not sure what I'm missing.
Any idea ?
variable "do_token" {}
# Configure the DigitalOcean Provider
provider "digitalocean" {
token = "${var.do_token}"
}
# Create nodes
resource "digitalocean_droplet" "consul" {
count = "1"
image = "ubuntu-14-04-x64"
name = "consul-${count.index+1}"
region = "lon1"
size = "1gb"
ssh_keys = ["7b:51:d3:e3:ae:6e:c6:e2:61:2d:40:56:17:54:fc:e3"]
connection {
type = "ssh"
user = "root"
agent = true
}
provisioner "file" {
source = "consul_0.7.1_linux_amd64.zip"
destination = "/tmp/consul_0.7.1_linux_amd64.zip"
}
provisioner "remote-exec" {
inline = [
"sudo unzip -d /usr/local/bin /tmp/consul_0.7.1_linux_amd64.zip"
]
}
}
Terraform requires that you specify the private SSH key to use for the connection with private_key You can create a new variable containing the path to your private key for use with Terraform's file interpolation function:
connection {
type = "ssh"
user = "root"
agent = true
private_key = "${file("${var.private_key_path}")}"
}
You face this issue, because you have a ssh key protected by a password. To solve this issue you should generate a key without password.