I am keen on using PouchDB in browser memory for an Angular application. This PouchDB will replicate from a remote LevelDB database that is fed key-value pairs from an algorithm. So, on the remote end, I would install PouchDB-Server. On the local end, I would do the following (as described here) on a node prompt.
var localDB = new PouchDB('mylocaldb')
var remoteDB = new PouchDB('https://remote-ip-address:5984/myremotedb')
localDB.sync(remoteDB, {
live: true
}).on('change', function (change) {
// yo, something changed!
}).on('error', function (err) {
// yo, we got an error! (maybe the user went offline?)
});
How do we start a PouchDB instance that supports TLS for live replication as described in the snippet above?
How do I start a PouchDB instance that supports TLS for live replication?
So after some more searching, it is clear from this topic, HTTPS is not supported for PocuhDB-Server.
Sorry, I misunderstood your question. I thought you intend to connect to a CouchDB server with PouchDB through HTTPS. Therefore, the following answer actually doesn't answer your question.
I created a server.js file like below to communicate with my CouchDB through HTTPS. Please note that the SSL certificate is (in my case) self-signed, and also CouchDB listens by default on port 6984 in the case of TLS:
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0"; // Ignore rejection, becasue CouchDB SSL certificate is self-signed
//import PouchDB from 'pouchdb'
const PouchDB = require('pouchdb')
const db = new PouchDB('https://admin:****#192.168.1.106:6984/reproduce')
db.allDocs({
include_docs: true,
attachments: false
}).then(function (result) {
// handle result
console.log(result)
}).catch(function (err) {
console.log(err);
});
I'm running the above file with $ node server.js and I'm getting the expected results:
$ node server.js
{ total_rows: 3,
offset: 0,
rows:
[ { id: '5d6590d3-41c7-4011-be5d-b21f80079ae5',
key: '5d6590d3-41c7-4011-be5d-b21f80079ae5',
value: [Object],
doc: [Object] },
{ id: 'ec6a36d1-952e-4d86-9865-3587c6079fb5',
key: 'ec6a36d1-952e-4d86-9865-3587c6079fb5',
value: [Object],
doc: [Object] },
{ id: 'f508e7aa-b4dc-42fc-96be-b7c1ffa54172',
key: 'f508e7aa-b4dc-42fc-96be-b7c1ffa54172',
value: [Object],
doc: [Object] } ] }
I created the above code with NodeJS on server-side. However, if you want to communicate with CouchDB through HTTPS inside the browser, i.e. on client-side, you have to enable CORS on CouchDB.
Related
Using #golevelup/nestjs-rabbitmq I tried the connection manager to not wait for a connection. According to the readme it can handle reconnections and wait for a connection without crashing the app. However, when I use the connectionInitOptions as stated and set wait to false, I get a connection error. When I don't use it (default behavior setting wait to true) , it connects to the RabbitMQ server. Below are examples importing the RabbitMQModule in a NestJS module.
This works and connects to the RabbitMQ server
RabbitMQModule.forRoot(RabbitMQModule, {
exchanges: [{ type: 'topic', name: 'main' }],
uri: 'amqp://guest:guest#localhost:5672',
}
This doesn't work and won't connect
RabbitMQModule.forRoot(RabbitMQModule, {
exchanges: [{ type: 'topic', name: 'main' }],
uri: 'amqp://guest:guest#localhost:5672',
connectionInitOptions: {
wait: false,
},
With the second option I get the following error:
Error: AMQP connection is not available
at AmqpConnection.publish (/home/xxx/node_modules/#golevelup/nestjs-rabbitmq/src/amqp/connection.ts:424:13)
at BootstrapService.onApplicationBootstrap (/home/xxx/src/bootstrap/bootstrap.service.ts:20:25)
at MapIterator.iteratee (/home/xxx/node_modules/#nestjs/core/hooks/on-app-bootstrap.hook.js:22:43)
at MapIterator.next (/home/xxx/node_modules/iterare/src/map.ts:9:39)
at IteratorWithOperators.next (/home/xxx/node_modules/iterare/src/iterate.ts:19:28)
at Function.from (<anonymous>)
at IteratorWithOperators.toArray (/home/xxx/node_modules/iterare/src/iterate.ts:227:22)
at callOperator (/home/xxx/node_modules/#nestjs/core/hooks/on-app-bootstrap.hook.js:23:10)
at callModuleBootstrapHook (/home/xxx/node_modules/#nestjs/core/hooks/on-app-bootstrap.hook.js:43:23)
at NestApplication.callBootstrapHook (/home/xxx/node_modules/#nestjs/core/nest-application-context.js:199:55)
at NestApplication.init (/home/xxx/node_modules/#nestjs/core/nest-application.js:98:9)
at NestApplication.listen (/home/xxx/node_modules/#nestjs/core/nest-application.js:155:33)
at bootstrap (/home/xxx/src/main.ts:12:3)
The last line (main.ts:12:3) is the app.listen(3000) statement.
There are other options you can set with the connectionInitOptions (reject and timeout) and I've tried the combinations but still no connection.
RabbitMQ is running in a docker container on Linux but that should be no problem. I posted the same question on NestJS discord but got no reply, so hopefully someone on SO has an idea.
Any idea what could be the cause?
Found the problem, I was using the connection in a onApplicationBootstrap method and then the connection is apparently not present yet.
you can wait for connection asynchronously 'onApplicationBootstrap':
or on :
async onModuleInit() {
await this.amqpConnection.managedChannel.waitForConnect(async () => {
await this.assertQueueAndBindToExchange(
transferRequestQueueName,
transferRequestExchangeName,
createdRoutingKey
);
I'm using nodeJS to connect my a Digital Ocean droplet (Ubuntu 20.04) to a Digital Ocean managed Redis instance. I'm using the ioredis npm library.
Consider the simple trivial code below. This code works perfectly with the public network name albeit taking around 400ms. If I use the private network name the entire script hangs. I've also tried the private IP 10...* but that doesn't work either.
Does anyone have any experience here or insight as to how to connect directly with the VPC? Is there a specific way to use the private network name?
const Redis = require("ioredis");
(async () => {
// Spin up a redis client
const redis = new Redis({
host: "db-redis-**************-0.b.db.ondigitalocean.com",
port: *****,
username: "******",
password: "**********",
tls: {
key: "",
cert: "",
},
});
console.time("Total time to write/read a 10 character string to redis");
// Generate a random string
const generateRandomString = (length = 6) =>
Math.random().toString(20).substr(2, length);
// Save data to the redis server with a TTL of 2 miniutes
redis.set("redisTest", generateRandomString(10), "EX", 120);
// Now read it back
await redis.get("redisTest", function (err, result) {
if (err) {
console.error(err);
} else {
console.log("Data retrieved: ", result);
}
});
// Done
console.log("Done.");
console.timeEnd("Total time to write/read a 10 character string to redis");
})();
If using the private network address hangs during opening the connection, it's likely because your Droplet is not in the same VPC as your Redis database. In your case, it turned out that the Droplet and Redis were in different regions, so moving them to the same region (and ensuring they're in the same VPC within that region) should resolve the issue.
I am trying to put a value in Infinispan cache using Hotrod nodeJS client. The code runs fine if the server is installed locally. However, when I run the same code with Infinispan server hosted on docker container I get the following error
java.lang.SecurityException: ISPN006017: Unauthorized 'PUT' operation
try {
client = await infinispan.client({
port: 11222,
host: '127.0.0.1'
}, {
cacheName: 'testcache'
});
console.log(`Connected to cache`);
await client.put('test', 'hello 1');
await client.disconnect();
} catch (e) {
console.log(e);
await client.disconnect();
}
I have tried setting CORS Allow all option on the server as well
Need to provide custom config.yaml to docker with following configurations
endpoints:
hotrod:
auth: false
enabled: false
qop: auth
serverName: infinispan
Unfortunately the nodejs client doesn't support authentication yet. The issue to implement this is https://issues.redhat.com/projects/HRJS/issues/HRJS-36
I am trying to setup AWS API Gateway to access a fargate container in a private VPC as described here. For this I am using AWS CDK as described below. But when I curl the endpoint after successful cdk deploy I get "Internal Server Error" as a response. I can't find any additional information. For some reason API GW can't reach the container.
So when I curl the endpoint like this:
curl - i https://xxx.execute-api.eu-central-1.amazonaws.com/prod/MyResource
... I get the following log output in cloud watch:
Extended Request Id: NpuEPFWHliAFm_w=
Verifying Usage Plan for request: 757c6b9e-c4af-4dab-a5b1-542b15a1ba21. API Key: API Stage: ...
PI Key authorized because method 'ANY /MyResource/{proxy+}' does not require API Key. Request will not contribute to throttle or quota limits
Usage Plan check succeeded for API Key and API Stage ...
Starting execution for request: 757c6b9e-c4af-4dab-a5b1-542b15a1ba21
HTTP Method: GET, Resource Path: /MyResource/test
Execution failed due to configuration error: There was an internal error while executing your request
CDK Code
First I create a network load balanced fargate service:
private setupService(): NetworkLoadBalancedFargateService {
const vpc = new Vpc(this, 'MyVpc');
const cluster = new Cluster(this, 'MyCluster', {
vpc: vpc,
});
cluster.connections.allowFromAnyIpv4(Port.tcp(5050));
const taskDefinition = new FargateTaskDefinition(this, 'MyTaskDefinition');
const container = taskDefinition.addContainer('MyContainer', {
image: ContainerImage.fromRegistry('vad1mo/hello-world-rest'),
});
container.addPortMappings({
containerPort: 5050,
hostPort: 5050,
});
const service = new NetworkLoadBalancedFargateService(this, 'MyFargateServie', {
cluster,
taskDefinition,
assignPublicIp: true,
});
service.service.connections.allowFromAnyIpv4(Port.tcp(5050));
return service;
}
Next I create the VpcLink and the API Gateway:
private setupApiGw(service: NetworkLoadBalancedFargateService) {
const api = new RestApi(this, `MyApi`, {
restApiName: `MyApi`,
deployOptions: {
loggingLevel: MethodLoggingLevel.INFO,
},
});
// setup api resource which forwards to container
const resource = api.root.addResource('MyResource');
resource.addProxy({
anyMethod: true,
defaultIntegration: new HttpIntegration('http://localhost.com:5050', {
httpMethod: 'ANY',
options: {
connectionType: ConnectionType.VPC_LINK,
vpcLink: new VpcLink(this, 'MyVpcLink', {
targets: [service.loadBalancer],
vpcLinkName: 'MyVpcLink',
}),
},
proxy: true,
}),
defaultMethodOptions: {
authorizationType: AuthorizationType.NONE,
},
});
resource.addMethod('ANY');
this.addCorsOptions(resource);
}
Anyone has a clue what is wrong with this config?
After hours of trying I finally figured out that the security groups do not seem to be updated correctly when setting up the VpcLink with CDK. Broadening the allowed connection with
service.service.connections.allowFromAnyIpv4(Port.allTraffic())
solved it. Still need to figure out which minimum set needs to be set instead of allTrafic()
Additionally I replaced localhost in the HttpIntegration by the endpoint of the load balancer like this:
resource.addMethod("ANY", new HttpIntegration(
'http://' + service.loadBalancer.loadBalancerDnsName,
{
httpMethod: 'ANY',
options: {
connectionType: ConnectionType.VPC_LINK,
vpcLink: new VpcLink(this, 'MyVpcLink', {
targets: [service.loadBalancer],
vpcLinkName: 'MyVpcLink',
})
},
}
))
I am trying to use Socket.IO in Node.js, and am trying to allow the server to give an identity to each of the Socket.IO clients. As the socket code is outside the scope of the http server code, it doesn't have easy access to the request information sent, so I'm assuming it will need to be sent up during the connection. What is the best way to
1) get the information to the server about who is connecting via Socket.IO
2) authenticate who they say they are (I'm currently using Express, if that makes things any easier)
Use connect-redis and have redis as your session store for all authenticated users. Make sure on authentication you send the key (normally req.sessionID) to the client. Have the client store this key in a cookie.
On socket connect (or anytime later) fetch this key from the cookie and send it back to the server. Fetch the session information in redis using this key. (GET key)
Eg:
Server side (with redis as session store):
req.session.regenerate...
res.send({rediskey: req.sessionID});
Client side:
//store the key in a cookie
SetCookie('rediskey', <%= rediskey %>); //http://msdn.microsoft.com/en-us/library/ms533693(v=vs.85).aspx
//then when socket is connected, fetch the rediskey from the document.cookie and send it back to server
var socket = new io.Socket();
socket.on('connect', function() {
var rediskey = GetCookie('rediskey'); //http://msdn.microsoft.com/en-us/library/ms533693(v=vs.85).aspx
socket.send({rediskey: rediskey});
});
Server side:
//in io.on('connection')
io.on('connection', function(client) {
client.on('message', function(message) {
if(message.rediskey) {
//fetch session info from redis
redisclient.get(message.rediskey, function(e, c) {
client.user_logged_in = c.username;
});
}
});
});
I also liked the way pusherapp does private channels.
A unique socket id is generated and
sent to the browser by Pusher. This is
sent to your application (1) via an
AJAX request which authorizes the user
to access the channel against your
existing authentication system. If
successful your application returns an
authorization string to the browser
signed with you Pusher secret. This is
sent to Pusher over the WebSocket,
which completes the authorization (2)
if the authorization string matches.
Because also socket.io has unique socket_id for every socket.
socket.on('connect', function() {
console.log(socket.transport.sessionid);
});
They used signed authorization strings to authorize users.
I haven't yet mirrored this to socket.io, but I think it could be pretty interesting concept.
I know this is bit old, but for future readers in addition to the approach of parsing cookie and retrieving the session from the storage (eg. passport.socketio ) you might also consider a token based approach.
In this example I use JSON Web Tokens which are pretty standard. You have to give to the client page the token, in this example imagine an authentication endpoint that returns JWT:
var jwt = require('jsonwebtoken');
// other requires
app.post('/login', function (req, res) {
// TODO: validate the actual user user
var profile = {
first_name: 'John',
last_name: 'Doe',
email: 'john#doe.com',
id: 123
};
// we are sending the profile in the token
var token = jwt.sign(profile, jwtSecret, { expiresInMinutes: 60*5 });
res.json({token: token});
});
Now, your socket.io server can be configured as follows:
var socketioJwt = require('socketio-jwt');
var sio = socketIo.listen(server);
sio.set('authorization', socketioJwt.authorize({
secret: jwtSecret,
handshake: true
}));
sio.sockets
.on('connection', function (socket) {
console.log(socket.handshake.decoded_token.email, 'has joined');
//socket.on('event');
});
The socket.io-jwt middleware expects the token in a query string, so from the client you only have to attach it when connecting:
var socket = io.connect('', {
query: 'token=' + token
});
I wrote a more detailed explanation about this method and cookies here.
Here is my attempt to have the following working:
express: 4.14
socket.io: 1.5
passport (using sessions): 0.3
redis: 2.6 (Really fast data structure to handle sessions; but you can use others like MongoDB too. However, I encourage you to use this for session data + MongoDB to store other persistent data like Users)
Since you might want to add some API requests as well, we'll also use http package to have both HTTP and Web socket working in the same port.
server.js
The following extract only includes everything you need to set the previous technologies up. You can see the complete server.js version which I used in one of my projects here.
import http from 'http';
import express from 'express';
import passport from 'passport';
import { createClient as createRedisClient } from 'redis';
import connectRedis from 'connect-redis';
import Socketio from 'socket.io';
// Your own socket handler file, it's optional. Explained below.
import socketConnectionHandler from './sockets';
// Configuration about your Redis session data structure.
const redisClient = createRedisClient();
const RedisStore = connectRedis(Session);
const dbSession = new RedisStore({
client: redisClient,
host: 'localhost',
port: 27017,
prefix: 'stackoverflow_',
disableTTL: true
});
// Let's configure Express to use our Redis storage to handle
// sessions as well. You'll probably want Express to handle your
// sessions as well and share the same storage as your socket.io
// does (i.e. for handling AJAX logins).
const session = Session({
resave: true,
saveUninitialized: true,
key: 'SID', // this will be used for the session cookie identifier
secret: 'secret key',
store: dbSession
});
app.use(session);
// Let's initialize passport by using their middlewares, which do
//everything pretty much automatically. (you have to configure login
// / register strategies on your own though (see reference 1)
app.use(passport.initialize());
app.use(passport.session());
// Socket.IO
const io = Socketio(server);
io.use((socket, next) => {
session(socket.handshake, {}, next);
});
io.on('connection', socketConnectionHandler);
// socket.io is ready; remember that ^this^ variable is just the
// name that we gave to our own socket.io handler file (explained
// just after this).
// Start server. This will start both socket.io and our optional
// AJAX API in the given port.
const port = 3000; // Move this onto an environment variable,
// it'll look more professional.
server.listen(port);
console.info(`🌐 API listening on port ${port}`);
console.info(`🗲 Socket listening on port ${port}`);
sockets/index.js
Our socketConnectionHandler, I just don't like putting everything inside server.js (even though you perfectly could), especially since this file can end up containing quite a lot of code pretty quickly.
export default function connectionHandler(socket) {
const userId = socket.handshake.session.passport &&
socket.handshake.session.passport.user;
// If the user is not logged in, you might find ^this^
// socket.handshake.session.passport variable undefined.
// Give the user a warm welcome.
console.info(`⚡︎ New connection: ${userId}`);
socket.emit('Grettings', `Grettings ${userId}`);
// Handle disconnection.
socket.on('disconnect', () => {
if (process.env.NODE_ENV !== 'production') {
console.info(`⚡︎ Disconnection: ${userId}`);
}
});
}
Extra material (client):
Just a very basic version of what the JavaScript socket.io client could be:
import io from 'socket.io-client';
const socketPath = '/socket.io'; // <- Default path.
// But you could configure your server
// to something like /api/socket.io
const socket = io.connect('localhost:3000', { path: socketPath });
socket.on('connect', () => {
console.info('Connected');
socket.on('Grettings', (data) => {
console.info(`Server gretting: ${data}`);
});
});
socket.on('connect_error', (error) => {
console.error(`Connection error: ${error}`);
});
References:
I just couldn't reference inside the code, so I moved it here.
1: How to set up your Passport strategies: https://scotch.io/tutorials/easy-node-authentication-setup-and-local#handling-signupregistration
This article (http://simplapi.wordpress.com/2012/04/13/php-and-node-js-session-share-redi/) shows how to
store sessions of the HTTP server in Redis (using Predis)
get these sessions from Redis in node.js by the session id sent in a cookie
Using this code you are able to get them in socket.io, too.
var io = require('socket.io').listen(8081);
var cookie = require('cookie');
var redis = require('redis'), client = redis.createClient();
io.sockets.on('connection', function (socket) {
var cookies = cookie.parse(socket.handshake.headers['cookie']);
console.log(cookies.PHPSESSID);
client.get('sessions/' + cookies.PHPSESSID, function(err, reply) {
console.log(JSON.parse(reply));
});
});
use session and Redis between c/s
Server side
io.use(function(socket, next) {
// get here session id
console.log(socket.handshake.headers.cookie); and match from redis session data
next();
});
this should do it
//server side
io.sockets.on('connection', function (con) {
console.log(con.id)
})
//client side
var io = io.connect('http://...')
console.log(io.sessionid)