Redis issue on module-redis-fork - redis

Issue
Hi everyone,
I have an issue while trying to interact with Redis in those conditions:
Redis instance with Redisearch module,
Create node-redis client before Redis module fork is ongoing,
Redis module fork is on-going
The behaviour that I get is that "send_command" stays idle until the fork stops.
When the fork ends I get this error:
debug mode ->
Redis connection is gone from end event
client error ->
AbortError: Redis connection lost and command aborted. It might have been processed.
After I get this error the commands from the same client (without creating a new client) come back to works fine.
On every fork, I got the same behaviour.
Additional Info:
keys: 37773168,
used_memory_human: '87.31G'
Code Example:
This is a simple express app,
'use strict';
const express = require('express');
const Redis = require('redis');
// Redis.debug_mode = true;
const router = express.Router();
let client = null;
router.get('/redisearch/connect', async (req, res, next) => {
const conf = {
'host': '127.0.0.1',
'port': 6379,
'db': 0,
};
try {
if (!client) client = Redis.createClient(conf.port, conf.host, { db: conf.db });
res.send('Connected');
} catch (err) {
res.send(err);
}
});
router.get('/redisearch/d', async (req, res, next) => {
const num = 10;
const dArgs = ['testIndexName', `#ic:[${num} ${num}]`, 'GROUPBY', 1, '#d'];
try {
client.send_command('FT.AGGREGATE', dArgs, (err, reply) => {
if (err) {
res.send({ err: err });
};
res.send({ d: reply });
});
} catch (err) {
res.send(err);
}
});
module.exports = router;
this is the simplest way I have to replicate the problem.
I don't know if there is a way to force redis to use the fork, in my case it appears following a massive search on index followed by delete and insert of records.
Redis however during these operations (insert/delete) works normally,
I can launch commands from the redis-cli;
By creating a new instance of the node-redis client while the fork is present everything works normally and when the fork goes away everything keep working.
Environment
Node.js Version: v14.15.1
Redis Version: 6.0.4
redisearch Version: 1.6.15
node-redis Version: 3.2
Platform: Server 128GB RAM, 8 Core, Debian

Related

Is there a way to multi browser communicate each other without peers?still able to communicate after lose peers connecting?

Is there a way to multi browser communicate each other without peers?or still able to communicate after lose peers connecting?
I created sample with gun.js like below:
server.js:
const express = require('express')
const Gun = require('gun')
const app = express()
const port = 8000
app.use(Gun.serve)
const server = app.listen(port, () => {
console.log("Listening at: http://localhost://" + port)
})
Gun({web: server})
test.ts on angular demo:
gun = GUN({
peers: ['http:localhost:8000/gun']
});
data: any;
initDate(): void {
this.gun.get('mark').put({
name: "Mark",
email: "mark#gun.eco",
});
}
listenDate(): void {
this.gun.get('mark').on((data, key) => {
console.log("realtime updates:", data);
this.data = data;
});
}
submit(): void {
this.gun.get('mark').get('live').put(Math.random());
}
I start server.js as a peer and start angular app,open two broswer with same url,the two broswer communicate well.
but after i stop server.js , the two broswer are unable to communicate each other.
Is there a way to the two browser communicate each other without server.js?or how still able to communicate after I stop server.js?

Sveltekit development with workers KV -- hot reloading

Is it possible to use CloudFlare's Workers KV when developing a Svelte/kit application?
It is possible to build the app then run wrangler dev when using the CloudFlare Workers adapter:
npm build
wrangler dev
However, I haven't gotten hot module reloading working:
npm dev & wrangler dev
As far as I know, there's no way to emulate Workers KV locally. However, I setup a local Redis instance as a substitute.
Then, I created some wrapper functions for the KV store. In development, it talks to Redis, and in production it talks to Workers KV. For instance, here's the wrapper function for get.
import { dev } from '$app/env'
import redis from 'redis'
const client = redis.createClient()
const get = promisify(client.get).bind(client)
export const getKvValue = async (key: string): Promise<string | null> => {
return dev ? await get(key) : await KV.get(key)
}
Update: You can actually make things much simpler by just using an object in JavaScript—no need to download and run a Redis binary. Just make sure to JSON.stringify the values before setting them.
import { dev } from '$app/env'
const devKvStore = {}
const devGetKvValue = (key: string) => {
return new Promise((resolve) => {
resolve(devKvStore[key] ?? null)
})
}
const devSetKvValue = (key: string, value: unknown) => {
return new Promise((resolve) => {
devKvStore[key] = JSON.stringify(value)
resolve()
})
}
export const getKvValue = async (key: string): Promise<string | null> => {
return dev ? await devGetKvValue(key) : await KV.get(key)
}
export const setKvValue = async (key: string, value: unknown): Promise<void> => {
return dev ? await devSetKvValue(key, value) : await KV.put(key, value)
}

502 Bad Gateway in Kubernetes cluster after I added mongoDB connection code

I was chugging along with my Kubernetes cluster project when after creating a User model to start creating users in my application, I get a 502 Bad Gateway error in my Postman client.
So I was so focused on my ingress-nginx yaml file, staring at it for typos, rewriting it, uninstalling, reinstalling and still getting that error, that I decided to take it the next step further.
Via the current user route handler:
import express from "express";
const router = express.Router();
router.get("/api/users/currentuser", (req, res) => {
res.send("howdy!");
});
export { router as currentUserRouter };
I have always been able to go to my browser and successfully see howdy! rendered when I went to mywebsite.com/api/users/currentuser
but then I added some logic to index.ts file I did not particular care for from the https://expressjs.com/en/guide/routing.html:
app.all("*", async (req, res) => {
throw new NotFoundError();
});
Well, sure enough that killed my ability to go to mywebsite.com/api/users/currentuser and see howdy! rendered and instead I was getting a 502 Bad Gateway. So I said okay I will just leave that one out then.
But then I noticed a huge chunk of very important code was breaking my ability to visit that url as well:
// const start = async () => {
// try {
// await mongooose.connect("mongodb://auth-mongo-srv:27017/auth", {
// useNewUrlParser: true,
// useUnifiedTopology: true,
// useCreateIndex: true,
// });
// console.log("Connected to MongoDB");
// } catch (error) {
// app.listen(3000, () => {
// console.log("Listening on port 3000!!!!!");
// });
// }
// };
// start();
All of the above is what I need to connect to my local MongoDB server and start creating users.
So I started to even get more granular and slowly commenting code back in. Well, the app.all() is not a problem anymore, the problem seems to be throwing my mongoDB connection code inside of a try/catch statement, but I have no idea why that would have created the problem. Any ideas anyone?
So instead if I just run it like this:
const start = async () => {
await mongooose.connect("mongodb://auth-mongo-srv:27017/auth", {
useNewUrlParser: true,
useUnifiedTopology: true,
useCreateIndex: true,
});
console.log("Connected to MongoDB");
app.listen(3000, () => {
console.log("Listening on port 3000!!!!!");
});
};
start();
It all works fine again.

How do I mock redis when using supertest in Express?

I am building a REST API in Express and I'm trying to mock Redis in my Jasmine unit tests (using redis-mock).
If supertest is making a request to the API, how do I tell my app to use the mock Redis instead of the actual redis? .I know I'll probably need to create separate modules for redis that I can swap out somehow, just not sure how to swap it out for a regular request vs a supertest request.
Unit test:
describe('Instance API v1', () => {
it('returns a list of instances', (done) => {
request(app.default)
.get('/v1/instance')
.set('Authorization', 'Bearer '+authToken)
.expect(200)
.expect('Content-Type', 'application/json; charset=utf-8')
.end((error) => (error) ? done.fail(error) : done());
});
});
Route handler:
getAll = (request: express.Request, response: express.Response) => {
let redis: RedisClient = request.app.locals.redisclient;
let keys:string[] = [];
let prefix = 'instance/*';
const scanner = new RedisScan(redis);
scanner.scan('instances/*', (err, matchingKeys) => {
if (err) throw(err);
// matchingKeys will be an array of strings if matches were found
// otherwise it will be an empty array.
console.log(matchingKeys);
response.json(matchingKeys);
});
};
in my previous experience, I didn't mock Redis in integration test so I can test the flow in full functionality.
If you want to mock the Redis, you must do it before you require and initiate your application in the test something like:
before(function() {
const matchingKeys = '1234';
sinon.stub(RedisScan.prototype, 'scan').yields(null, matchingKeys);
const app = require('./app');
return app.init(); // your init or whatever function to initiate your express app
});
Hope it helps

How to run Gun server with Hapi?

I follow this tutorial to create Gun server. But I need to do it with Hapi.
Now, I get the following error:
> node server.js
Hello wonderful person! :) Thanks for using GUN, feel free to ask for help on https://gitter.im/amark/gun and ask StackOverflow questions tagged with 'gun'!
0.8 WARNING! Breaking changes, test that your app works before upgrading! The adapter interface has been upgraded (non-default storage and transport layers probably won't work). Also, `.path()` and `.not()` are outside core and now in 'lib/'.
WARNING! This `file.js` module for gun is intended for local development testing only!
/home/trex/dev/learn/gun/server/server.js:17
gun.wsp(server);
^
TypeError: gun.wsp is not a function
at Object.<anonymous> (/home/trex/dev/learn/gun/server/server.js:17:5)
Server source code:
const Hapi = require('hapi');
const Gun = require('gun');
const gun = new Gun();
const server = new Hapi.Server();
server.connection({ port: 3000, host: 'localhost' });
server.ext('onRequest', () => gun.wsp.server);
gun.wsp(server);
server.start((err) => {
if (err) {
throw err;
}
console.log(`Server running at: ${server.info.uri}`);
});
What is wrong here?
The bug solved in gun 0.8.8 https://github.com/amark/gun/pull/423
const Hapi = require('hapi');
const Inert = require('inert');
const Gun = require('../gun/');
const server = new Hapi.Server;
server.connection({ port: 8080 });
server.connections.forEach(c => Gun({ web: c.listener, file: 'data.json' }));
server.register(Inert, () => {});
server.route({
method: 'GET',
path: '/{param*}',
handler: {
directory: {
path: __dirname,
redirectToSlash: true,
index: true
}
}
});
server.start();