Close redis connection when using NestJS Queues - testing

I'm trying to setup E2E tests in a NestJS project, however, jest output looks like this:
Jest did not exit one second after the test run has completed
After a lot of reading this is because there are some resources, not yet liberated, after some debugging it turns there's an open connection to redis created by ioredis which is used by bull which is used by NestJS to do task queue processing. The thing is that I don't have a reference to the connection in the test code, so how can I close it? I'm tearing down the Nest application in the afterAll jest's hook like this:
afterAll(async () => {
await app.close();
});
but it does nothing, the connection is still there and the jest error message persists. I know I can just add the --forceExit to the jest command but that is not solving anything, it's just hiding the problem under the rug.

This took me awhile to figure out. You need to close the module in the afterAll hook. I was able to find this from looking at the tests in the nestJS Bull repo.
describe('RedisTest', () => {
let module: TestingModule;
beforeAll(async () => {
module = Test.createTestingModule({
imports: [
BullModule.registerQueueAsync({
name: 'test2',
}),
],
});
});
afterAll(async () => {
await module.close();
});
});
https://github.com/nestjs/bull/blob/master/e2e/module.e2e-spec.ts

After struggling almost to getting depressed i found a solution that worked for me.
I'm using "#nestjs/bull": "^0.3.1", "bull": "^3.21.1".
because the queue from bull package uses redis, it keeps the connection open although the module & app are closed.
await moduleRef.close();
await app.close();
i realized that when using --detectOpenHandles while relying on leaked-handles library for more information, you will see something like this in the console:
tcp stream {
fd: 20,
readable: true,
writable: false,
address: {},
serverAddr: null
}
tcp handle leaked at one of:
at /media/user/somePartitionName/Workspace/Nest/project-
name/node_modules/ioredis/built/connectors/StandaloneConnector.js:58:45
tcp stream {
fd: 22,
readable: true,
writable: true,
address: { address: '127.0.0.1', family: 'IPv4', port: 34876 },
serverAddr: null
}
Solution
using beforEach() & afterEach()
in beforEach(), add this instruction to get an instance of your queue :
queue = moduleRef.get(getQueueToken("queuename"));
in afterEach(), close the queue like so: (also close your app & module for better practice)
await queue.close();
Note
using beforAll() & afterAll() doesn't work and the same problem occurs, at least from what i have tried, both beforEach() & afterEach() work combined.

You don't have to add queue.close() to every test, just close queues in their own service/provider using OnModuleDestroy Hook:
#Injectable()
export class ServicesConsumer implements OnModuleDestroy {
constructor(
#InjectQueue('services')
private readonly servicesQueue: Queue,
) { }
async onModuleDestroy() {
await this.servicesQueue.close();
}

Related

Mongoose connection closing too soon before tests have run

I am a beginner using jest to test a node/express app with mongo database.
I am getting an issue where different tests are failing each time I run the tests and sometimes they all pass/all fail. I think it is because of a time-out or things not happening in the right order because I'm getting this error:
MongoPoolClosedError: Attempted to check out a connection from closed connection pool
a) can you let me know if you think I'm on the right track?
b) if so, is the solution to make this into an async function and how can I do that? (I have tried making it into async await, using .then and also putting the code that clears the database collections into the test files instead of the helper and have had no success so far.
beforeAll( (done) => {
mongoose.connect("mongodb://127.0.0.1/jobBuddy_test", {
useNewUrlParser: true,
useUnifiedTopology: true,
})
var db = mongoose.connection;
const users = db.collection('users')
users.deleteMany({})
const applications = db.collection('applications')
applications.deleteMany({})
db.on("error", console.error.bind(console, "MongoDB connection error:"));
db.on("open", function () {
done();
});
});
afterAll(function (done) {
mongoose.connection.close(true, function () {
done();
});
});

Detox with Split.IO

I found a jest mock for split.io for react-native. I am now trying to use this mock so that I do not receive network timeouts because split.io is trying to sync in the background. Here is the mock:
jest.mock('#splitsoftware/splitio-react-native', () => {
const splitio = jest.requireActual('#splitsoftware/splitio-react-native');
return {
...splitio,
SplitFactory: () => {
return splitio.SplitFactory({
core: {
authorizationKey: 'localhost',
},
// Mock your splits and treatments here
features: {},
sync: {
localhostMode: splitio.LocalhostFromObject(),
},
});
},
};
});
I currently put this in my detox init.js file, but it doesn't seem to be doing anything. The only way, so far, I have been able to get my tests to run is to just immediately destroy my SplitFactory as soon as I create it (not through the mock). Obviously, this isn't ideal since I'd have to change the code every time I wanted to run it. I tried creating a .mock.ts file, but that also didn't get read, and when I tried to adjust my metro.config.js, it just failed to run at all. Does anyone have any ideas of how I can get this to run properly in detox for iOS, or have experience with this?
I had the same issue with split.io and detox, when a particular split.io would block my tests indefinitely. The only work around i found was
await device.disableSynchronization();
found here

Service Workers with Vue and Bundling

I use Vue 2 with Common.js to generate an AMD Bundle. I need to be able to automatically register my service worker on runtime. Something that works:
https://www.npmjs.com/package/worker-plugin
https://www.npmjs.com/package/worker-loader
The reason I need a service worker is for sending notifications. However, I am having trouble with this, as it seems that the only workers supported are in DedicatedWorkerGlobalScope or SharedWorkers. In order to dispatch "showNotification" however, I need the Service Worker type.
So basically what I do:
import Worker from "worker-loader!./Worker.js"
const worker = new Worker()
Works like charm, as does this (Worker Plugin):
const worker = new Worker('./worker.js', { type: 'module' });
However, always normal workers. Those arent service workers and I have been trying to figure this out since hours. Is there a configuration im missing to change the type? Some insight would be great.
To illustrate what im trying to achieve:
Registration of the Service Worker needs to happen automatically on Runtime, without me having to reference absolute or relative urls.
Any insight on how I can achieve what im trying to accomplish?
I did not use your plugins but I used workbox plugin for Vue. The configuration is quite simple and well documented.
Installing in an Already Created Project
vue add workbox-pwa
Config example
// Inside vue.config.js
module.exports = {
// ...other vue-cli plugin options...
pwa: {
name: 'My App',
themeColor: '#4DBA87',
msTileColor: '#000000',
appleMobileWebAppCapable: 'yes',
appleMobileWebAppStatusBarStyle: 'black',
manifestOptions: {
start_url: '/'
},
// configure the workbox plugin
workboxPluginMode: 'GenerateSW', // 'GenerateSW' will lead to a new service worker file being created each time you rebuild your web app.
workboxOptions: {
swDest: 'service-worker.js'
}
}
}
You could use service-worker-loader instead (which is based on worker-loader).
Install service-worker-loader:
npm i -D service-worker-loader
Create a service worker script (e.g., at ./src/sw.js) with the following example contents:
import { format } from 'date-fns'
self.addEventListener('install', () => {
console.log('Service worker installed', format(new Date(), `'Today is a' eeee`))
})
Import the service worker script in your entry file:
// main.js
import registerServiceWorker from 'service-worker-loader!./sw'
registerServiceWorker({ scope: '/' }).then(registration => {
//...
registration.showNotification('Notification Title', {
body: 'Hello world!',
})
})
demo

Multiple connections with same name are created in e2e test of NestJs with in memory database

I have NestJs application with TypeORM configured with mysql. I want to have e2e(integration) test and for that reason I want to have in memory database in the tests which I configured this way:
{
type: 'sqlite',
database: ':memory:',
synchronize: true,
dropSchema: true,
entities: [`dist/**/*.entity{.ts,.js}`],
}
And the setup of the tests
beforeEach(async () => {
const moduleFixture: TestingModule =
await Test.createTestingModule({imports: [AppModule, UserModule]})
.overrideProvider(TypeOrmConfigService).useClass(MockTypeOrmConfigService)
.compile();
app = await moduleFixture.createNestApplication();
await app.init();
});
. When running the test I got
AlreadyHasActiveConnectionError: Cannot create a new connection named "default", because connection with such name already exist and it now has an active connection session.
at new AlreadyHasActiveConnectionError (/Users/user/workspace/app/src/error/AlreadyHasActiveConnectionError.ts:8:9)
at ConnectionManager.Object.<anonymous>.ConnectionManager.create (/Users/user/workspace/app/src/connection/ConnectionManager.ts:57:23)
at Object.<anonymous> (/Users/user/workspace/app/src/index.ts:228:35)
at step (/Users/user/workspace/app/node_modules/tslib/tslib.js:136:27)
at Object.next (/Users/user/workspace/app/node_modules/tslib/tslib.js:117:57)
at /Users/user/workspace/app/node_modules/tslib/tslib.js:110:75
at new Promise (<anonymous>)
at Object.__awaiter (/Users/user/workspace/app/node_modules/tslib/tslib.js:106:16)
at Object.createConnection (/Users/user/workspace/app/node_modules/typeorm/index.js:186:20)
at rxjs_1.defer (/Users/user/workspace/app/node_modules/#nestjs/typeorm/dist/typeorm-core.module.js:151:29)
(node:19140) UnhandledPromiseRejectionWarning: AlreadyHasActiveConnectionError: Caught error after test environment was torn down
If I move the setup from beforeEach in beforeAll block it's ok, but I'm afraid that when I create several specs the error will come back. How should be handled properly?
EDIT:
The problem was that each test is making a setup of the application and so creates a new connection.The solution was to use "keepConnectionAlive: true," in order all tests to reuse same connection.
keepCOnnectionAlive: true is the way to go
Using keepConnectionAlive: true produced the following error for me.
Jest did not exit one second after the test run has completed.
This usually means that there are asynchronous operations that weren't
stopped in your tests. Consider running Jest with
--detectOpenHandles to troubleshoot this issue.
Adding the below to each e2e test fixed my issue:
afterEach(async () => {
await app.close();
});
Base on 0xCAP's answer, you can do something like this also.
// jest.setup.ts
jest.mock("/path/to/database/config/object", () => {
const { databaseConfig, ...rest } = jest.requireActual("/path/to/database/config/object")
return {
...rest,
databaseConfig: {
...databaseConfig,
keepConnectionAlive: true // replace old config
}
}
})
// jest.config.js
module.exports = {
...other options
setupFilesAfterEnv: ["jest.setup.ts"],
}

React-native socket.io emit / on actions not firing

I am working with a react-native application that involves socket.io. I have installed socket.ion successfully and imported it in my component like:
import React, { Component } from 'react';
import io from 'socket.io-client/dist/socket.io';
const connectionConfig = {
jsonp: false,
reconnection: true,
reconnectionDelay: 100,
reconnectionAttempts: 100000/// you need to explicitly tell it to use websockets
};
socket = io('http://192.168.1.100:3001', connectionConfig);
type Props = {};
export default class Socket extends Component<Props> {
constructor(){
super();
socket.emit("Button",'pressed')
socket.on('userid',(id)=>{
this.setState({userid:{id}});
Alert.alert(id);
})
}
And the code for my server side using express is:
io.on('connection', function(socket){
console.log(socket.id);
socket.on('Button',function(data){
console.log("ButtonPressed");
});
socket.emit('userid',socket.id);
})
What is strange, after like every 1.5s the server console logs a different socket.id when i ran the application on an android device. I assume the socket.io connects successfully but again disconnects in the 1.5s i mentioned such that the socket.emit and socket.on event don't get to be executed.I have tried many options provided but cannot get the right way to fix this. Please if you know a work-around i highly appreciate. Thank you.
I realised on android if you use the option transports: ['websocket'] and you are in the development mode, then first enable the debug remotely by shaking your phone and sockets will work fine for you. So basiclly you should have something like
import io from 'socket.io-client';
const connectionConfig = {
jsonp: false,
reconnection: true,
reconnectionDelay: 100,
reconnectionAttempts: 5000,
transports: ['websocket']/// you need to explicitly tell it to use websockets
};
socket = io('http://192.168.1.100:3001', connectionConfig);