Lots of connected clients in Redis
I'm using bull-board as my monitoring tool.
In bull-board, i can notice that there is a really big (and unexpected) number of "connected clients" in Redis.
Connected Clients: 362 / Blocked clients 36
Currently running:
- 4 application instances
- 6 different queues
- 11 unique sandboxed processors
My questions are:
- Could you help me identify if this number makes sense?
- Am i supposed to close the queues in the sandbox processors?
Sample Code:
/* ---------------------------- Queues Module Start ---------------------------- */
const mainQueue = new Queue('main-queue', {
host: process.env.REDIS_HOST,
port: parseInt(process.env.REDIS_PORT, 10),
}, {
redis: {
tls: {},
connectTimeout: 30000,
},
});
module.exports = {
mainQueue,
... // All other queues
};
/* --------------------------- Queues Module End ----------------------------- */
/* ----------------------------- Worker Start ------------------------------------ */
/*
Importing here, from queues module.
Main queue is serving 8 different types of processors
The jobs are dispatched internally in the application
*/
mainQueue.process('main-type-1', 2, `${__dirname}/processors/main-type-1`);
mainQueue.process('main-type-2', 0, `${__dirname}/processors/main-type-2`);
mainQueue.process('main-type-3', 0, `${__dirname}/processors/main-type-3`);
mainQueue.process('main-type-4', 0, `${__dirname}/processors/main-type-4`);
mainQueue.process('main-type-5', 0, `${__dirname}/processors/main-type-5`);
mainQueue.process('main-type-6', 0, `${__dirname}/processors/main-type-6`);
mainQueue.process('main-type-7', 0, `${__dirname}/processors/main-type-7`);
mainQueue.process('main-type-8', 0, `${__dirname}/processors/main-type-8`);
/* Repeated Job Queues */
cronQueue.add('scheduler', {}, { repeat: { cron: '* * * * *' } });
cronQueue.process('scheduler', 1, `${__dirname}/scheduler`);
cronQueue2.add('scheduler-2', {}, { repeat: { cron: '0 2 * * *' } });
cronQueue2.process('scheduler-2', 1, `${__dirname}/scheduler2`);
cronQueue3.add('scheduler-3', {}, { repeat: { cron: '0 3 * * *' } });
cronQueue3.process('scheduler-3', 1, `${__dirname}/scheduler3`);
cronQueue4.add('scheduler-4', {}, { repeat: { cron: '0 4 * * *' } });
cronQueue4.process('scheduler-4', 1, `${__dirname}/scheduler4`);
cronQueue5.add('scheduler-5', {}, { repeat: { cron: '0 5 * * *' } });
cronQueue5.process('scheduler-5', 1, `${__dirname}/scheduler5`);
/* ----------------------------- Worker End ------------------------------------ */
/* ----------------------------- Main Type 1 Processor Start ------------------------------------ */
const { success, failure } = require('../helpers/tasks');
module.exports = async (job) => {
try {
// Do stuff
return success(job, log, log);
} catch (error) {
// Creating error
return failure(job, failedError, log);
}
};
/* ----------------------------- Main Type 1 Processor End ------------------------------------ */
/* ----------------------------- Helpers > Tasks Start ------------------------------------ */
const { handleFailure } = require('../dispatcher');
const success = async (job, log, resolveValue) => {
job.log(log);
job.progress(100);
return Promise.resolve(resolveValue);
};
const failure = async (job, error, log) => {
job.log(log);
job.progress(100);
await handleFailure(job, error);
return Promise.reject(error);
};
/* ----------------------------- Helpers > Tasks End ------------------------------------ */
Note: Since i have revisited the codebase, i'm noticing:
- On tasks module methods, the async keyword is obsolete
- Maybe the issue is happening because i'm returning a promise (cause i use async) in the processor?
Bull version: 4.8.5