Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • B bull
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 175
    • Issues 175
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 9
    • Merge requests 9
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • OptimalBits
  • bull
  • Issues
  • #2460
Closed
Open
Issue created Nov 08, 2022 by Makis Theotokis@mtheotokis

Lots of connected clients in Redis

I'm using bull-board as my monitoring tool.

In bull-board, i can notice that there is a really big (and unexpected) number of "connected clients" in Redis.

Connected Clients: 362 / Blocked clients 36

Currently running:

  • 4 application instances
  • 6 different queues
  • 11 unique sandboxed processors

My questions are:

  • Could you help me identify if this number makes sense?
  • Am i supposed to close the queues in the sandbox processors?

Sample Code:

/* ---------------------------- Queues Module Start ---------------------------- */
const mainQueue = new Queue('main-queue', {
  host: process.env.REDIS_HOST,
  port: parseInt(process.env.REDIS_PORT, 10),
}, {
  redis: {
    tls: {},
    connectTimeout: 30000,
  },
});

module.exports = {
  mainQueue,
  ... // All other queues
};
/* --------------------------- Queues Module End ----------------------------- */


/* ----------------------------- Worker Start ------------------------------------ */

/* 
  Importing here, from queues module.
  
  Main queue is serving 8 different types of processors 
  The jobs are dispatched internally in the application
 */
mainQueue.process('main-type-1', 2, `${__dirname}/processors/main-type-1`);
mainQueue.process('main-type-2', 0, `${__dirname}/processors/main-type-2`);
mainQueue.process('main-type-3', 0, `${__dirname}/processors/main-type-3`);
mainQueue.process('main-type-4', 0, `${__dirname}/processors/main-type-4`);
mainQueue.process('main-type-5', 0, `${__dirname}/processors/main-type-5`);
mainQueue.process('main-type-6', 0, `${__dirname}/processors/main-type-6`);
mainQueue.process('main-type-7', 0, `${__dirname}/processors/main-type-7`);
mainQueue.process('main-type-8', 0, `${__dirname}/processors/main-type-8`);

/* Repeated Job Queues */
cronQueue.add('scheduler', {}, { repeat: { cron: '* * * * *' } });
cronQueue.process('scheduler', 1, `${__dirname}/scheduler`);

cronQueue2.add('scheduler-2', {}, { repeat: { cron: '0 2 * * *' } });
cronQueue2.process('scheduler-2', 1, `${__dirname}/scheduler2`);

cronQueue3.add('scheduler-3', {}, { repeat: { cron: '0 3 * * *' } });
cronQueue3.process('scheduler-3', 1, `${__dirname}/scheduler3`);

cronQueue4.add('scheduler-4', {}, { repeat: { cron: '0 4 * * *' } });
cronQueue4.process('scheduler-4', 1, `${__dirname}/scheduler4`);

cronQueue5.add('scheduler-5', {}, { repeat: { cron: '0 5 * * *' } });
cronQueue5.process('scheduler-5', 1, `${__dirname}/scheduler5`);

/* ----------------------------- Worker End ------------------------------------ */

/* ----------------------------- Main Type 1 Processor Start ------------------------------------ */

const { success, failure } = require('../helpers/tasks');
module.exports = async (job) => {
    try {
      // Do stuff
      return success(job, log, log);
    } catch (error) {
      // Creating error
      return failure(job, failedError, log);
    }
};

/* ----------------------------- Main Type 1 Processor End ------------------------------------ */

/* ----------------------------- Helpers > Tasks Start ------------------------------------ */
const { handleFailure } = require('../dispatcher');

const success = async (job, log, resolveValue) => {
  job.log(log);
  job.progress(100);
  return Promise.resolve(resolveValue);
};

const failure = async (job, error, log) => {
  job.log(log);
  job.progress(100);
  await handleFailure(job, error);
  return Promise.reject(error);
};
/* ----------------------------- Helpers > Tasks End ------------------------------------ */

Note: Since i have revisited the codebase, i'm noticing:

  • On tasks module methods, the async keyword is obsolete
  • Maybe the issue is happening because i'm returning a promise (cause i use async) in the processor?

Bull version: 4.8.5

Assignee
Assign to
Time tracking