Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • B bull
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 175
    • Issues 175
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 9
    • Merge requests 9
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • OptimalBits
  • bull
  • Issues
  • #1712
Closed
Open
Issue created Apr 27, 2020 by Administrator@rootContributor

Sanboxed workers doesn't trigger `failed` events when the worker was killed.

Created by: kennethpdev

Description

I have a worker in a separate process (sanboxed).

const worker = path.resolve(__dirname, 'worker.js');
queue.process(2, worker);

But whenever this worker terminates or force killed like a server stop/restarts the queue failed event never triggered and I always get the error below.

0|audio-worker  | /mnt/d/Git/audio-worker/node_modules/bull/lib/process/master.js:99
0|audio-worker  |   throw err;
0|audio-worker  |   ^
0|audio-worker  | Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
0|audio-worker  |     at process.target.send (internal/child_process.js:636:16)
0|audio-worker  |     at Promise.resolve.then.err (/mnt/d/Git/audio-worker/node_modules/bull/lib/process/master.js:76:21)

sometimes its Error: write EPIPE

Different from when I choose not to make it a separate process.

const worker = require('./worker');
queue.process(2, worker);

This setup always receive the failed event and no error. Is this normal, like does sandboxed never trigger the failed event or something was wrong?

Minimal, Working Test code to reproduce the issue.

My worker is simple I can replicate it with this simple reject.

module.exports = async (job) => {
  try {
    const { id: jobId } = job;
    console.log('Job worker started', jobId);

    // simulated heavy process that takes 30seconds to complete
    await new Promise((resolve, reject) => {
      const makeError = () => {
        console.log('Killing job', jobId);
        reject(new Error('Error from long process'));
      };

      process.on('SIGTERM', makeError);
      process.on('SIGINT', makeError);

      setTimeout(resolve, 30000);
    });

    // success
    return Promise.resolve();
  } catch (err) {
    console.log('worker error', err.message);
    return Promise.reject(new Error("This should trigger the failed event"));
  }
};

for this particular test I use pm2 to start the app, and then pm2 stop <name> to properly trigger the SIGINT. But I think ctrl+c also works.

To replicate, wait for a few seconds after the log Job worker started. Then terminate the process.

Bull version

v3.13.0

Additional information

node v10.20.1 ioredis v4.16.2

Assignee
Assign to
Time tracking