Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • B bull
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 175
    • Issues 175
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 9
    • Merge requests 9
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • OptimalBits
  • bull
  • Issues
  • #459
Closed
Open
Issue created Mar 01, 2017 by Manuel Astudillo@manastContributor

Store all jobs data as JSON inside a global HSET

There is one standing issue with current implementation where we need to create redis keys inside LUA scripts. One of the problems with this is that it breaks redis recommendations and it would be impossible to use the queues in a clustered setup. The only reason the jobs are stores as HSETs is so that the progress of a job can be updated without needing to serialize the whole job. So instead we can have one global HSET (jobs), where every hash is a unique job id and the value is the serialized JSON data of the job, and a separate HSET (progress), where the hash is the jobid and the value is the current progress of the job. This change will break backwards compatibility so it will require a major version.

Assignee
Assign to
Time tracking