Skip to content

OMjob operator tolerations are not working in scheduled jobs #27852

@sanjay-m-v

Description

@sanjay-m-v

Affected module

Backend

Describe the bug

After upgrading to 1.12.6 and redeploying all the pipeline jobs, scheduled pipelines fail. The scheduled run creates a pod with the correct image and nodeselector, but without the configured tolerations. As a result, pods remain Pending on tainted nodes.
but manual pipeline runs work correctly: the generated OMJob includes the configured nodeselector and tolerations, and the resulting pod schedules successfully.

Image I verified both the CRD'S omjobs and cronomjobs reflect the new toleration field

What is more critical is removing toleration or nodeselector from values does not remove it from the newly scheduled pods, even after redeployment of all pipelines.

To Reproduce

  1. Add a toleration field in the helm values
  2. Redeploy a scheduled job pipeline
  3. Once the job runs, inspect the manifest to see the absent added toleration
  4. Revert the toleration by removing the toleration field
  5. Redeploy pipelines
  6. The newly created jobs from scheduled jobs will still have the toleration field.

Expected behavior

  1. Tolerations present on the scheduled omjobs
  2. Removing toleration or nodeselector reflect change on the newly creaed omjobs(with or without redeployment)

OS

Amazon linux 2023

Python version

No response

OpenMetadata version

1.12.6

OpenMetadata Ingestion package version

No response

Additional context

No response

Pre-submission checklist

  • I searched for duplicate issues.
  • I removed credentials, hostnames, emails, and other sensitive data from logs and config.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions