Skip to content

CNTRLPLANE-3323: Update openshift-hack/rebase.sh #2701

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JSampsonIV wants to merge 1 commit into
base: master
Choose a base branch
from
+105 −51
Open

CNTRLPLANE-3323: Update openshift-hack/rebase.sh #2701

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 57 additions & 15 deletions REBASE.openshift.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,9 +360,8 @@ The following repositories have been already bumped as well:

<URLs to api/client-go/library-go/apiserver-library-go/operators>

Followup work has been assigned to appropriate teams
through bugzillas linked in the code. Please treat
them as the highest priority after landing the bump.
A Jira ticket has been opened for the rebase process.
It has been linked to the pull request.

Finally, this means we are blocking ALL PRs to our
kubernetes fork.
Expand Down Expand Up @@ -404,10 +403,55 @@ them as the highest priority and release blockers for your team:
1. Update cluster-kube-apiserver-operator `pre-release-lifecycle` alert's
`removed_release` version similarly to https://github.com/openshift/cluster-kube-apiserver-operator/pull/1382.

## Updating with `git merge`
## Updating with `redhat-chai-bot`
Comment thread
JSampsonIV marked this conversation as resolved.

*This is the preferred way to update to patch releases of kubernetes*

[chai-bot](slack://app?team=T027F3GAJ&id=A0AJUKWDUR1&tab=messages) (ship-help-bot) is an internal tool has been given instructions to
periodically check for upstream patch releases and complete the rebase autonomously. The steps taken are outlined below:


1. Fetch upstream tags over the past 31 days:
```
git fetch --tags upstream && git tag --sort=-creatordate | grep -E 'v[0-9]+\.[0-9]+\.[0-9]+$' | awk -v cutoff="$(date -d '31 days ago' +%s)" '$2 >= cutoff {print $1}'
```
where `upstream` points at https://github.com/kubernetes/kubernetes/

2. Determine whether any patch release have occurred that have not yet been rebased and merged. If such patch releases exist,
the corresponding openshift branches are found and marked to be rebased. Otherwise, chai-bot reports that there is no rebase necessary.
- *Branches tracking master are skipped.*

3. Output a mapping table of upstream release to openshift version, as well as whether that branch needs to be rebased, to the appropriate team:
```
*Kubernetes patch rebase check*

Upstream Release OCP Branch Current k8s Status Notes
───────────────── ────────────── ──────────── ───────── ──────────────────────────
v1.35.6 master 1.35.3 REBASE Patch 3 → 6
v1.35.6 release-5.1 1.35.3 SKIP Points at openshift/master
v1.35.6 release-4.22 1.35.5 REBASE Patch 5 → 6
v1.34.9 release-4.21 1.34.8 REBASE Patch 8 → 9
```

4. For each branch requiring a rebase, chai-bot runs [rebase.sh](https://github.com/openshift/kubernetes/blob/master/openshift-hack/rebase.sh) (see usage below) with
the appropriate parameters of `kubernetes_tag` and `openshift_release`.

5. After each successful rebase, a message is posted to the appropriate team:
```
*:white_check_mark: Rebase of openshift/kubernetes:{openshift_release} → {kubernetes_tag} complete*

Branch pushed: <fork_repo>:{branch_name}
• Kubernetes version: {old_version} → {kubernetes_tag}
• Merge conflicts: {list of files or "None"}

_Open a PR against `openshift/kubernetes:{openshift_release}` when ready._
```
On rebase failure, a message containing failure details is posted instead and the next branch is rebased.

6. A final summary is posted and permission to open a pull request for each rebase is requested.

## Updating with `git merge`

After the initial bump as described above it is possible to update
to newer released version using `git merge`. To do that follow these steps:

Expand Down Expand Up @@ -512,26 +556,24 @@ etcd version 3.5.6 or greater required
Grab newer version of etcd from https://github.com/etcd-io/etcd/releases/ and place
it in `/usr/local/bin/etcd`.

## Updating with `rebase.sh` (experimental)
## Updating with `rebase.sh`

The above steps are available as a script that will merge and rebase along the happy path without automatic conflict
resolution and at the end will create a PR for you.
In the event that chai-bot fails to rebase, [rebase.sh](https://github.com/openshift/kubernetes/blob/master/openshift-hack/rebase.sh) can also be run manually.

Here are the steps:
1. Create a new BugZilla with the respective OpenShift version to rebase (Target Release stays ---),
Prio&Severity to High with a proper description of the change logs.
See [BZ2021468](https://bugzilla.redhat.com/show_bug.cgi?id=2021468) as an example.
1. Create a new Jira ticket under OCPBUGS with the respective OpenShift version to rebase,
Target Backport Version to the previous minor version + .z
2. It's best to start off with a fresh fork of [openshift/kubernetes](https://github.com/openshift/kubernetes/). Stay on the master branch.
3. This script requires `jq`, `git`, `podman` and `bash`, `gh` is optional.
3. This script requires `git`, `podman` and `bash`, and optionally uses `gh` to create a pull request.
4. In the root dir of that fork run:
```
openshift-hack/rebase.sh --k8s-tag=v1.25.2 --openshift-release=release-4.12 --bugzilla-id=2003027
openshift-hack/rebase.sh --k8s-tag=v1.25.2 --openshift-release=release-4.12 --jira-id=OCPBUGS-90150
```
Comment thread
coderabbitai[bot] marked this conversation as resolved.

where `k8s-tag` is the [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes/) release tag, the `openshift-release`
is the OpenShift release branch in [openshift/kubernetes](https://github.com/openshift/kubernetes/) and the `bugzilla-id` is the
BugZilla ID created in step (1).
is the OpenShift release branch in [openshift/kubernetes](https://github.com/openshift/kubernetes/) and the `jira-id` is the
Jira ticket number created in step (1).

5. In case of conflicts, it will ask you to step into another shell to resolve those. The script will continue by committing the resolution with `UPSTREAM: <drop>`.
6. At the end, there will be a "rebase-$VERSION" branch pushed to your fork.
7. If you have `gh` installed and are logged in, it will attempt to create a PR for you by opening a web browser.
7. A pull request will be created with the title `$jira_id: Rebase $k8s_tag in $openshift_release` against the corresponding openshift branch.
84 changes: 48 additions & 36 deletions openshift-hack/rebase.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# READ FIRST BEFORE USING THIS SCRIPT
#
# This script requires jq, git, podman and bash to work properly (dependencies are checked for you).
# This script requires git, podman and bash to work properly (dependencies are checked for you).
# The Github CLI "gh" is optional, but convenient to create a pull request automatically at the end.
#
# This script generates a git remote structure described in:
Expand All @@ -11,16 +11,16 @@
#
# The usage is described in /Rebase.openshift.md.

# validate input args --k8s-tag=v1.21.2 --openshift-release=release-4.8 --bugzilla-id=2003027
# validate input args --k8s-tag=v1.21.2 --openshift-release=release-4.8 --jira-id=OCPBUGS-91759
k8s_tag=""
openshift_release=""
bugzilla_id=""
jira_id=""

usage() {
echo "Available arguments:"
echo " --k8s-tag (required) Example: --k8s-tag=v1.21.2"
echo " --openshift-release (required) Example: --openshift-release=release-4.8"
echo " --bugzilla-id (optional) creates new PR against openshift/kubernetes:${openshift-release}: Example: --bugzilla-id=2003027"
echo " --jira-id (optional) Include Jira ticket in PR title: Example: --jira-id=OCPBUGS-1234"
}

for i in "$@"; do
Expand All @@ -33,8 +33,8 @@ for i in "$@"; do
openshift_release="${i#*=}"
shift
;;
--bugzilla-id=*)
bugzilla_id="${i#*=}"
--jira-id=*)
jira_id="${i#*=}"
shift
;;
*)
Expand All @@ -61,19 +61,14 @@ fi
echo "Processed arguments are:"
echo "--k8s_tag=${k8s_tag}"
echo "--openshift_release=${openshift_release}"
echo "--bugzilla_id=${bugzilla_id}"
echo "--jira_id=${jira_id}"

# prerequisites (check git, podman, ... is present)
if ! command -v git &>/dev/null; then
echo "git not installed, exiting"
exit 1
fi

if ! command -v jq &>/dev/null; then
echo "jq not installed, exiting"
exit 1
fi

if ! command -v podman &>/dev/null; then
echo "podman not installed, exiting"
exit 1
Expand All @@ -98,9 +93,13 @@ git fetch upstream --tags -f
git remote add openshift git@github.com:openshift/kubernetes.git
git fetch openshift

#git checkout --track "openshift/$openshift_release"
git checkout --track "openshift/$openshift_release"
git pull openshift "$openshift_release"

if [ -z "$(git tag -l "$k8s_tag")" ]; then
echo "No such tag exists in upstream for: $k8s_tag"
exit 1
fi
git merge "$k8s_tag"
Comment thread
JSampsonIV marked this conversation as resolved.
# shellcheck disable=SC2181
if [ $? -eq 0 ]; then
Expand All @@ -125,51 +124,64 @@ fi

# openshift-hack/images/hyperkube/Dockerfile.rhel still has FROM pointing to old tag
# we need to remove the prefix "v" from the $k8s_tag to stay compatible
sed -i -E "s/(io.openshift.build.versions=\"kubernetes=)(1.[1-9]+.[1-9]+)/\1${k8s_tag:1}/" openshift-hack/images/hyperkube/Dockerfile.rhel
podman run --rm -v "$(pwd):/workspace:Z" docker.io/library/alpine:latest \
sed -i -E "s/(io.openshift.build.versions=\"kubernetes=)(1.[1-9]+.[1-9]+)/\1${k8s_tag:1}/" \
/workspace/openshift-hack/images/hyperkube/Dockerfile.rhel
go_mod_go_ver=$(grep -E 'go 1\.[1-9][0-9]?' go.mod | sed -E 's/go (1\.[1-9][0-9]?)/\1/' | cut -d '.' -f 1,2) # Need to handle mod versions like 1.23 and 1.23.4; our release images only have major.minor
tag="rhel-8-release-golang-${go_mod_go_ver}-openshift-${openshift_release#release-}"

# update openshift go.mod dependencies
sed -i -E "/=>/! s/(\tgithub.com\/openshift\/[a-z|-]+) (.*)$/\1 $openshift_release/" go.mod
tag=$(grep "^ tag:" .ci-operator.yaml | head -n1 | sed -E 's/.*: (.*)/\1/')

echo "> go mod tidy && hack/update-vendor.sh"
podman run -it --rm -v "$(pwd):/go/k8s.io/kubernetes:Z" \
podman run --rm -v "$(pwd):/go/k8s.io/kubernetes:Z" \
--workdir=/go/k8s.io/kubernetes \
"registry.ci.openshift.org/openshift/release:$tag" \
go mod tidy && hack/update-vendor.sh
/bin/bash -c "go mod tidy && hack/update-vendor.sh"

# shellcheck disable=SC2181
if [ $? -ne 0 ]; then
echo "updating the vendor folder failed, is any dependency missing?"
exit 1
fi

podman run -it --rm -v "$(pwd):/go/k8s.io/kubernetes:Z" \
echo "> make clean to remove stale _output directory"
podman run --rm -v "$(pwd):/go/k8s.io/kubernetes:Z" \
--workdir=/go/k8s.io/kubernetes \
"registry.ci.openshift.org/openshift/release:$tag" \
make clean
# shellcheck disable=SC2181
if [ $? -ne 0 ]; then
echo "make clean failed — check filesystem permissions on _output/"
exit 1
fi

podman run --rm -v "$(pwd):/go/k8s.io/kubernetes:Z" \
--workdir=/go/k8s.io/kubernetes \
"registry.ci.openshift.org/openshift/release:$tag" \
make update OS_RUN_WITHOUT_DOCKER=yes
# shellcheck disable=SC2181
if [ $? -ne 0 ]; then
echo "make update failed"
exit 1
fi

git add -A
git commit -m "UPSTREAM: <drop>: hack/update-vendor.sh, make update and update image"

remote_branch="rebase-$k8s_tag"
git push origin "$openshift_release:$remote_branch"

XY=$(echo "$k8s_tag" | sed -E "s/v(1\.[0-9]+)\.[0-9]+/\1/")
ver=$(echo "$k8s_tag" | sed "s/\.//g")
link="https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-$XY.md#$ver"
if [ -n "${bugzilla_id}" ]; then
if command -v gh &>/dev/null; then
XY=$(echo "$k8s_tag" | sed -E "s/v(1\.[0-9]+)\.[0-9]+/\1/")
ver=$(echo "$k8s_tag" | sed "s/\.//g")
link="https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-$XY.md#$ver"

# opens a web browser, because we can't properly create PRs against remote repositories with the GH CLI (yet):
# https://github.com/cli/cli/issues/2691
gh pr create \
--title "Bug $bugzilla_id: Rebase $k8s_tag" \
--body "CHANGELOG $link" \
--web
if command -v gh &>/dev/null; then

@dusk125 dusk125 Jul 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the documentation and earlier in the script, we say and check that we require the gh cli, however this condition is checking whether or not gh is a valid command. So we could either remove the condition, or remove the requirement on the gh cli, with the original caveat (in rebase.md) that if the gh cli is install, this script can automatically create a PR.

Since our automated rebase would be kinda useless without the automatic PR creation, we could probably remove this check and keep the requirement on the gh cli. It would protect us if someone removed gh from chai.

@JSampsonIV JSampsonIV Jul 22, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On further thought, it may be better to remove the requirement for gh. chai-bot never uses this block explicitly due to its inability to open PRs automatically, so at least on the automated side it's not doing anything.

@dusk125 dusk125 Jul 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh good call!

XY=$(echo "$k8s_tag" | sed -E "s/v(1\.[0-9]+)\.[0-9]+/\1/")
ver=$(echo "$k8s_tag" | sed "s/\.//g")
link="https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-$XY.md#$ver"

title="Rebase $k8s_tag in $openshift_release"
if [ -n "${jira_id}" ]; then
title="$jira_id: $title"
fi

gh pr create \
--title "$title" \
--body "CHANGELOG $link" \
--base "$openshift_release" \
--head "$remote_branch"
fi