Skip to content

ZOOKEEPER-4985 Fix zkCleanup.sh script to not fail on non-standard config directories #2324

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lfrancke wants to merge 7 commits into
base: master
Choose a base branch
from
+20 −13
Open

ZOOKEEPER-4985 Fix zkCleanup.sh script to not fail on non-standard config directories #2324

Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d4c905f
ZOOKEEPER-4985 Fix zkCleanup.sh script to not fail on non-standard co…
lfrancke Oct 6, 2025
6931054
Fix shfmt error
lfrancke Oct 6, 2025
237f5b7
Update bin/zkCleanup.sh
lfrancke Nov 19, 2025
fb5eb34
Address review comments
lfrancke Nov 19, 2025
63b2e59
Address review comments
lfrancke Mar 10, 2026
e31a646
Handle greedy sed matching and whitespace trimming
lfrancke Mar 10, 2026
d4b8d01
Merge branch 'master' into feat/ZOOKEEPER-4985-rebased
lfrancke Mar 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 26 additions & 10 deletions bin/zkCleanup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,27 +31,43 @@
# use POSIX interface, symlink is followed automatically
ZOOBIN="${BASH_SOURCE-$0}"
ZOOBIN="$(dirname "$ZOOBIN")"
ZOOBINDIR="$(cd "$ZOOBIN" && pwd)"
ZOOBINDIR="$(cd "$ZOOBIN" && pwd)" || exit

if [[ -e "$ZOOBIN/../libexec/zkEnv.sh" ]]; then
# shellcheck source=bin/zkEnv.sh
. "$ZOOBINDIR"/../libexec/zkEnv.sh
. "$ZOOBINDIR"/../libexec/zkEnv.sh "$@"
else
# shellcheck source=bin/zkEnv.sh
. "$ZOOBINDIR"/zkEnv.sh
. "$ZOOBINDIR"/zkEnv.sh "$@"
fi

ZOODATADIR="$(grep "^[[:space:]]*dataDir=" "$ZOOCFG" | sed -e 's/.*=//')"
ZOODATALOGDIR="$(grep "^[[:space:]]*dataLogDir=" "$ZOOCFG" | sed -e 's/.*=//')"
ZOODATADIR=""
ZOODATALOGDIR=""

# Only try to read config if ZOOCFG exists
if [[ -f $ZOOCFG ]]; then
ZOODATADIR="$(grep "^[[:space:]]*dataDir=" "$ZOOCFG" 2>/dev/null | sed -e 's/.*=//')"
ZOODATALOGDIR="$(grep "^[[:space:]]*dataLogDir=" "$ZOOCFG" 2>/dev/null | sed -e 's/.*=//')"
Copy link
Copy Markdown
Member

@ctubbsii ctubbsii Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is perhaps something for another PR, since it's a bit out of scope here, and there may be other scripts that would also benefit from this change, but I noticed that sed has greedy matching here and doesn't trim the whitespace in the value. cut -f2- -d= could be used instead to avoid the greedy matching, but still doesn't trim whitespace in the extracted property value.

It can be done with bash regular expressions and a simple capture group, though, without calling another process:

  re='^[^=]*=[[:space:]]*(.*)[[:space:]]*$'
  ZOODATADIR="$(grep "^[[:space:]]*dataDir=" "$ZOOCFG" 2>/dev/null)"
  [[ $ZOODATADIR =~ $re ]] && ZOODATADIR="${BASH_REMATCH[1]}"
  ZOODATALOGDIR="$(grep "^[[:space:]]*dataLogDir=" "$ZOOCFG" 2>/dev/null)"
  [[ $ZOODATALOGDIR =~ $re ]] && ZOODATALOGDIR="${BASH_REMATCH[1]}"

Copy link
Copy Markdown
Member Author

@lfrancke lfrancke Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your bash/regex skills are far beyond mine :) I'd never have caught this.

I have included your suggestion as-is. I believe I understand the regex and it looks good to me

e31a646 (this PR)

fi

ZOO_LOG_FILE=zookeeper-$USER-cleanup-$HOSTNAME.log

# shellcheck disable=SC2206
flags=($JVMFLAGS)
Copy link
Copy Markdown
Member

@ctubbsii ctubbsii Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" is added in all cases. So, it could just be added to flags here and removed from the subsequent calls:

Suggested change
flags=($JVMFLAGS)
flags=("-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" $JVMFLAGS)

Copy link
Copy Markdown
Member Author

@lfrancke lfrancke Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed manually (see below for a reason why) in 63b2e59 (this PR)

if [[ -z $ZOODATALOGDIR ]]; then
"$JAVA" "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" "${flags[@]}" \
org.apache.zookeeper.server.PurgeTxnLog "$ZOODATADIR" "$@"

# If config provides directories, use them; otherwise pass all args to PurgeTxnLog
if [[ -n $ZOODATADIR ]]; then
if [[ -z $ZOODATALOGDIR ]]; then
# Only dataDir specified
"$JAVA" "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" \
"${flags[@]}" org.apache.zookeeper.server.PurgeTxnLog "$ZOODATADIR" "$@"
else
# Both dataDir and dataLogDir specified
"$JAVA" "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" \
"${flags[@]}" org.apache.zookeeper.server.PurgeTxnLog "$ZOODATALOGDIR" "$ZOODATADIR" "$@"
fi
else
"$JAVA" "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" "${flags[@]}" \
org.apache.zookeeper.server.PurgeTxnLog "$ZOODATALOGDIR" "$ZOODATADIR" "$@"
# No config or config doesn't specify directories - pass all args to PurgeTxnLog
"$JAVA" "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" \
"${flags[@]}" org.apache.zookeeper.server.PurgeTxnLog "$@"
fi
Copy link
Copy Markdown
Member

@ctubbsii ctubbsii Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The complexity of this if/else can be entirely removed if you just build up the command-line arguments. Combining with my earlier suggestion to add the other system properties to the flags array, this simplifies to:

Suggested change
if [[ -n $ZOODATADIR ]]; then
if [[ -z $ZOODATALOGDIR ]]; then
# Only dataDir specified
"$JAVA" "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" \
"${flags[@]}" org.apache.zookeeper.server.PurgeTxnLog "$ZOODATADIR" "$@"
else
# Both dataDir and dataLogDir specified
"$JAVA" "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" \
"${flags[@]}" org.apache.zookeeper.server.PurgeTxnLog "$ZOODATALOGDIR" "$ZOODATADIR" "$@"
fi
else
"$JAVA" "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" "${flags[@]}" \
org.apache.zookeeper.server.PurgeTxnLog "$ZOODATALOGDIR" "$ZOODATADIR" "$@"
# No config or config doesn't specify directories - pass all args to PurgeTxnLog
"$JAVA" "-Dzookeeper.log.dir=$ZOO_LOG_DIR" "-Dzookeeper.log.file=$ZOO_LOG_FILE" \
"${flags[@]}" org.apache.zookeeper.server.PurgeTxnLog "$@"
fi
directories=()
[[ -n $ZOODATADIR ]] && directories=("$ZOODATADIR")
[[ -n $ZOODATALOGDIR ]] && directories=("$ZOODATALOGDIR" "${directories[@]}")
"$JAVA" "${flags[@]}" org.apache.zookeeper.server.PurgeTxnLog "${directories[@]}" "$@"

This also covers the case where only dataLogDir is set, which I think wasn't covered before.

Copy link
Copy Markdown
Member Author

@lfrancke lfrancke Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that is a much better approach :(

I couldn't apply the patch directly from GitHub due to

Applying suggestions on deleted lines is currently not supported.

So I did it manually in 63b2e59 (this PR) together with your suggestions for the flags