Skip to content

Updated Gemini Automatic Code Review Agent - #268

Merged
koolpoolo merged 16 commits into
mainfrom
markpete/gemini
Aug 14, 2026
Merged

Updated Gemini Automatic Code Review Agent#268
koolpoolo merged 16 commits into
mainfrom
markpete/gemini

Conversation

@markpete

@markpete markpete commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Google is ending support for their consumer version of Google Code Assist. According to my research this change MAY still work and enable automatic as well as trigger (by /gemini command) code reviews in our repository.

Unfortunately, this needs to be checked into main to see if it works...

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

⚠️ Failed to generate review from Gemini: HTTP Error 404: Not Found

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

⚠️ Failed to generate review from Gemini: HTTP Error 503: Service Unavailable

@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown

🤖 Gemini Code Review

Hey team! Automating initial code reviews using Gemini is a fantastic idea. This will give students instant feedback on their PRs before mentors do a manual pass, which will really speed up our development cycles.

Here is my review of the workflow file. Overall, the logic is solid, but there are a couple of key bugs and improvements to address before we merge this.


🚨 Critical Issues & Bugs

1. Invalid Gemini Model Name

In the inline Python script, the endpoint URL specifies model gemini-3.6-flash:

url = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-3.6-flash:generateContent'
  • Bug: gemini-3.6-flash is not a valid Gemini model identifier. The API call will fail with an HTTP 404 Not Found error.
  • Fix: Update the endpoint to a valid model name, such as gemini-1.5-flash or gemini-2.0-flash:
url = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent'

2. Workflow Failures on Forked PRs / Missing Secret

When a PR is opened from a repository fork (e.g., a student working on their own fork), GitHub Actions intentionally strips access to repository secrets like GEMINI_API_KEY for security reasons.

Currently, if GEMINI_API_KEY is missing, the Python script executes sys.exit(1), causing the entire GitHub Action job to fail with a red ❌ status:

if not api_key:
    print('Error: GEMINI_API_KEY secret is not set.')
    sys.exit(1)
  • Impact: Student PRs from forks will fail CI builds.
  • Fix: Gracefully handle missing API keys by posting a friendly comment or exiting with status 0 so CI passes without breaking the build checks:
if not api_key:
    print('Warning: GEMINI_API_KEY secret is not set. Skipping review.')
    with open('review.md', 'w', encoding='utf-8') as f:
        f.write('🤖 **Gemini Code Review**: Skipping review (GEMINI_API_KEY secret not accessible on this run).\n')
    sys.exit(0)

💡 Suggested Enhancements

1. Diff Size Limit vs. Gemini Context Window

In Step 2, the diff is truncated to 20,000 characters (~5,000 tokens):

head -c 20000 pr_diff.txt > trimmed_diff.txt
  • Observation: Modern models like gemini-1.5-flash have context windows exceeding 1,000,000 tokens. FRC robot code diffs—especially when refactoring subsystems or autonomous command groups—can easily exceed 20,000 characters.
  • Recommendation: You can safely increase this limit (e.g., 100000 or 200000 characters) so larger PRs aren't cut off mid-subsystem.

2. Filtering Non-Code Files (FRC-Specific)

WPILib projects frequently include large JSON vendor dependency files (vendordeps/), WPILib build configs (build.gradle), or AdvantageKit/PhotonVision JSON logs. Passing these to the AI can waste context on non-robot logic.

  • Recommendation: Consider updating gh pr diff or filtering out standard generated files:
# Example: Exclude vendordeps and gradle wrapper updates from diff
gh pr diff "$PR_NUM" --repo "${{ github.repository }}" | grep -v "vendordeps/" > pr_diff.txt

Summary Checklist

  • Change gemini-3.6-flash to gemini-1.5-flash or gemini-2.0-flash.
  • Ensure missing secrets exit cleanly with exit code 0 instead of failing the workflow step.
  • (Optional) Increase character limit for trimmed_diff.txt.

Great work setting up CI automation for the team! Make these quick tweaks and tag me for re-review! 🚀

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

⚠️ Failed to generate review from Gemini: HTTP Error 404: Not Found

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

Hey team! Great initiative setting up automated PR reviews for our robot codebase. Using Gemini to automatically check PRs for WPILib best practices and safety bugs is going to save us a lot of time during build season!

I reviewed the workflow configuration file. We have a few critical bugs that will prevent the workflow from running successfully, along with a couple of improvements for robustness and API best practices.


🚨 Critical Issues (Will Cause Workflow Failure)

1. Missing PR Diff Generation (pr_diff.txt)

In Step 2: Fetch PR Diff, the script attempts to trim pr_diff.txt:

# Example: Exclude vendordeps and gradle wrapper updates from diff
head -c 200000 pr_diff.txt > trimmed_diff.txt

However, pr_diff.txt is never created in this step! head will fail with a file-not-found error, causing the entire CI workflow to crash.

Fix: Use gh pr diff to fetch the diff before trimming it.

# Fetch the diff for the PR
gh pr diff "$PR_NUM" --repo "${{ github.repository }}" > pr_diff.txt

# Trim the diff size
head -c 200000 pr_diff.txt > trimmed_diff.txt

2. Model Endpoint Version (gemini-3.6-flash)

In the Python script, the API URL specifies gemini-3.6-flash:

url = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-3.6-flash:generateContent'

This version identifier does not exist in the Google Gemini API (standard models are currently gemini-1.5-flash, gemini-2.0-flash, etc.). The API request will fail with a 404 Not Found error.

Fix: Update the endpoint model name to gemini-2.0-flash or gemini-1.5-flash:

url = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent'

💡 Improvements & Best Practices

1. Safe JSON Response Parsing

Currently, the script assumes candidates and parts will always exist in the API response:

review_content = res_data['candidates'][0]['content']['parts'][0]['text']

If Gemini blocks a prompt due to safety filters, returns an error status, or hits a rate limit, candidates may be empty or missing, raising an IndexError or KeyError.

Fix: Add safe checking or standard get() navigation:

candidates = res_data.get('candidates', [])
if candidates and 'content' in candidates[0]:
    review_content = candidates[0]['content']['parts'][0]['text']
else:
    review_content = f"⚠️ Could not generate review. Response: {res_data}"

2. Native System Instructions Payload

Gemini's REST API supports a dedicated system_instruction parameter in the payload, which helps separate the instruction rules from the user input (the diff).

Fix: Structure the JSON payload using native system instructions:

payload = {
    "system_instruction": {
        "parts": [{"text": system_instruction}]
    },
    "contents": [
        {"parts": [{"text": f"Diff:\n{diff_text}"}]}
    ]
}
data = json.dumps(payload).encode('utf-8')

3. Note on Fork PR Secrets (GitHub Actions Security)

Keep in mind that standard pull_request workflows run in a restricted context when triggered by external forks. secrets.GEMINI_API_KEY will be empty for community/outside PRs. Your if not api_key: fallback handles this gracefully, which is great safety design!


Suggested Updated Step 2 & Step 3 Summary

Here is how Step 2 and Step 3 look with the fixes applied:

      # Step 2: Fetch PR Diff
      - name: Fetch PR Diff
        id: fetch_diff
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          if [ "${{ github.event_name }}" = "pull_request" ]; then
            PR_NUM="${{ github.event.pull_request.number }}"
          else
            PR_NUM="${{ github.event.issue.number }}"
          fi

          echo "PR_NUMBER=$PR_NUM" >> $GITHUB_ENV

          # Fetch diff using gh CLI and trim to stay within token context limits
          gh pr diff "$PR_NUM" --repo "${{ github.repository }}" > pr_diff.txt
          head -c 200000 pr_diff.txt > trimmed_diff.txt

      # Step 3: Run Gemini & Post Comment using GitHub CLI
      - name: Post Gemini Review
        env:
          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          EVENT_NAME: ${{ github.event_name }}
          USER_COMMENT: ${{ github.event.comment.body }}
        run: |
          export SYSTEM_INSTRUCTION="You are a lead mentor for an FRC programming team. Review this PR code diff for WPILib best practices, safety, and bugs. Keep feedback clear, educational, and constructive."

          if [ "$EVENT_NAME" = "issue_comment" ]; then
            EXTRA=$(echo "$USER_COMMENT" | sed 's/\/gemini//g' | xargs)
            if [ -n "$EXTRA" ]; then
              export SYSTEM_INSTRUCTION="$SYSTEM_INSTRUCTION Student note: $EXTRA"
            fi
          fi

          python3 - << 'EOF'
          import urllib.request, json, os, sys

          api_key = os.environ.get('GEMINI_API_KEY', '').strip()
          if not api_key:
              print('Warning: GEMINI_API_KEY secret is not set. Skipping review.')
              with open('review.md', 'w', encoding='utf-8') as f:
                  f.write('🤖 **Gemini Code Review**: Skipping review (GEMINI_API_KEY secret not accessible on this run).\n')
              sys.exit(0)

          url = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent'

          try:
              with open('trimmed_diff.txt', 'r', encoding='utf-8', errors='replace') as f:
                  diff_text = f.read()
          except Exception as e:
              print(f'Error reading diff: {e}')
              sys.exit(1)

          if not diff_text.strip():
              review_content = 'No code changes detected in diff.'
          else:
              system_instruction = os.environ.get('SYSTEM_INSTRUCTION', '')
              
              payload = {
                  "system_instruction": {
                      "parts": [{"text": system_instruction}]
                  },
                  "contents": [
                      {"parts": [{"text": f"Diff:\n{diff_text}"}]}
                  ]
              }
              data = json.dumps(payload).encode('utf-8')
              
              req = urllib.request.Request(
                  url,
                  data=data,
                  headers={
                      'Content-Type': 'application/json',
                      'x-goog-api-key': api_key
                  }
              )

              try:
                  with urllib.request.urlopen(req) as res:
                      res_data = json.loads(res.read().decode('utf-8'))
                      candidates = res_data.get('candidates', [])
                      if candidates and 'content' in candidates[0]:
                          review_content = candidates[0]['content']['parts'][0]['text']
                      else:
                          review_content = f"⚠️ Gemini did not return a valid review. Response payload: {res_data}"
              except Exception as e:
                  review_content = f'⚠️ Failed to generate review from Gemini: {e}'

          with open('review.md', 'w', encoding='utf-8') as f:
              f.write('🤖 **Gemini Code Review**\n\n' + review_content + '\n')
          EOF

          gh pr comment "$PR_NUMBER" --repo "${{ github.repository }}" --body-file review.md

Once these changes are pushed, test it out on a test PR! Great job setting up this automation for Team 🤖!

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

Hey team! Great initiative setting up an automated code review workflow using Gemini and GitHub Actions! This will be a super helpful tool for providing quick feedback to students on their pull requests before mentors do a manual review.

While reviewing the workflow file, I found a couple of critical issues that will cause the action to fail when triggered, along with a few recommendations to make it more robust.


🚨 Critical Issues (Will Cause Workflow Failure)

1. pr_diff.txt is referenced before it is created

In Step 2 (Fetch PR Diff), the script attempts to trim pr_diff.txt:

# Step 2: Fetch PR Diff
...
# Example: Exclude vendordeps and gradle wrapper updates from diff
head -c 200000 pr_diff.txt > trimmed_diff.txt

However, pr_diff.txt is never created prior to running head. The step will fail with head: cannot open 'pr_diff.txt' for reading: No such file or directory.

Fix: Use the GitHub CLI (gh pr diff) to fetch the diff first:

gh pr diff "$PR_NUM" --repo "${{ github.repository }}" > pr_diff.txt
head -c 200000 pr_diff.txt > trimmed_diff.txt

2. Invalid Model Endpoint Target (gemini-3.6-flash)

In the Python script:

url = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-3.6-flash:generateContent'

gemini-3.6-flash is not a valid Gemini API model identifier and will return a 404 Not Found API error.

Fix: Update the endpoint to use a standard model like gemini-2.0-flash or gemini-1.5-flash:

url = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent'

💡 Best Practices & Improvements

1. Handle Response Parsing Safely

If Gemini blocks a prompt due to safety filters, candidates[0]['content'] may not be present in the JSON response, leading to a KeyError or IndexError.

Recommended Improvement:

candidates = res_data.get('candidates', [])
if candidates and 'content' in candidates[0] and 'parts' in candidates[0]['content']:
    review_content = candidates[0]['content']['parts'][0].get('text', '')
else:
    finish_reason = candidates[0].get('finishReason', 'UNKNOWN') if candidates else 'NO_CANDIDATES'
    review_content = f"⚠️ Could not generate review. Reason: {finish_reason}"

2. Clean up Comment Parameter Parsing

In Step 3, using sed and xargs in shell scripts can break if a comment contains multiline text or special characters. Moving the logic entirely inside Python keeps things clean and cross-platform.


🛠️ Suggested Code Fix

Here is how Steps 2 and 3 should look:

      # Step 2: Fetch PR Diff
      - name: Fetch PR Diff
        id: fetch_diff
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          if [ "${{ github.event_name }}" = "pull_request" ]; then
            PR_NUM="${{ github.event.pull_request.number }}"
          else
            PR_NUM="${{ github.event.issue.number }}"
          fi

          echo "PR_NUMBER=$PR_NUM" >> $GITHUB_ENV

          # Fetch diff using gh CLI and trim large diffs
          gh pr diff "$PR_NUM" --repo "${{ github.repository }}" > pr_diff.txt
          head -c 200000 pr_diff.txt > trimmed_diff.txt

      # Step 3: Run Gemini & Post Comment using GitHub CLI
      - name: Post Gemini Review
        env:
          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          EVENT_NAME: ${{ github.event_name }}
          USER_COMMENT: ${{ github.event.comment.body }}
        run: |
          export SYSTEM_INSTRUCTION="You are a lead mentor for an FRC programming team. Review this PR code diff for WPILib best practices, safety, and bugs. Keep feedback clear, educational, and constructive."

          python3 - << 'EOF'
          import urllib.request, json, os, sys

          api_key = os.environ.get('GEMINI_API_KEY', '').strip()
          if not api_key:
              print('Warning: GEMINI_API_KEY secret is not set. Skipping review.')
              with open('review.md', 'w', encoding='utf-8') as f:
                  f.write('🤖 **Gemini Code Review**: Skipping review (GEMINI_API_KEY secret not set).\n')
              sys.exit(0)

          url = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent'

          try:
              with open('trimmed_diff.txt', 'r', encoding='utf-8', errors='replace') as f:
                  diff_text = f.read()
          except Exception as e:
              print(f'Error reading diff: {e}')
              sys.exit(1)

          if not diff_text.strip():
              review_content = 'No code changes detected in diff.'
          else:
              system_instruction = os.environ.get('SYSTEM_INSTRUCTION', '')
              event_name = os.environ.get('EVENT_NAME', '')
              user_comment = os.environ.get('USER_COMMENT', '')

              if event_name == 'issue_comment' and '/gemini' in user_comment:
                  extra = user_comment.replace('/gemini', '').strip()
                  if extra:
                      system_instruction += f" Student note: {extra}"

              payload = {
                  "system_instruction": {
                      "parts": [{"text": system_instruction}]
                  },
                  "contents": [
                      {"parts": [{"text": f"Diff:\n{diff_text}"}]}
                  ]
              }
              data = json.dumps(payload).encode('utf-8')
              req = urllib.request.Request(
                  url,
                  data=data,
                  headers={
                      'Content-Type': 'application/json',
                      'x-goog-api-key': api_key
                  }
              )

              try:
                  with urllib.request.urlopen(req) as res:
                      res_data = json.loads(res.read().decode('utf-8'))
                      candidates = res_data.get('candidates', [])
                      if candidates and 'content' in candidates[0] and 'parts' in candidates[0]['content']:
                          review_content = candidates[0]['content']['parts'][0].get('text', '')
                      else:
                          review_content = f"⚠️ Could not generate review. Response: {res_data}"
              except Exception as e:
                  review_content = f'⚠️ Failed to generate review from Gemini: {e}'

          with open('review.md', 'w', encoding='utf-8') as f:
              f.write('🤖 **Gemini Code Review**\n\n' + review_content + '\n')
          EOF

          gh pr comment "$PR_NUMBER" --repo "${{ github.repository }}" --body-file review.md

Awesome job setting this up! Once these two fixes are pushed, this automation should work smoothly for team PRs. Let me know if you have any questions!

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

Hey team! This is a great initiative to automate code reviews and help teach team members WPILib best practices right in Pull Requests.

I reviewed the GitHub Actions workflow configuration and found a few critical issues that will cause the workflow to fail at runtime, as well as an unused file fetch.


🚨 Critical Issues & Bugs

1. pr_diff.txt is referenced before being generated

In Step 2: Fetch PR Diff, the script tries to truncate pr_diff.txt:

head -c 200000 pr_diff.txt > trimmed_diff.txt

However, the command to actually fetch the diff (e.g., gh pr diff "$PR_NUM" > pr_diff.txt) was omitted. When this runs in CI, head will fail with No such file or directory, causing the job to crash.

Fix: Add gh pr diff "$PR_NUM" > pr_diff.txt before running head.


2. reviewer-rules.md is fetched but never sent to Gemini

In Step 2, you fetch reviewer-rules.md into rules.txt:

gh api "repos/${{ github.repository }}/contents/.github/reviewer-rules.md" ... > rules.txt

However, in Step 3, the Python script never reads rules.txt to pass its contents into the Gemini system_instruction prompt payload. As a result, the custom team instructions (WPILib rules, CAN bus safety rules, etc.) won't be used during the AI review.

Fix: Read rules.txt inside the Python script and append it to system_instruction.


💡 Mentoring & Best Practice Suggestions

  1. Handling PRs from Forks (GEMINI_API_KEY):
    GitHub Actions blocks secrets (like GEMINI_API_KEY) on standard pull_request workflows triggered by forks for security reasons. Your Python script handles the missing key gracefully with a skip message, which is great fallback handling! If your team ever opens PRs from personal forks, consider using issue_comment triggers or appropriate secret management.

🛠️ Suggested Fixes for gemini-review.yml

Here is how you can update Step 2 and Step 3 in .github/workflows/gemini-review.yml:

      # Step 2: Fetch PR Diff & Reviewer Rules
      - name: Fetch PR Diff
        id: fetch_diff
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          if [ "${{ github.event_name }}" = "pull_request" ]; then
            PR_NUM="${{ github.event.pull_request.number }}"
          else
            PR_NUM="${{ github.event.issue.number }}"
          fi

          echo "PR_NUMBER=$PR_NUM" >> $GITHUB_ENV

          # Fetch actual diff from GH CLI
          gh pr diff "$PR_NUM" --repo "${{ github.repository }}" > pr_diff.txt
          head -c 200000 pr_diff.txt > trimmed_diff.txt

          # Fetch reviewer-rules.md via GitHub API if it exists
          gh api "repos/${{ github.repository }}/contents/.github/reviewer-rules.md" \
            -H "Accept: application/vnd.github.raw+json" > rules.txt 2>/dev.null || touch rules.txt

And in Step 3's inline Python script, read rules.txt:

          # Combine default instruction with reviewer-rules.md content
          system_instruction = os.environ.get('SYSTEM_INSTRUCTION', '')
          if os.path.exists('rules.txt'):
              with open('rules.txt', 'r', encoding='utf-8', errors='ignore') as f:
                  rules_content = f.read().strip()
                  if rules_content:
                      system_instruction += f"\n\nTeam Specific Guidelines:\n{rules_content}"

Once these changes are made, the workflow should run smoothly! Great work setting this up for the team.

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

Hey team! Great work setting up an automated PR review workflow with Gemini. Adding automated checks like this is a huge help for keeping our WPILib standards high and giving students quick feedback before mentors jump in for final reviews.

Here is a review of the workflow setup and a few small improvements we can make for safety, maintainability, and reliability.


Key Observations & Recommendations

1. Bash String Handling for User Comments (Robustness)

In gemini-review.yml:

if [ "$EVENT_NAME" = "issue_comment" ]; then
  EXTRA=$(echo "$USER_COMMENT" | sed 's/\/gemini//g' | xargs)
  if [ -n "$EXTRA" ]; then
    export SYSTEM_INSTRUCTION="$SYSTEM_INSTRUCTION Student note: $EXTRA"
  fi
fi
  • Why it's an issue: If a student types a comment with special shell characters, newlines, or quotes (e.g., /gemini Fix for shooter CAN ID #3), running echo and xargs directly on the input in Bash can cause unexpected behavior or syntax errors.
  • Educational Tip: Whenever processing freeform user input, it's safer and cleaner to handle string manipulation inside Python using os.environ.get('USER_COMMENT') rather than Bash pipelines.

Suggested Fix: Move the comment parsing into the Python block:

user_comment = os.environ.get('USER_COMMENT', '')
if os.environ.get('EVENT_NAME') == 'issue_comment' and '/gemini' in user_comment:
    extra_note = user_comment.replace('/gemini', '').strip()
    if extra_note:
        system_instruction += f"\nStudent note: {extra_note}"

2. API Error Response Details (Debugging)

In the Python review script:

try:
    with urllib.request.urlopen(req) as res:
        ...
except Exception as e:
    review_content = f'⚠️ Failed to generate review from Gemini: {e}'
  • Why it's an issue: When urllib.request.urlopen fails due to an HTTP error (e.g., 400 Bad Request or 429 Rate Limit), e will just evaluate to HTTP Error 400: Bad Request, which leaves out the details returned in the JSON response body.
  • Recommendation: Catch urllib.error.HTTPError explicitly and read e.read().decode('utf-8') to capture the underlying API error message. This makes troubleshooting much easier when testing new prompts or rules.

Example Catch Block:

import urllib.error

try:
    with urllib.request.urlopen(req) as res:
        res_data = json.loads(res.read().decode('utf-8'))
        # ... process response
except urllib.error.HTTPError as e:
    err_body = e.read().decode('utf-8')
    review_content = f'⚠️ Gemini API Error ({e.code}): {err_body}'
except Exception as e:
    review_content = f'⚠️ Failed to generate review from Gemini: {e}'

3. Rules Fetching from Branches

In Step 2:

gh api "repos/${{ github.repository }}/contents/.github/reviewer-rules.md" \
  -H "Accept: application/vnd.github.raw+json" > rules.txt 2>/dev.null || touch rules.txt
  • Note: Calling the API endpoint without a ?ref= parameter defaults to fetching the file from the default branch (main). If a student PR modifies .github/reviewer-rules.md, those rule updates won't take effect until merged into main.
  • Recommendation: If you want rule changes tested within the PR that edits them, pass ?ref=${PR_BRANCH} or check out the branch using actions/checkout. If intended to enforce only main-approved rules, the current behavior is completely fine!

Summary

Overall, this is a clean, dependency-free Python + GitHub CLI implementation that avoids external library overhead. Once we tweak the string handling and HTTP error logging, this workflow will be rock solid for the team. Nice job!

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

Hey team! Great job setting up this automated code review action for our repository. Integrating LLM-assisted code reviews into our PR workflow is a fantastic way to give students quick feedback on WPILib conventions and safety before mentor reviews.

Here is a review of the workflow and configuration files from a best practices, safety, and reliability standpoint.


Highlights & Good Practices 👏

  1. Graceful Degradation for Secrets: Handling GEMINI_API_KEY when missing (e.g., PRs submitted from external forks where secrets are suppressed by GitHub) prevents workflow step crashes and posts a helpful message instead.
  2. UTF-8 Truncation Safety: Using errors='replace' when reading trimmed_diff.txt in Python protects against potential byte-splitting issues when truncating multi-byte UTF-8 characters via head -c.
  3. Domain Guidelines: Embedding reviewer-rules.md into the prompt system instruction ensures domain-specific context (like CAN bus safety and addRequirements) is consistently passed to the model.

Potential Issues & Recommended Improvements 🔧

1. Fragile Shell Manipulation of User Comments (xargs / sed)

In gemini-review.yml step 3:

if [ "$EVENT_NAME" = "issue_comment" ]; then
  EXTRA=$(echo "$USER_COMMENT" | sed 's/\/gemini//g' | xargs)
  if [ -n "$EXTRA" ]; then
    export SYSTEM_INSTRUCTION="$SYSTEM_INSTRUCTION Student note: $EXTRA"
  fi
fi

Why this is an issue:

  • xargs fails or behaves unexpectedly if the user comment contains single/double quotes, quotes across multiple lines, or backslashes (e.g., /gemini Can we review the SparkMax initialization?).
  • Multi-line user comments will be collapsed or stripped incorrectly by xargs.

Recommended Fix:
Handle user comment extraction directly in Python where string processing is safer and cleaner.

# Inside the inline Python script:
event_name = os.environ.get('EVENT_NAME', '')
user_comment = os.environ.get('USER_COMMENT', '')

system_instruction = os.environ.get('SYSTEM_INSTRUCTION', '')

if event_name == 'issue_comment' and '/gemini' in user_comment:
    extra_note = user_comment.replace('/gemini', '').strip()
    if extra_note:
        system_instruction += f"\n\nStudent note: {extra_note}"

2. Workflow Trigger Scope for Issue Comments

In gemini-review.yml:

on:
  pull_request:
    types: [opened, synchronize, reopened]
  issue_comment:
    types: [created]

Why this matters:
The issue_comment event fires for both standard repository issues and PR comments. The job filter (if: ... github.event.issue.pull_request ...) correctly handles this by skipping standard issue comments, which is good! However, ensure team members know that typing /gemini on regular GitHub Issues won't trigger a review, only PR comments.


Summary Checklist

Area Status Notes
Model Version ✅ Pass Pinning gemini-3.6-flash as specified in rules.
Permissions ✅ Pass Minimal write permissions for PR/issue commenting.
String Parsing ⚠️ Needs Attention Replace shell sed/xargs with native Python parsing.

Once the shell string handling is refactored into Python, this PR is ready to merge! Great work setting this infrastructure up for the team.

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review Summary

This PR adds a automated code review workflow using Gemini via GitHub Actions, alongside reviewer instructions in .github/reviewer-rules.md. The workflow script fetches PR diffs and posts high-level summaries and inline review comments. A few improvements can be made to the workflow's PR ref fetching, string parsing robustness, and error handling when posting comments.

Comment thread .github/workflows/gemini-review.yml
Comment thread .github/workflows/gemini-review.yml
Comment thread .github/workflows/gemini-review.yml
Comment thread .github/workflows/gemini-review.yml Outdated
@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review Summary

Reviewed the workflow configuration and the Python script used for the code review. The pipeline securely handles payload values using environment variables rather than direct bash interpolations. Recommended improvements focus on protecting against null references in GitHub Actions expressions and adding robust error handling for highly nested API responses.

Comment thread .github/workflows/gemini-review.yml
Comment thread .github/workflows/gemini-review.yml
@koolpoolo
koolpoolo self-requested a review August 14, 2026 06:41
@koolpoolo
koolpoolo merged commit 6c098eb into main Aug 14, 2026
3 checks passed
@koolpoolo
koolpoolo deleted the markpete/gemini branch August 14, 2026 06:43
@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review Summary

Added project reviewer rules documentation and configured a GitHub Actions workflow for automated AI PR reviews using Gemini.


# Download PR diff
gh pr diff "$PR_NUM" --repo "${{ github.repository }}" > pr_diff.txt
head -c 25000 pr_diff.txt > trimmed_diff.txt

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Using head -c 25000 truncates the diff by exact byte count, which can cut off a line or diff hunk in the middle. Using line-based truncation (e.g., head -n 500) preserves complete diff lines and prevents sending malformed diff fragments to the reviewer model.

raw_text = res_data['candidates'][0]['content']['parts'][0]['text'].strip()
# Clean potential markdown wraps from response
if raw_text.startswith("```json"):
raw_text = raw_text[7:]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Trailing whitespace or newlines after the closing backticks can cause raw_text.endswith("```") to evaluate to False. Calling raw_text = raw_text.strip() right before checking endswith("```") ensures markdown code block formatting is reliably stripped before JSON parsing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants