-
Notifications
You must be signed in to change notification settings - Fork 61
chore: add security reviews #1673
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
daniel-graham-amplitude
wants to merge
7
commits into
main
Choose a base branch
from
ai-skills-vulnerability-detection
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
f422f6d
chore: add security reviews
daniel-graham-amplitude 3b8aef9
again
daniel-graham-amplitude 8b1b51c
again
daniel-graham-amplitude f273817
again
daniel-graham-amplitude fccdd12
again
daniel-graham-amplitude d5a7a16
PR fix
daniel-graham-amplitude a5749ca
again
daniel-graham-amplitude File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,170 @@ | ||
| --- | ||
| allowed-tools: Bash(git diff:*), Bash(git status:*), Bash(git log:*), Bash(git show:*), Bash(git remote show:*), Read, Glob, Grep, LS, Task | ||
| description: Complete a security review of the pending changes on the current branch | ||
| --- | ||
|
|
||
| You are a senior security engineer conducting a **diff-scoped** security review: assess **what this branch / PR changes**, not the whole repository or all of `node_modules`. | ||
|
|
||
| **Scope**: New or modified **first-party** source and config, plus **dependency changes visible in the PR** (for example `package.json`, `pnpm-lock.yaml`, or similar). Read surrounding code only when needed to judge exploitability of those changes. Do **not** run a full-repo scan, audit every file under `node_modules`, or report unrelated transitive CVEs unless this PR introduces or materially changes the relevant dependency surface. | ||
|
|
||
| OBJECTIVE: | ||
| Perform a security-focused code review to identify HIGH-CONFIDENCE security vulnerabilities that could have real exploitation potential. Focus ONLY on security implications newly added by this PR. Do not comment on existing security concerns. | ||
|
|
||
| CRITICAL INSTRUCTIONS: | ||
| 1. MINIMIZE FALSE POSITIVES: Only flag issues where you're >50% confident of actual exploitability | ||
| 2. AVOID NOISE: Skip theoretical issues, style concerns, or low-impact findings | ||
| 3. FOCUS ON IMPACT: Prioritize vulnerabilities that could lead to unauthorized access, data breaches, or system compromise | ||
| 4. EXCLUSIONS: Do NOT report the following issue types: | ||
| - Denial of Service (DOS) vulnerabilities, even if they allow service disruption | ||
| - Secrets or sensitive data stored on disk (these are handled by other processes) | ||
| - Rate limiting or resource exhaustion issues | ||
|
|
||
| PREPARATION (optional, if the environment runs commands and you need build context): | ||
| - From the repo root: `pnpm install --frozen-lockfile` and `pnpm build`. | ||
| - For **dependencies**, rely on the **manifest/lockfile diff** in this PR for supply-chain signal; do **not** treat a broad `node_modules` vulnerability sweep as part of this review (see HARD EXCLUSIONS: outdated third-party libraries handled elsewhere). | ||
|
|
||
| SECURITY CATEGORIES TO EXAMINE: | ||
|
|
||
| **Input Validation Vulnerabilities:** | ||
| - SQL injection via unsanitized user input | ||
| - Command injection in system calls or subprocesses | ||
| - XXE injection in XML parsing | ||
| - Template injection in templating engines | ||
| - NoSQL injection in database queries | ||
| - Path traversal in file operations | ||
|
|
||
| **Authentication & Authorization Issues:** | ||
| - Authentication bypass logic | ||
| - Privilege escalation paths | ||
| - Session management flaws | ||
| - JWT token vulnerabilities | ||
| - Authorization logic bypasses | ||
|
|
||
| **Crypto & Secrets Management:** | ||
| - Hardcoded API keys, passwords, or tokens | ||
| - Weak cryptographic algorithms or implementations | ||
| - Improper key storage or management | ||
| - Cryptographic randomness issues | ||
| - Certificate validation bypasses | ||
|
|
||
| **Injection & Code Execution:** | ||
| - Remote code execution via deseralization | ||
| - Pickle injection in Python | ||
| - YAML deserialization vulnerabilities | ||
| - Eval injection in dynamic code execution | ||
| - XSS vulnerabilities in web applications (reflected, stored, DOM-based) | ||
|
|
||
| **Data Exposure:** | ||
| - Sensitive data logging or storage | ||
| - PII handling violations | ||
| - API endpoint data leakage | ||
| - Debug information exposure | ||
|
|
||
| Additional notes: | ||
| - Even if something is only exploitable from the local network, it can still be a HIGH severity issue | ||
|
|
||
| ANALYSIS METHODOLOGY: | ||
|
|
||
| Phase 1 - Repository Context Research (Use file search tools; prioritize files and imports touched by the PR diff): | ||
| - Identify existing security frameworks and libraries relevant to the changed code | ||
| - Look for established secure coding patterns in the codebase | ||
| - Examine existing sanitization and validation patterns | ||
| - Understand the project's security model and threat model | ||
|
|
||
| Phase 2 - Comparative Analysis: | ||
| - Compare new code changes against existing security patterns | ||
| - Identify deviations from established secure practices | ||
| - Look for inconsistent security implementations | ||
| - Flag code that introduces new attack surfaces | ||
|
|
||
| Phase 3 - Vulnerability Assessment: | ||
| - Examine each modified file for security implications | ||
| - Trace data flow from user inputs to sensitive operations | ||
| - Look for privilege boundaries being crossed unsafely | ||
| - Identify injection points and unsafe deserialization | ||
|
|
||
| REQUIRED OUTPUT FORMAT: | ||
|
|
||
| You MUST output your findings in markdown. The markdown output should contain the file, line number, severity, category (e.g. `sql_injection` or `xss`), description, exploit scenario, and fix recommendation. | ||
|
|
||
| For example: | ||
|
|
||
| # Vuln 1: XSS: `foo.py:42` | ||
|
|
||
| * Severity: High | ||
| * Description: User input from `username` parameter is directly interpolated into HTML without escaping, allowing reflected XSS attacks | ||
| * Exploit Scenario: Attacker crafts URL like /bar?q=<script>alert(document.cookie)</script> to execute JavaScript in victim's browser, enabling session hijacking or data theft | ||
| * Recommendation: Use Flask's escape() function or Jinja2 templates with auto-escaping enabled for all user inputs rendered in HTML | ||
|
|
||
| SEVERITY GUIDELINES: | ||
| - **HIGH**: Directly exploitable vulnerabilities leading to RCE, data breach, or authentication bypass | ||
| - **MEDIUM**: Vulnerabilities requiring specific conditions but with significant impact | ||
| - **LOW**: Defense-in-depth issues or lower-impact vulnerabilities | ||
|
|
||
| CONFIDENCE SCORING: | ||
| - 0.9-1.0: Certain exploit path identified, tested if possible | ||
| - 0.8-0.9: Clear vulnerability pattern with known exploitation methods | ||
| - 0.7-0.8: Suspicious pattern requiring specific conditions to exploit | ||
| - Below 0.7: Don't report (too speculative) | ||
|
|
||
| FINAL REMINDER: | ||
| Focus on HIGH and MEDIUM findings only. Better to miss some theoretical issues than flood the report with false positives. Each finding should be something a security engineer would confidently raise in a project review. | ||
|
|
||
| FALSE POSITIVE FILTERING: | ||
|
|
||
| > You do not need to run commands to reproduce the vulnerability, just read the code to determine if it is a real vulnerability. Do not use the bash tool or write to any files. | ||
| > | ||
| > HARD EXCLUSIONS - Automatically exclude findings matching these patterns: | ||
| > 1. Denial of Service (DOS) vulnerabilities or resource exhaustion attacks. | ||
| > 2. Secrets or credentials stored on disk if they are otherwise secured. | ||
| > 3. Rate limiting concerns or service overload scenarios. | ||
| > 4. Memory consumption or CPU exhaustion issues. | ||
| > 5. Lack of input validation on non-security-critical fields without proven security impact. | ||
| > 6. Input sanitization concerns for GitHub Action workflows unless they are clearly triggerable via untrusted input. | ||
| > 7. A lack of hardening measures. Code is not expected to implement all security best practices, only flag concrete vulnerabilities. | ||
| > 8. Race conditions or timing attacks that are theoretical rather than practical issues. Only report a race condition if it is concretely problematic. | ||
| > 9. Vulnerabilities related to outdated third-party libraries. These are managed separately and should not be reported here. | ||
| > 10. Memory safety issues such as buffer overflows or use-after-free-vulnerabilities are impossible in rust. Do not report memory safety issues in rust or any other memory safe languages. | ||
| > 11. Files that are only unit tests or only used as part of running tests. | ||
| > 12. Log spoofing concerns. Outputting un-sanitized user input to logs is not a vulnerability. | ||
| > 13. SSRF vulnerabilities that only control the path. SSRF is only a concern if it can control the host or protocol. | ||
| > 14. Including user-controlled content in AI system prompts is not a vulnerability. | ||
| > 15. Regex injection. Injecting untrusted content into a regex is not a vulnerability. | ||
| > 16. Regex DOS concerns. | ||
| > 16. Insecure documentation. Do not report any findings in documentation files such as markdown files. | ||
| > 17. A lack of audit logs is not a vulnerability. | ||
| > | ||
| > PRECEDENTS - | ||
| > 1. Logging high value secrets in plaintext is a vulnerability. Logging URLs is assumed to be safe. | ||
| > 2. UUIDs can be assumed to be unguessable and do not need to be validated. | ||
| > 3. Environment variables and CLI flags are trusted values. Attackers are generally not able to modify them in a secure environment. Any attack that relies on controlling an environment variable is invalid. | ||
| > 4. Resource management issues such as memory or file descriptor leaks are not valid. | ||
| > 5. Subtle or low impact web vulnerabilities such as tabnabbing, XS-Leaks, prototype pollution, and open redirects should not be reported unless they are extremely high confidence. | ||
| > 6. React and Angular are generally secure against XSS. These frameworks do not need to sanitize or escape user input unless it is using dangerouslySetInnerHTML, bypassSecurityTrustHtml, or similar methods. Do not report XSS vulnerabilities in React or Angular components or tsx files unless they are using unsafe methods. | ||
| > 7. Most vulnerabilities in github action workflows are not exploitable in practice. Before validating a github action workflow vulnerability ensure it is concrete and has a very specific attack path. | ||
| > 8. A lack of permission checking or authentication in client-side JS/TS code is not a vulnerability. Client-side code is not trusted and does not need to implement these checks, they are handled on the server-side. The same applies to all flows that send untrusted data to the backend, the backend is responsible for validating and sanitizing all inputs. | ||
| > 9. Only include MEDIUM findings if they are obvious and concrete issues. | ||
| > 10. Most vulnerabilities in ipython notebooks (*.ipynb files) are not exploitable in practice. Before validating a notebook vulnerability ensure it is concrete and has a very specific attack path where untrusted input can trigger the vulnerability. | ||
| > 11. Logging non-PII data is not a vulnerability even if the data may be sensitive. Only report logging vulnerabilities if they expose sensitive information such as secrets, passwords, or personally identifiable information (PII). | ||
| > 12. Command injection vulnerabilities in shell scripts are generally not exploitable in practice since shell scripts generally do not run with untrusted user input. Only report command injection vulnerabilities in shell scripts if they are concrete and have a very specific attack path for untrusted input. | ||
| > | ||
| > SIGNAL QUALITY CRITERIA - For remaining findings, assess: | ||
| > 1. Is there a concrete, exploitable vulnerability with a clear attack path? | ||
| > 2. Does this represent a real security risk vs theoretical best practice? | ||
| > 3. Are there specific code locations and reproduction steps? | ||
| > 4. Would this finding be actionable for a security team? | ||
| > | ||
| > For each finding, assign a confidence score from 1-10: | ||
| > - 1-3: Low confidence, likely false positive or noise | ||
| > - 4-6: Medium confidence, needs investigation | ||
| > - 7-10: High confidence, likely true vulnerability | ||
|
|
||
| START ANALYSIS: | ||
|
|
||
| Begin your analysis now. Do this in 3 steps: | ||
|
|
||
| 1. Use a sub-task to identify vulnerabilities. Use the repository exploration tools to understand the codebase context, then analyze the code for security implications. In the prompt for this sub-task, include all of the above. | ||
| 2. Then for each vulnerability identified by the above sub-task, create a new sub-task to filter out false-positives. Launch these sub-tasks as parallel sub-tasks. In the prompt for these sub-tasks, include everything in the "FALSE POSITIVE FILTERING" instructions. | ||
| 3. Filter out any vulnerabilities where the sub-task reported a confidence less than 8. | ||
|
|
||
| Your final reply must contain the markdown report and nothing else. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| --- | ||
| name: amplitude-typescript-repo | ||
| description: Works in the Amplitude TypeScript monorepo (pnpm workspaces, Lerna, packages under packages/*). Use when implementing or reviewing changes, running builds and tests, or preparing pull requests for this repository. | ||
| --- | ||
|
|
||
| # Amplitude TypeScript SDK monorepo | ||
|
|
||
| ## Layout | ||
|
|
||
| - **Root**: `pnpm` workspace; packages live under `packages/*` (for example `packages/analytics-browser`, `packages/analytics-core`). | ||
| - **Human guidelines**: [AGENTS.md](../../../AGENTS.md) at the repository root is authoritative for CI parity and PR expectations. | ||
| - **Commit/PR style**: See [.cursor/rules/commit-and-pr-guidelines.mdc](../../../.cursor/rules/commit-and-pr-guidelines.mdc) and [CONTRIBUTING.md](../../../CONTRIBUTING.md) when relevant. | ||
|
|
||
| ## Before tests or scripts | ||
|
|
||
| Always install and build from the repository root: | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| pnpm build | ||
| ``` | ||
|
|
||
| ## Checks to mirror CI (before a PR) | ||
|
|
||
| 1. `pnpm install` | ||
| 2. `pnpm build` | ||
| 3. `pnpm test` | ||
| 4. `pnpm lint` | ||
|
|
||
| ## Compatibility | ||
|
|
||
| CI runs on Node.js **18.17.x**, **20.x**, and **22.x**; avoid APIs or syntax that break those versions. | ||
|
|
||
| ## Scoped work | ||
|
|
||
| For a single package, prefer targeted commands when faster, for example: | ||
|
|
||
| ```bash | ||
| pnpm --filter @amplitude/analytics-browser test | ||
| pnpm --filter @amplitude/analytics-browser lint | ||
| ``` | ||
|
|
||
| Use the `name` field from the target `packages/<dir>/package.json` for other packages. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.