Skip to content

fix(core): prevent indirect prompt injection via build file modifications and untrusted flags - #29250

Merged
DavidAPierce merged 19 commits into
google-gemini:mainfrom
villahernandez-coder:FixBug-cla-445881265
Sep 11, 2026
Merged

DavidAPierce merged 19 commits into
google-gemini:mainfrom
villahernandez-coder:FixBug-cla-445881265

Conversation

@villahernandez-coder

@villahernandez-coder villahernandez-coder commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR implements robust mechanisms to improve workspace boundary validation (specifically focusing on build configuration files and external command parameters) under restricted workspace mode. It refactors built-in execution paths (including shell, edit, and write_file) to check for command flags or arguments guided by external context tags (such as Google Docs, web fetch, or MCP server responses) and require explicit user confirmation.

Details

  1. External Context Processor: Refactored the tracking utilities to scan conversation history, extract tokens from <untrusted_context> blocks, and verify whether a command contains parameters guided by external input.
  2. Build Configuration Tracking: Implemented tracking of build configuration files (e.g., package.json, Makefile, pyproject.toml, BUILD.bazel) when edited or created. If build configuration changes are detected within the current session, any subsequent build or test command execution (such as npm run, make, cargo, blaze) is surfaced for explicit user confirmation.
  3. Refined User Confirmation UI: Updated ToolConfirmationMessage to display detailed context to the user, highlighting specific parameters or recent build modifications, and explaining the execution options.

Related Issues

Related to FixBug-cla-445881265

How to Validate

  1. Run unit tests in CLI to verify warning UI:
    npm test -w @google/gemini-cli

Pre-Merge Checklist

  • [ x] Updated relevant documentation and README (if needed)
  • [ x] Added/updated tests (if needed)
  • Noted breaking changes (if any)
  • [ x] Validated on required platforms/methods:
    • MacOS
      • npm run
      • npx
      • Docker
      • Podman
      • Seatbelt
    • Windows
      • npm run
      • npx
      • Docker
    • [ x] Linux
      • [ x] npm run
      • npx
      • Docker

@villahernandez-coder
villahernandez-coder requested review from a team as code owners September 8, 2026 18:45
@google-cla

google-cla Bot commented Sep 8, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@github-actions github-actions Bot added the size/xl An extra large PR label Sep 8, 2026
@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

📊 PR Size: size/XL

  • Lines changed: 1593
  • Additions: +1529
  • Deletions: -64
  • Files changed: 19

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a robust security layer to mitigate indirect prompt injection attacks by tracking untrusted context and enforcing strict confirmation requirements for build-related operations. It adds mechanisms to detect untrusted shell flags and mandates user approval when modifying or executing build configuration files, significantly hardening the workspace against unauthorized modifications and command execution.

Highlights

  • Untrusted Context Tracker: Introduced a utility to monitor conversation history for external, unauthenticated inputs wrapped in <untrusted_context> tags and flag potentially malicious shell command parameters.
  • Build File Protection: Implemented mandatory user confirmation for editing or writing to critical build configuration files such as package.json, Makefile, and BUILD.bazel.
  • Build Execution Security: Added tracking for build file modifications within a session, requiring explicit user confirmation for subsequent build or test commands.
  • UI/UX Enhancements: Updated ToolConfirmationMessage to display critical security warnings and suppress persistent approvals for sensitive operations involving untrusted flags or build modifications.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces build file protection and untrusted context tracking to prevent unauthorized build modifications and command execution with untrusted flags. The reviewer provided critical feedback highlighting several security and architectural vulnerabilities: naive whitespace splitting and command parsing in findUntrustedFlags and isBuildOrTestCommand can be bypassed with quotes or prefixed environment variables; the policy engine's build file protection can be bypassed via MCP filesystem tools; and tracking modified build files using a module-level global Set violates repository rules against global state, risking race conditions in concurrent environments. The reviewer recommended using shell-quote and getCommandRoots for robust command parsing, expanding tool name matching to cover MCP tools, adopting a session-scoped WeakMap for tracking modified files, and suppressing persistent approval options in the UI when modified build files are detected.

Note: Security Review did not run due to the size of the PR.

Comment thread packages/core/src/utils/untrustedContextTracker.ts Outdated
Comment thread packages/core/src/utils/untrustedContextTracker.ts
Comment thread packages/core/src/policy/policy-engine.ts Outdated
Comment thread packages/core/src/utils/untrustedContextTracker.ts
Comment thread packages/core/src/utils/untrustedContextTracker.ts Outdated
Comment thread packages/cli/src/ui/components/messages/ToolConfirmationMessage.tsx Outdated
Comment thread packages/core/src/tools/edit.ts
Comment thread packages/core/src/tools/write-file.ts
Comment thread packages/core/src/tools/shell.ts
Comment thread packages/core/src/tools/shell.ts Outdated
@gemini-cli gemini-cli Bot added the status/need-issue Pull requests that need to have an associated issue. label Sep 8, 2026
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces robust build file protection and untrusted command flag detection to enhance execution security. It adds utilities to identify build files and track untrusted context, ensuring that modifications to build configuration files or execution of commands with untrusted flags always require explicit user confirmation (downgrading to ASK_USER or DENY in non-interactive mode) and suppress persistent approvals. Additionally, it updates the CLI UI to display critical security warnings and prevent truncation of build file diffs. A high-severity issue was identified in packages/core/src/tools/shell.ts where getModifiedBuildFiles() is called without passing the session key (this.context.config), which would cause it to default to the global session key and fail to retrieve modified build files recorded with the active configuration.

Note: Security Review did not run due to the size of the PR.

Comment thread packages/core/src/tools/shell.ts Outdated
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

1 similar comment
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces robust build file protection and untrusted command flag detection mechanisms to prevent malicious code execution. It adds utilities to identify build files and track untrusted context, updates the policy engine to require user confirmation for build file modifications, and enhances the CLI confirmation UI with critical security warnings. The review identified two critical issues: first, getModifiedBuildFiles in the shell tool is called without the session key, causing it to return an empty array and bypass the build file caution warning; second, applyShellHeuristics in the policy engine is missing the shellDirPath argument, which bypasses workspace boundary checks.

Note: Security Review did not run due to the size of the PR.

Comment thread packages/core/src/tools/shell.ts Outdated
Comment thread packages/core/src/policy/policy-engine.ts

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces robust security features, including Build File Protection and Untrusted Command Flags Detection, to prevent unauthorized modifications to build configurations and execution of untrusted commands. It adds utilities to identify build files and track untrusted context, updates the policy engine to require user confirmation for build file edits, and enhances the CLI to display critical security warnings and suppress persistent approvals. Feedback on the changes highlights a critical bug in shell.ts where getModifiedBuildFiles() is called without the session key, bypassing the caution warning, and a missing dependency (availableTerminalHeight) in a useCallback hook within ToolConfirmationMessage.tsx.

Note: Security Review did not run due to the size of the PR.

Comment thread packages/core/src/tools/shell.ts Outdated
Comment thread packages/cli/src/ui/components/messages/ToolConfirmationMessage.tsx
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a robust Build File Protection mechanism and untrusted command flag detection to enhance CLI security. It ensures that modifications to build configuration files (such as Bazel BUILD, package.json, and Makefiles) always require explicit user confirmation, even in automated or YOLO modes. It also tracks build files modified during a session to warn users before executing subsequent build commands, and detects untrusted command flags sourced from external contexts. The review feedback identifies a critical security vulnerability where redirection targets (e.g., shell output redirection) are ignored during token extraction, which could allow arbitrary file writes to bypass detection. Additionally, it is recommended to replace the module-level global regular expression with String.prototype.matchAll() to prevent state leakage and potential race conditions.

Note: Security Review did not run due to the size of the PR.

Comment thread packages/core/src/utils/untrustedContextTracker.ts
Comment thread packages/core/src/utils/untrustedContextTracker.ts Outdated
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces Build File Protection and Untrusted Command Flags Detection to prevent unauthorized build configuration modifications and execution of untrusted command arguments. It adds utilities to identify build files, track untrusted conversation context, and warn users during tool execution. The review feedback highlights critical security bypasses on Windows platforms (due to path normalization issues and backslash escaping in shell parsing) and recommends avoiding module-level global state in the tracker to prevent cross-session data leakage in concurrent environments.

Comment thread packages/core/src/policy/policy-engine.ts
Comment thread packages/core/src/utils/untrustedContextTracker.ts
Comment thread packages/core/src/utils/untrustedContextTracker.ts Outdated
Comment thread packages/core/src/tools/shell.test.ts Outdated
Comment thread packages/core/src/tools/shell.test.ts Outdated

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a 'Build File Protection' feature to enhance security. It introduces a utility to identify build configuration files (e.g., BUILD, package.json, Dockerfile) and updates the PolicyEngine to require explicit user confirmation (ASK_USER) when these files are modified. Additionally, it adds tracking for modified build files and untrusted command flags during a session, surfacing these as critical security warnings in the UI. The DiffRenderer and colorizeCode components were updated to support disabling truncation for these sensitive diffs, and extensive tests were added to verify the new security policies and warning mechanisms. I have no feedback to provide as there were no review comments.

Note: Security Review did not run due to the size of the PR.

Comment thread packages/core/src/policy/policy-engine.ts Fixed
Comment thread packages/core/src/utils/buildFileUtils.ts Fixed
@github-actions

Copy link
Copy Markdown

✅ 41 tests passed successfully on gemini-3-flash-preview.

🧠 Model Steering Guidance

This PR modifies files that affect the model's behavior (prompts, tools, or instructions).

  • ⚠️ Consider adding Evals: No behavioral evaluations (evals/*.eval.ts) were added or updated in this PR. Consider adding a test case to verify the new behavior and prevent regressions.

This is an automated guidance message triggered by steering logic signatures.

@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces robust security protections for build file modifications and untrusted command flags. It adds policy checks to downgrade build file modifications to user confirmation, tracks modified build files and untrusted flags across the session, and updates the CLI to display critical security warnings and suppress persistent approvals when these risks are detected. Additionally, it disables truncation in the diff renderer for build files to ensure complete visibility. A critical security issue was identified in ShellToolInvocation.getConfirmationDetails where early returns for additional permissions (sandbox expansion) can bypass the newly introduced security checks for untrusted flags and modified build files.

Note: Security Review did not run due to the size of the PR.

Comment thread packages/core/src/tools/shell.ts Outdated
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a robust "Build File Protection" mechanism and untrusted command flag detection to enhance security. It ensures that modifications to build configuration files (such as Bazel BUILD, package.json, Makefile, etc.) or execution of shell commands containing untrusted flags require explicit user confirmation and suppress persistent approvals. Additionally, build file diffs are displayed without truncation in the UI. Feedback on the changes points out a potential security bypass on POSIX platforms due to a mismatch in backslash normalization between shell commands and untrusted texts, and suggests normalizing backslashes to forward slashes before performing substring checks.

Note: Security Review did not run due to the size of the PR.

Comment thread packages/core/src/utils/untrustedContextTracker.ts
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces robust security features to protect against unauthorized build file modifications and untrusted command flag execution. It adds utilities to identify build configuration files, updates the PolicyEngine to downgrade modifications of these files to require explicit user confirmation, and tracks modified build files to warn users before executing subsequent build or test commands. Additionally, it detects and warns against executing shell commands containing flags sourced from untrusted conversation history. Feedback on these changes suggests enhancing the untrusted token indexing by splitting on slashes to capture individual path segments and filenames, and improving security warning clarity by capturing both the flag and its untrusted argument when a match is detected.

Comment thread packages/core/src/utils/untrustedContextTracker.ts
Comment thread packages/core/src/utils/untrustedContextTracker.ts
@villahernandez-coder

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements security enhancements by introducing Build File Protection and Untrusted Flag Detection. It adds utilities to identify build files and track modifications during a session, updates the PolicyEngine to require user confirmation for build file edits and commands using untrusted flags, and improves the UI to display relevant security warnings. Additionally, it introduces a disableTruncation feature for build file diffs to ensure full visibility of changes. I have no feedback to provide.

This branch was successfully deployed

1 active deployment
eval-gate — 9f9e6277 Deployed Sep 10, 2026 by villahernandez-coder via Evaluate Steering & Regressions #1951
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xl An extra large PR status/need-issue Pull requests that need to have an associated issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants