HOSHODocs / Git Bot

Products

Git Bot

A GitHub Action that reviews every PR touching a prompt file.

What it is

A GitHub Action that reviews every pull request touching a prompt file and posts one comment: a verdict, what got better and worse, and suggested fixes with before/after snippets. It runs in your CI on GitHub's own token and comments as github-actions[bot] - no Hosho GitHub App, no new permissions granted to anyone. On later pushes the same comment is updated, never duplicated.

Install - three steps

  • Mint your pfr_... key at /setup/git and add it as a repository secret named HOSHO_API_KEY (Settings → Secrets and variables → Actions → New repository secret).
  • Create .github/workflows/hosho-prompt-review.yml with the workflow below.
  • Commit a .github/hosho/models.md so model-specific checks know which model each prompt runs on - without it they are guessed or skipped.
name: Hosho Prompt Review
run-name: >-
  Hosho Prompt Review -
  ${{ github.event_name == 'pull_request'
      && format('PR #{0}', github.event.pull_request.number)
      || github.event_name == 'issue_comment'
      && format('PR #{0} (slash cmd)', github.event.issue.number)
      || inputs.prompt_file }}

on:
  pull_request:
    paths:
      - '**/*system-prompt*.md'   # Adjust to match your prompt file naming pattern
      # To match multiple patterns, add more lines:
      # - '**/*user-prompt*.md'
  issue_comment:
    types: [created]              # Enables /hosho-review and /hosho-improve slash commands
  workflow_dispatch:
    inputs:
      prompt_file:
        description: "Path to prompt file to review"
        required: true

concurrency:
  group: hosho-review-${{ github.event.pull_request.number || github.event.issue.number || github.run_id }}
  cancel-in-progress: true

permissions:
  contents: read
  pull-requests: write
  issues: write       # Required - GitHub's PR comment API uses the issues endpoint
  actions: write      # Optional - enables showing prompt names in the run list

jobs:
  review:
    runs-on: ubuntu-latest
    if: >-
      github.event_name == 'workflow_dispatch'
      || (github.event_name == 'pull_request' && github.event.pull_request.draft == false)
      || (github.event_name == 'issue_comment'
          && github.event.issue.pull_request != null
          && (contains(github.event.comment.body, '/hosho-review')
              || contains(github.event.comment.body, '/hosho-improve')))
    steps:
      # Slash commands need to look up the PR branch before checkout
      - name: Get PR branch (slash command only)
        if: github.event_name == 'issue_comment'
        id: pr_details
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          PR_NUMBER=${{ github.event.issue.number }}
          PR_BRANCH=$(gh pr view $PR_NUMBER --repo ${{ github.repository }} --json headRefName -q '.headRefName')
          if [ -z "$PR_BRANCH" ]; then
            echo "::error::Could not find PR branch for #$PR_NUMBER"
            exit 1
          fi
          echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT
          echo "pr_branch=$PR_BRANCH" >> $GITHUB_OUTPUT

      - uses: actions/checkout@v4
        if: github.event_name != 'issue_comment'
        with:
          fetch-depth: 0   # Required - the action needs git history to compare versions

      - uses: actions/checkout@v4
        if: github.event_name == 'issue_comment'
        with:
          fetch-depth: 0
          ref: ${{ steps.pr_details.outputs.pr_branch }}

      - uses: HOSHO-AI/Hosho-prompt-optimization-public@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}   # Provided automatically by GitHub - do not create this secret
        with:
          api_key: ${{ secrets.HOSHO_API_KEY }}
          file_pattern: '**/*system-prompt*.md'
          prompt_file: ${{ github.event.inputs.prompt_file || '' }}
          pr_number: ${{ steps.pr_details.outputs.pr_number || '' }}
          models_config: .github/hosho/models.md   # which model each prompt runs on - see step 3

Three lines in there earn their keep: fetch-depth: 0 (the Action compares versions, so it needs history), the permissions block (issues: write is required - GitHub's PR comment API is the issues endpoint), and GITHUB_TOKEN (provided automatically by GitHub - do not create that secret yourself).

models.md - pin the model per prompt

# Models - which model each prompt runs on

| Prompt path (glob)   | Model            |
| -------------------- | ---------------- |
| `agents/router/**`   | openai/reasoning |
| `agents/**`          | claude           |
| `**/*prompt*.md`     | gemini           |

Families: claude · openai · gemini · deepseek · qwen · kimi · glm. Class is reasoning or standard (default standard). The most specific glob wins; a path you do not list falls back to Hosho's own detection, and a typo never fails a review. CI only - in the editor and the MCP you pick the model when you review.

Three ways to trigger it

  • On the PR - every pull request whose diff touches a matching prompt file. Draft PRs are skipped until marked ready.
  • Slash commands - comment /hosho-review to re-run the review, or /hosho-improve for the full evaluation with detailed scoring and improvement suggestions beyond the PR.
  • On demand - workflow_dispatch reviews one prompt file by hand; the result goes to the Job Summary, and it draws the review-run pool rather than the PR pool.

Configuration - every input

InputWhat it does
api_keyRequired. Your pfr_ key, from the HOSHO_API_KEY secret.
file_patternWhich files are prompts - minimatch globs, comma-separated for several. Keep it in sync with the workflow's paths: filter; the two select together.
prompt_pathAlternative to file_pattern: a directory prefix.
prompt_fileOn-demand mode: the one file to review.
models_configPath to your models.md (see above).
system_overviewPath to a markdown file describing the system this prompt lives in - context the reviewer reads.
custom_principlesPath to your team's own prompt-writing principles; violations are flagged alongside the standard findings.
skills_dirDirectory of skill files to bundle with each prompt, so a prompt is reviewed with the skills it actually runs with.
bundle_siblingsBundle sibling prompt/addendum files from the same directory into the review context. Default false.
assembly_configPath to .github/hosho/assembly.yml - declare shared references to inject when a prompt mentions them (inject_when_referenced) and references a prompt must contain (require_reference, with severity critical or suggestion).
dedupeDefault true. Skips re-reviewing files whose content has not changed since the last review on the PR - see below. Turning it off re-reviews everything on every push.
timeoutSeconds per file, default 600. Evaluations take 60-90 seconds per file.
pr_numberSet by the slash-command trigger; leave the template's expression as is.
api_urlOverride the review endpoint - for enterprise setups.

There is deliberately no exclude list - selection is positive-only, via file_pattern / prompt_path and the workflow's paths: filter. Deleted files are skipped; brand-new files are scored but get no better/worse analysis (there is no before side).

Dedupe - why your allowance is safe on busy PRs

GitHub re-fires the workflow on every push to a PR, and evaluates the paths: filter against the PR's whole diff - so once a PR touches one prompt file, every later push would re-review every prompt file the PR ever touched. The Action therefore remembers what it reviewed: files whose content is unchanged are skipped and their previous verdict carried forward, and a push that changes nothing leaves the comment untouched entirely. Billing is deduped harder still: one PR is one PR review per month, however many files it touches and however many pushes it takes.

When things fail

The check never turns red for a billing condition.

SituationWhat you see
No prompt files matchedThe job exits early, green. No comment.
A file's review failedThe comment covers the successes; the failure is a warning in the Actions log, and the file is retried on the next push.
Every file's review failedThe check goes red with the error - the one situation that fails the job.
Monthly PR allowance reachedA banner at the top of the comment names the files not reviewed this run; they are reviewed automatically on the next push once the allowance resets or the plan is upgraded. The check stays green.
The comment outgrows GitHub's size limitWhole file sections are dropped from the tail with a note; the full report is always in the Job Summary in the Actions tab.