Products
Git Bot
A GitHub Action that reviews every PR touching a prompt file.
What it is
A GitHub Action that reviews every pull request touching a prompt file and posts one comment: a verdict, what got better and worse, and suggested fixes with before/after snippets. It runs in your CI on GitHub's own token and comments as github-actions[bot] - no Hosho GitHub App, no new permissions granted to anyone. On later pushes the same comment is updated, never duplicated.
Install - three steps
- Mint your
pfr_...key at /setup/git and add it as a repository secret namedHOSHO_API_KEY(Settings → Secrets and variables → Actions → New repository secret). - Create
.github/workflows/hosho-prompt-review.ymlwith the workflow below. - Commit a
.github/hosho/models.mdso model-specific checks know which model each prompt runs on - without it they are guessed or skipped.
name: Hosho Prompt Review
run-name: >-
Hosho Prompt Review -
${{ github.event_name == 'pull_request'
&& format('PR #{0}', github.event.pull_request.number)
|| github.event_name == 'issue_comment'
&& format('PR #{0} (slash cmd)', github.event.issue.number)
|| inputs.prompt_file }}
on:
pull_request:
paths:
- '**/*system-prompt*.md' # Adjust to match your prompt file naming pattern
# To match multiple patterns, add more lines:
# - '**/*user-prompt*.md'
issue_comment:
types: [created] # Enables /hosho-review and /hosho-improve slash commands
workflow_dispatch:
inputs:
prompt_file:
description: "Path to prompt file to review"
required: true
concurrency:
group: hosho-review-${{ github.event.pull_request.number || github.event.issue.number || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
issues: write # Required - GitHub's PR comment API uses the issues endpoint
actions: write # Optional - enables showing prompt names in the run list
jobs:
review:
runs-on: ubuntu-latest
if: >-
github.event_name == 'workflow_dispatch'
|| (github.event_name == 'pull_request' && github.event.pull_request.draft == false)
|| (github.event_name == 'issue_comment'
&& github.event.issue.pull_request != null
&& (contains(github.event.comment.body, '/hosho-review')
|| contains(github.event.comment.body, '/hosho-improve')))
steps:
# Slash commands need to look up the PR branch before checkout
- name: Get PR branch (slash command only)
if: github.event_name == 'issue_comment'
id: pr_details
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
PR_NUMBER=${{ github.event.issue.number }}
PR_BRANCH=$(gh pr view $PR_NUMBER --repo ${{ github.repository }} --json headRefName -q '.headRefName')
if [ -z "$PR_BRANCH" ]; then
echo "::error::Could not find PR branch for #$PR_NUMBER"
exit 1
fi
echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT
echo "pr_branch=$PR_BRANCH" >> $GITHUB_OUTPUT
- uses: actions/checkout@v4
if: github.event_name != 'issue_comment'
with:
fetch-depth: 0 # Required - the action needs git history to compare versions
- uses: actions/checkout@v4
if: github.event_name == 'issue_comment'
with:
fetch-depth: 0
ref: ${{ steps.pr_details.outputs.pr_branch }}
- uses: HOSHO-AI/Hosho-prompt-optimization-public@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Provided automatically by GitHub - do not create this secret
with:
api_key: ${{ secrets.HOSHO_API_KEY }}
file_pattern: '**/*system-prompt*.md'
prompt_file: ${{ github.event.inputs.prompt_file || '' }}
pr_number: ${{ steps.pr_details.outputs.pr_number || '' }}
models_config: .github/hosho/models.md # which model each prompt runs on - see step 3Three lines in there earn their keep: fetch-depth: 0 (the Action compares versions, so it needs history), the permissions block (issues: write is required - GitHub's PR comment API is the issues endpoint), and GITHUB_TOKEN (provided automatically by GitHub - do not create that secret yourself).
models.md - pin the model per prompt
# Models - which model each prompt runs on | Prompt path (glob) | Model | | -------------------- | ---------------- | | `agents/router/**` | openai/reasoning | | `agents/**` | claude | | `**/*prompt*.md` | gemini |
Families: claude · openai · gemini · deepseek · qwen · kimi · glm. Class is reasoning or standard (default standard). The most specific glob wins; a path you do not list falls back to Hosho's own detection, and a typo never fails a review. CI only - in the editor and the MCP you pick the model when you review.
Three ways to trigger it
- On the PR - every pull request whose diff touches a matching prompt file. Draft PRs are skipped until marked ready.
- Slash commands - comment
/hosho-reviewto re-run the review, or/hosho-improvefor the full evaluation with detailed scoring and improvement suggestions beyond the PR. - On demand -
workflow_dispatchreviews one prompt file by hand; the result goes to the Job Summary, and it draws the review-run pool rather than the PR pool.
Configuration - every input
| Input | What it does |
|---|---|
api_key | Required. Your pfr_ key, from the HOSHO_API_KEY secret. |
file_pattern | Which files are prompts - minimatch globs, comma-separated for several. Keep it in sync with the workflow's paths: filter; the two select together. |
prompt_path | Alternative to file_pattern: a directory prefix. |
prompt_file | On-demand mode: the one file to review. |
models_config | Path to your models.md (see above). |
system_overview | Path to a markdown file describing the system this prompt lives in - context the reviewer reads. |
custom_principles | Path to your team's own prompt-writing principles; violations are flagged alongside the standard findings. |
skills_dir | Directory of skill files to bundle with each prompt, so a prompt is reviewed with the skills it actually runs with. |
bundle_siblings | Bundle sibling prompt/addendum files from the same directory into the review context. Default false. |
assembly_config | Path to .github/hosho/assembly.yml - declare shared references to inject when a prompt mentions them (inject_when_referenced) and references a prompt must contain (require_reference, with severity critical or suggestion). |
dedupe | Default true. Skips re-reviewing files whose content has not changed since the last review on the PR - see below. Turning it off re-reviews everything on every push. |
timeout | Seconds per file, default 600. Evaluations take 60-90 seconds per file. |
pr_number | Set by the slash-command trigger; leave the template's expression as is. |
api_url | Override the review endpoint - for enterprise setups. |
There is deliberately no exclude list - selection is positive-only, via file_pattern / prompt_path and the workflow's paths: filter. Deleted files are skipped; brand-new files are scored but get no better/worse analysis (there is no before side).
Dedupe - why your allowance is safe on busy PRs
GitHub re-fires the workflow on every push to a PR, and evaluates the paths: filter against the PR's whole diff - so once a PR touches one prompt file, every later push would re-review every prompt file the PR ever touched. The Action therefore remembers what it reviewed: files whose content is unchanged are skipped and their previous verdict carried forward, and a push that changes nothing leaves the comment untouched entirely. Billing is deduped harder still: one PR is one PR review per month, however many files it touches and however many pushes it takes.
When things fail
The check never turns red for a billing condition.
| Situation | What you see |
|---|---|
| No prompt files matched | The job exits early, green. No comment. |
| A file's review failed | The comment covers the successes; the failure is a warning in the Actions log, and the file is retried on the next push. |
| Every file's review failed | The check goes red with the error - the one situation that fails the job. |
| Monthly PR allowance reached | A banner at the top of the comment names the files not reviewed this run; they are reviewed automatically on the next push once the allowance resets or the plan is upgraded. The check stays green. |
| The comment outgrows GitHub's size limit | Whole file sections are dropped from the tail with a note; the full report is always in the Job Summary in the Actions tab. |