HOSHO
← Hosho Research

The messy problem of prompting

July 2026

The problem: tiny details, big swings

Small details in prompts change outcomes far more than most teams expect.

  • Changing nothing but the formatting of a prompt - separators, spacing, casing - swings accuracy by up to 76 points on some models1.
  • Requirements left unstated get met around a quarter less often than requirements written down, because the model has to guess what you wanted, and it guesses right less than half the time2.
  • Adding a single irrelevant sentence to a maths problem is enough to lower accuracy on problems the model could otherwise solve3.
  • And the defects hide: when instructions conflict, models notice around 87% of the time but almost never tell you - they quietly drop one rule and carry on4 - and wording that reads two ways gets interpreted inconsistently5.

One prompt doesn’t travel across models and providers.

  • What counts as a good prompt changes by provider. Anthropic wants XML tags, long documents at the top and the ask at the end10; OpenAI wants a Markdown skeleton with key instructions repeated around long context11; Gemini wants one delimiter scheme and has to be asked explicitly for detailed output12.
  • Even within one provider, reasoning and standard models need opposite prompts. Few-shot examples are textbook advice for standard models, yet DeepSeek reports they “consistently degrade” its reasoning model’s performance13.
  • So every model switch quietly invalidates parts of your prompt, and an agent running on general prompt folklore will confidently apply the wrong rules to the model you actually use.

Optimizing to lower-cost tokens amplifies every one of these issues.

  • Prompt optimization is what makes the cheaper model viable: structured prompting compensates for limited model capacity, and the smaller the model, the bigger the relative gain14.
  • Cheaper models are also far less forgiving: as instructions pile up they collapse rather than degrade - under heavy load, frontier models still follow about two-thirds of their rules while small, cheap models follow fewer than one in ten6.

Prompt editing is becoming an organizational issue.

  • Prompts are now edited across the whole org - engineering, product, business - by people with no way of knowing how a wording change lands on the target model, and those edits ship through pull requests where reviewers judge the code but nobody reviews the prompt.

Why your coding agent won’t fix it

  • It has one reflex: add. When something fails it appends a rule, but instruction-following decays as rules pile up - top models fall to roughly 68% adherence at high instruction counts, and earlier rules beat later ones6 - and agents almost never delete, even though typical prompts carry dead weight that could go without losing anything7.
  • It patches the incident, not the prompt. One bad output becomes a narrow “if X then Y” rule, and ten incidents later you have overlapping special cases and your first contradiction. Fixing a case with an example is riskier still, because examples that break your written rules pull the model toward the examples8.
  • It cannot review its own writing. Improving your own prompt is self-review without a criterion, which is measured not to work: without an external anchor, LLM self-correction tends to leave output unchanged or make it worse9. The agent that wrote the sentence knows what it meant, so it cannot see the second reading.

How Hosho solves your prompt problems

The latest prompting research, applied to your everyday prompts.

  • Hosho reviews every prompt against its proprietary research, triangulated published prompt-engineering studies, and each provider’s own best practices - so every check is evidence-backed rather than a model’s opinion of the day.
  • Every review identifies the gaps and the fixes: each finding points at the exact text, comes with a recommended edit, and rolls up to a score you can track across prompt versions.

From your coding agent, through review, back to validation.

  • Hosho runs as an MCP you access within your IDE, with a dedicated UI for heavier editing sessions - where your coding agent joins you, bringing the full context of your repo.
  • You get a clear view of every gap and its recommended improvement, and apply the ones you agree with.
  • Then you validate the change like any other: evals and A/B testing, old prompt against new on the same inputs, to confirm the score gain shows up in the outputs.

Hosho Git catches the edits that never went through Hosho.

  • Prompt changes land from across the business, so Hosho Git reviews the prompt diff on every pull request and flags whether the change improves or worsens the prompt - expert review on every edit, whoever made it.

The loop runs end to end inside the tools you already use: draft with your coding agent, review with Hosho, apply, merge, and validate with evals and A/B tests. Your agent stays in the loop the whole way as the hands - it just should not be the judge.

Sign in and put your hardest prompt through a review.

Sources

  1. 1.Sclar et al., Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design (FormatSpread), ICLR 2024. arXiv:2310.11324 ↑
  2. 2.What Prompts Don’t Say: Understanding and Managing Underspecified Prompts. arXiv:2505.13360 ↑
  3. 3.Shi et al., Large Language Models Can Be Easily Distracted by Irrelevant Context, ICML 2023. arXiv:2302.00093 ↑
  4. 4.ConInstruct: conflict detection and resolution under conflicting instructions. arXiv:2511.14342 ↑
  5. 5.Liu et al., We’re Afraid Language Models Aren’t Modeling Ambiguity (AmbiEnt), EMNLP 2023. arXiv:2304.14399 ↑
  6. 6.IFScale: How Many Instructions Can LLMs Follow at Once? arXiv:2507.11538 ↑
  7. 7.Jiang et al., LLMLingua: Compressing Prompts for Accelerated Inference, EMNLP 2023. arXiv:2310.05736 ↑
  8. 8.Do as I Say, Not as I Do: in-context demonstrations under pressure against written instructions. arXiv:2605.20382 ↑
  9. 9.Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet, ICLR 2024. arXiv:2310.01798 ↑
  10. 10.Anthropic prompt engineering documentation (XML tags; long-context tips), read July 2026. link ↑
  11. 11.OpenAI GPT-4.1 prompting guide and reasoning best practices, OpenAI developer docs, read July 2026. link ↑
  12. 12.Google Gemini prompt design strategies, read July 2026. link ↑
  13. 13.DeepSeek-R1 paper (Nature, 2025) and the official DeepSeek-R1 GitHub README usage recommendations, read July 2026. arXiv:2501.12948 ↑
  14. 14.PRIME: Policy-Reinforced Iterative Multi-agent Execution for Algorithmic Reasoning in Large Language Models. arXiv:2602.11170 ↑
So what matters in a prompt? →