Skip to content

framework·AI

AI Use Case Selection Framework

Hallucination Isn't AI's Biggest Problem. Uncertainty Is

·2 min read

How do you decide when you should use LLMs? LLMs hallucinate, and the challenge is not hallucination, but knowing when it hallucinates. The flowchart provides a framework for identifying whether a project is suitable for AI.

flowchart TD
    A["Can the task be solved deterministically?"]
    B["Do Not Use AI
Use software, rules engines"] C["Is the business value high?"] D["Too Much Effort to Verify
Avoid AI"] E["Can outputs be verified cheaply and reliably?"] F["Fully Automatable Tasks
Automate Aggressively"] G["Useful but Needs Review
AI drafts, humans decide"] H["Idea Generation & Exploration"] A -- Yes --> B A -- No --> C C -- No --> D C -- Yes --> E E -- Yes --> F E -- No --> G H -.-> E

Verification Economics Matrix

Hallucination RiskVerification CostValueExample Use CasesStrategic Action
LowLowHighEmail classification, chatbot FAQs, GitHub Copilot, SQL query writing, code debugging, receipt processingAutomate aggressively. Design for scale with minimal human intervention.
LowLowLowMeeting notes, email summaries, reply suggestions, social media captions, message templatesAutomate opportunistically. Deploy selectively for incremental gains.
LowHighHighLegal contract review, medical record summarization, resume screening, compliance document analysisReduce verification cost before scaling. Use structured retrieval and strong guardrails.
LowHighLowAuto-generated status reports requiring full review, heavily-edited presentations and marketing contentAvoid. Verification overhead destroys productivity gains.
HighLowHighProduct brainstorming, market research, competitive analysis, strategic planning workshopsUse for ideation and discovery.
HighLowLowCreative writing experiments, marketing tagline testing, design mockups, brand concept explorationSafe for exploration.
HighHighHighLegal brief writing, clinical decision support, compliance reporting, financial analysis, automated trading, database changesRestrict to assistive roles only.
HighHighLowUnsupervised customer commitments, auto-posting to social media, heavily-reviewed customer service emailsAvoid entirely.
Near-Zero Required–HighPayment processing, price calculations, user authentication, inventory trackingDo not use LLMs. Prefer deterministic software, rules engines, or RPA.
Near-Zero RequiredLowModerateDatabase backups, scheduled reporting, batch email sends, file synchronizationTyped pipelines outperform probabilistic systems here. AI adds unnecessary risk.

Related

Choosing the Right LLM: An Economic Perspective

The best model out there is not always required

analysis·AI·3 min·

Search