Agentic AI — Week 1, Day 1
Day 1 Exercise — Deciding What AI Should Not Do
What We're Going to Cover
By the end of this exercise you will have:
- Placed real product capabilities on the AI Capability Ladder and defended the hard calls
- Declined at least three proposed features — on architectural grounds, not vague discomfort
- Tested your reasoning against an LLM arguing the opposite side
- Sketched the one capability that genuinely warrants AI, with its human oversight pattern and risk boundary
Overview
- Time: ~40 minutes required, plus optional paths
- Difficulty: Moderate — no technical setup, but the judgment is the hard part
- Format: Individual
- Setup: An LLM of your choice (Claude, ChatGPT, Gemini, whichever you use) and somewhere to write. That's it.
Learning Objective
Apply the Use Case Quality Filter and the "When NOT to Use AI" criteria to a realistic product request, and articulate a refusal a customer would accept.
Relevant Day 1 Material
AI Capability Ladder (Slide 8), Enterprise AI Risks (Slide 14), AI Product Thinking (Slides 17–18), Use Case Quality Filter (Slide 20), When NOT to Use AI (Slide 21), Human Oversight Patterns (Slide 23), Customer Advisory Framework (Slide 34).
Agentic AI — Week 1, Day 1
Context Introduction
FitPath is a fitness app with roughly 400,000 users. Their existing product logs workouts, tracks weight, and offers a library of pre-recorded exercise videos. Users skew 16–45. About 12% of the user base is under 18.
FitPath's leadership has come to you with a sentence you will hear many times in your career:
"We want to build an AI personal trainer."
Pressed for detail, the product team supplies a feature list. They want all of it, and they'd like to launch in Q4.
The Proposed Feature List
| # | Feature | As described by the product team |
|---|---|---|
| 1 | Rep counting | Uses the phone camera to count repetitions during a set and tell the user when they've hit their target. |
| 2 | Weekly workout plan generation | Produces a personalised week of workouts based on the user's stated goal, available equipment, and logged training history. |
| 3 | Calorie target setting | Asks the user their goal weight and target date, then sets a daily calorie limit to get them there. |
| 4 | Form correction | Watches the user through the camera and tells them what they're doing wrong — "your back is rounding," "knees past toes." |
| 5 | Injury rehab guidance | User describes a niggle or injury; the app suggests modified exercises and a return-to-training timeline. |
| 6 | "Shred Mode" | An aggressive 8-week programme combining a steep calorie deficit with high training volume. Marketed to users who want visible results fast. |
You are the Solution Architect. Your job is not to build this. Your job is to work out what should be built, what shouldn't, and what needs to be reshaped before anyone writes a line of code.
Agentic AI — Week 1, Day 1
Step 1 — Place the Capabilities
For each of the six features, place it on the AI Capability Ladder:
- Predictive AI — classification, scoring, forecasting
- Generative AI — text, images, code, summaries
- Retrieval-Augmented AI — grounded in enterprise knowledge
- Tool-Using AI — calls APIs, queries databases, triggers workflows
- Agentic AI — interprets goals, plans steps, adapts
Some of these don't need AI at all. If that's your conclusion, mark it Level 0 — deterministic and move on. That's a legitimate answer and you should expect to use it more than once.
| # | Feature | Ladder level | One-line reason |
|---|---|---|---|
| 1 | Rep counting | ||
| 2 | Weekly workout plan generation | ||
| 3 | Calorie target setting | ||
| 4 | Form correction | ||
| 5 | Injury rehab guidance | ||
| 6 | "Shred Mode" |
The Two Hardest Calls
Pick the two you found hardest to place. For each, write one sentence on why it was ambiguous, and which way you'd defend it if a customer pushed back.
Hardest call #1
Feature
Why it was ambiguous:
How I'd defend my placement:
Hardest call #2
Feature
Why it was ambiguous:
How I'd defend my placement:
Note: Don't overthink the taxonomy. The placement itself is worth little — the argument about the ambiguous ones is the whole point.
Agentic AI — Week 1, Day 1
Step 2 — Decline Three
Run the six features through both filters.
Use Case Quality Filter
- Clear user and business outcome
- Repetitive work with variation
- Enough data or context
- Measurable quality criteria
- Acceptable error handling path
- Human oversight for risk
When NOT to Use AI
- Rules are simple and deterministic
- Errors create unacceptable harm
- Data is unavailable or untrustworthy
- Process is not understood
- Output cannot be evaluated
- No one owns the decision
Write Your Refusals
Choose at least three features to decline. For each, write two to three sentences covering:
- Which criterion it fails
- What you'd recommend instead — a different technology, a reshaped scope, or nothing at all
- What you'd actually say to the product team
Refusal 1
Feature
Criterion it fails
Recommend instead
What I'd say to the team
Refusal 2
Feature
Criterion it fails
Recommend instead
What I'd say to the team
Refusal 3
Feature
Criterion it fails
Recommend instead
What I'd say to the team
⚠️ Requirement
At least one of your refusals must be on harm grounds, not efficiency grounds.
An efficiency refusal says this doesn't need a model, a rules engine does it better and cheaper.
A harm refusal says the failure mode here is unacceptable, and no one on this team owns the consequence.
Both are valid architectural arguments. Only one of them is uncomfortable to make in a room full of people who want to ship. Practise it here.
Framing
Frame every refusal as architecture and accountability — who owns this decision, what happens when it's wrong, what's the escalation path. You are not being asked to make an ethical case. You are being asked to make a professional one.
Agentic AI — Week 1, Day 1
Step 3 — Adversarial Round
Open your LLM of choice. Paste the prompt below, filling in your own material.
Starter prompt
Do this for at least one of your refusals — ideally the harm-based one.
Then answer, in writing
Which counterargument was strongest?
Does your refusal survive it? If yes, what's your response? If no, what would you change?
Did the model produce anything that sounded convincing but was actually wrong?
That last question matters. A fluent argument is not a correct one, and a model will produce fluent arguments for anything you point it at. Noticing the gap between persuasive and right is most of what this step is for.
Note
Your classmates are using different models and will get opponents of varying quality. If yours rolls over immediately, push it harder — tell it the counterarguments were weak and ask for better ones.
Agentic AI — Week 1, Day 1
Step 4 — Shape the Survivor
One feature on that list is a genuine fit for generative AI: weekly workout plan generation.
Sketch three fields only.
AI contribution
What is the model actually doing? Summarising, recommending, generating, classifying, or executing?
Human role
Who reviews, approves, supervises, or audits — and at what point? Name the oversight pattern: human in the loop, on the loop, after the loop, or as fallback.
Risk boundary
What must never happen? Be specific enough that someone could write a test for it.
Keep this. It's a sketch, not a document — and it's the starting point for Day 2. Save it somewhere you'll find it again.
Agentic AI — Week 1, Day 1
Optional Paths
Path A — The First Meeting
You have the vague sentence — "we want to build an AI personal trainer" — and nothing else.
Write the six questions you would ask in the first customer meeting, using the Customer Advisory Framework:
- What outcome matters?
- Which workflow changes?
- What data and context are required?
- What can the AI do?
- What risks must be controlled?
- How will value and quality be measured?
Then mark the one question whose answer would most likely kill the project, and write what that answer would be.
| # | Your question |
|---|---|
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 | |
| 6 |
The project-killing answer
Question would kill it if the answer were:
Path B — Reshape a Refusal
Take one feature you declined and find the version of it you would build.
Injury rehab guidance is a good candidate: what scope, what oversight, what disclaimer, what escalation to a human — and at what point does it stop being the same feature at all?
Feature I'm reshaping
Reduced scope
Oversight pattern
Escalation path
What I gave up
Is it still the same feature?
Agentic AI — Week 1, Day 1
Success Criteria
If you found yourself wanting to build all six, go back to Step 2.
If you declined all six, you've over-corrected — Feature 2 is a real opportunity, and an architect who can't recognise one is as unhelpful as one who can't say no.