The same week your team kicks off an agentic AI initiative, someone opens Jira and starts writing user stories.
The same week your team kicks off an agentic AI initiative, someone opens Jira and starts writing user stories. Nobody stops them. Nobody thinks to.
I have spent years implementing planning systems. I know the playbook cold — gather requirements, configure, test, train, go-live, hypercare, hope for the best. (That last step is unofficial. It is also the most important one.)
So when I decided to build an agentic AI layer on top of SAP IBP — on my own, over weekends, fuelled by curiosity — I assumed I knew how to start.
I did not.
User stories were the right upgrade from URDs (User Requirement documents). But they were designed for deterministic systems. Systems that do exactly what you tell them.
An agentic system does not execute your intent. It reasons toward a goal.
That needs a different input entirely. Not a story — a mandate. Goal. Constraints. Autonomy boundaries. Where the agent acts, and where it waits for a human before touching anything.
"As a demand planner, I want forecast risk identified, so that I can work on these products" tells an agent nothing. But "surface A-class SKUs where the baseline ignores a confirmed external signal, flag for planner review, never push a revision autonomously" — now we are getting somewhere.
The format does not exist in any methodology I have seen. We are building it in real time. (Either exciting or deeply concerning, depending on your risk appetite.)
The second thing that broke: testing
Traditional testing is a contract. Input X, output Y. Binary. Deterministic.
An agentic system does not honour that contract.
Real example. Agent receives a 12-week demand signal for an A-class SKU. Competitor confirms a product launch in week 8. Statistical baseline is flat — peacefully, dangerously flat.
A test asks: did it return the right number?
An eval asks: did it identify the competitive launch as a demand signal, adjust the baseline, and route it to the planner — rather than silently blessing a forecast that will embarrass someone in S&OP?
The answer might be a 6% revision. Or 9%. Or a recommendation to wait for week 6 sell-in data before touching the forecast at all. All three can be correct depending on context. A test cannot tell the difference. An eval can.
Requirements capture and system testing are not just SAP IBP implementation problems. They are the foundation of every implementation, everywhere.
What I thought were settled best practices in both turned out to be assumptions I had never actually tested, rather taken for granted.
User Stories are insufficient, and Testing? its is "EVALving" (I apologize for this terrible pun).
And that is either terrifying or interesting, depending on how much you like rebuilding things from scratch.
I am firmly in the second camp. Most days.