The result was, well, competent. The structure was right, the language was clear, and if I hadn't known better, it would have read as a real report. The fact was that it was a report about the data I'd given it, not about the project. When I forgot to include the Slack thread about the blocked API integration, the report didn't mention it; when the ticket data I pasted was two days old, the report reflected a two-day-old project. When I asked a follow-up question about timeline risk, the answer was generic because the model had no idea what our historical velocity looked like or what we'd committed to the client.
And then there was the deeper issue: none of this could run automatically. Every Friday, I still had to sit down, gather the data, construct the prompt, paste everything in, review the output, and fill in what the model couldn't know.
The report was still urgent, still manual, and still mine to figure out.
Why general-purpose LLMs are stateless, and why that matters for PMs
To understand why Claude and ChatGPT hit a wall with this kind of task, it helps to understand one fundamental thing about how they work — without getting into anything technical.
General-purpose LLMs have come a long way in continuity. ChatGPT Projects, Claude Projects, and similar features let you attach files, set standing instructions, and maintain conversation history across sessions. That's genuinely useful, and it's worth acknowledging.
But it didn't solve the problem I was running into. What persists is the context you've explicitly provided: the documents you've uploaded, the instructions you've written, and the history from previous conversations in that project. What they don't do is pull live data from your Jira board, notice what changed since last week's sprint, or run a report at 9 am Monday without you initiating the interaction. The gap isn't memory — it's the live connection to the systems where your project actually lives, and the ability to act on a schedule without a human in the loop.
This is what's meant by being stateless at the operational level, and it's a deliberate design choice that makes these models safe, privacy-preserving, and usable by millions of people for completely different purposes. A model that autonomously connects to every user's live systems would be a security nightmare.
But that statelessness is a significant limitation for project management work, because project management is fundamentally stateful. Every status update exists in relation to a previous one; every risk flag is meaningful only in the context of the current sprint, the historical velocity, the client's expectations, and what was agreed upon at the last review. Without that live continuity, an AI can write a report, without telling you what's actually happening in your project right now.
That distinction matters more than it might seem: a general-purpose LLM with a project attached knows what you told it. A project-aware agent knows what's happening.
When I built my Claude prompt, I was manually compensating for all of these limitations: I was the scheduler, the data pipeline, and the memory layer. The model was just helping me format the synthesis once I'd done all the work. At some point, I stopped compensating and started writing down what I actually needed, and turned it into a list.
So here's the list of what I actually needed to finally automate reporting:
- Access to live project data, not past snapshots. A useful reporting tool needs to see what's actually happening in Jira, GitHub, and Slack right now and not what I remembered to copy across an hour ago. Outdated inputs produce outdated conclusions, and outdated conclusions erode client trust.
- Memory of what came before. Last week's sprint matters for understanding this week's. A report that doesn't know whether we closed 80% of our planned scope last sprint has no basis for assessing whether this sprint's velocity is normal, concerning, or worth flagging. Historical context isn't a nice-to-have; it's what makes a status update meaningful rather than just descriptive.
- Knowledge of the project's specific context. Our project has specific milestone commitments, client sensitivities, team conventions, and risk thresholds. A reporting tool that doesn't know these produces generic output. Generic output requires a human to translate it into something truly useful, which defeats the purpose.
- The ability to run on a schedule without me. The whole point of automating a weekly report is that it runs weekly without my involvement. A tool that requires me to sit down every Friday and initiate the process hasn't automated anything; it's just changed the shape of the manual work.
- Output that goes somewhere. A report that exists only in a chat window is a draft. The output needs to reach the people who need it: sent to a client via email, posted to a team Slack channel, or delivered as a formatted document. Delivery is part of the task.
General-purpose LLMs meet the first requirement partially (with some manual effort) and none of the remaining four natively. That's a description of what they were designed to do. Matching the tool to the requirement is the actual efficiency gain.