ACM Queue Publishes CAFE(S): A Framework for Improving AI Coding Agent Effectiveness

ACM Queue publishes new research establishing five dimensions of context quality.

TEAM Anywhere/SAN FRANCISCO--(BUSINESS WIRE)--Atlassian Corporation (NASDAQ: TEAM), a leading provider of AI-powered collaboration and team productivity software, today announced that researchers from DX (a subsidiary of Atlassian Corporation), Capital One, GitHub, the University of Victoria, and Google published CAFE(S), a diagnostic framework designed to evaluate the context provided to AI agents. Published in ACM Queue, the research establishes an industry standard for diagnosing, designing, and maintaining the information environments that power AI coding agents across the software development lifecycle.

As software engineering organizations scale investments in AI coding agents, task failures are often attributed to issues with model capabilities or harness orchestration. However, research demonstrates that even frontier models degrade when provided with ambiguous, incomplete, or stale context.

The CAFE(S) framework introduces a shared diagnostic vocabulary for platform teams, developer productivity leaders, and software engineers to evaluate assembled context across five dimensions:

1. Clarity: Can the agent interpret the request the way the person intended? Ambiguity is often invisible to the person writing the request. CAFE(S) helps teams ensure the agent understands the work the same way a teammate would.
2. Actionability: Does the agent have clear goals, useful boundaries, and a way to know when it is done? A request can be clear but still leave out the goal, say too little about the constraints, or give no way to tell that the work is finished.
3. Fidelity: Is the context true at the moment the agent reads it? Accuracy decays over time. Stale documentation, old architectural decisions, and conflicting sources lead agents down the wrong path.
4. Efficiency: Is the context scoped to the task, without unnecessary token load? More context is not always better. Passing an entire repository when an agent needs a single function inflates costs and degrades performance.
5. Security: Should the agent have this context at all? The "S" is kept at a remove because the first four dimensions determine whether context helps the task, whereas security asks if the context is fundamentally appropriate, compliant, and safe to access in the first place.

"AI does not create a new need for good knowledge management; it dramatically increases the cost of poor knowledge management. When agents receive poor context, they are forced to guess. That forces humans to spend their time compensating for avoidable mistakes, creating developer rework, token waste, and reliability risks. CAFE(S) treats context quality as a first-class engineering discipline that teams can deliberately design, evaluate, and maintain." – Brian Houck, Distinguished Scientist at DX and co-author of the paper

The research emphasizes that capable models alone are insufficient to ensure reliable task completion and stronger delivery outcomes. Teams that systematically strengthen the information environments feeding their agents will see better results. CAFE(S) becomes a quality scorecard for context, designed to sit on top of the existing stack.

What’s next?

This research defines a framework, not a measurement system. Future work will be needed to develop reliable ways of assessing these properties at scale, and to understand how improvements in context quality influence collaboration, software delivery, and organizational outcomes. But meaningful measurement requires a shared definition of what is being measured. Before teams can improve context quality, they need a vocabulary for discussing it.

Download the full research paper: https://getdx.com/report/the-cafes-framework/

About Atlassian

Atlassian unleashes the potential of every team. A recognized leader in software development, work management, and enterprise service management software, Atlassian enables enterprises to connect their business and technology teams with an AI-powered system of work that unlocks productivity at scale. Atlassian’s collaboration software powers over 85% of the Fortune 500 and 350,000+ customers worldwide - including NASA, Rivian, Deutsche Bank, United Airlines, and Bosch - who rely on our solutions to drive work forward.

About DX

DX is an engineering intelligence platform focused on helping organizations understand and improve the systems, workflows, and information environments that shape software delivery. By translating research-backed methodologies into practical organizational metrics, DX helps engineering leaders reduce friction, optimize developer experience, and drive measurable velocity. To learn more, visit getdx.com.