🌏 閱讀中文版本
Let’s start with a representative situation: a customer-support manager opens a dashboard and sees that, after introducing generative AI, the team’s average response length and ticket throughput have both grown by double digits. The numbers feel reassuring. They match our intuition: the tool made people faster, and output increased.
But when we step back and look at the business outcomes behind those words, the picture often becomes more complex. If employees generate long responses without a clear focus to raise output, or engineers abandon a cleaner refactoring path to add lines of code, that “productivity” can become review work later.
The central argument of this article is straightforward: output metrics built around word count, ticket count, or lines of code (LOC) can become distorted in the AI era, and may even become an invisible ceiling on team effectiveness. Management needs to gradually move from measuring activity to measuring outcomes. The tradeoff is a higher cost of judgment and more uncertainty.
The Appeal and Blind Spots of Output Metrics
It is easy to understand why managers rely on word count, ticket count, or lines of code: the data is easy to collect, highly standardized, and convenient for comparing teams. Department A handles 500 tickets. Department B handles 400. A appears to be working harder. Early in a technology rollout, changes in output can also offer clues about the exploration phase. A sharp decline may point to adoption friction or a workflow still being adjusted.
These metrics do have value. They describe how busy a team is and the intensity of its activity. The difficulty is that people often mistake description for diagnosis.
AI can expand content quickly: turning a short instruction into a long document, or expanding a piece of logic into dozens of lines of code. If output volume is the evaluation criterion, more, longer, and more complex content becomes the rational choice. That is not necessarily a shortcoming in attitude or professional capability. It is the result of the incentive structure.
The key premise is this: easy-to-measure output metrics can align with business value only when quality thresholds are included in the measurement. If AI can generate large amounts of content quickly, but there is no quality threshold, clear review ownership, or testable outcome metric, output volume may move in the opposite direction from final value. The more generated, the greater the burden of later verification, correction, and integration.
What the Data Says: The 14% and Its Distribution
A common claim is that “AI doubles productivity.” The data is usually worth examining more closely than the slogan.
According to the 2023 NBER working paper Generative AI at Work (NBER Working Paper 31161), a study of outsourced customer-support centers found that employees using a generative AI tool increased productivity by an average of 14%.
The study measured cases resolved per hour. That is a proxy for completed work, and it comes closer than word count to whether service was completed. But whether it represents an outcome depends on whether “resolved” meets an established quality standard. It should be read alongside quality outcomes and reopening rates. If tickets are merely closed faster but lead to more reopenings, escalations, or customer friction, throughput alone cannot explain service value.
The study also found that the gains were not evenly distributed. Less-experienced employees benefited more. They used AI as a knowledge anchor and structural framework, quickly narrowing the gap with mid-level employees. Gains for senior employees were relatively smaller, with no significant difference in some complex situations. A reasonable inference is that AI’s marginal contribution for them may lie mainly in automating routine work rather than creating a breakthrough in the core demands of advanced engineering. The specific mechanisms still need more empirical evidence.
So if management evaluates only by output volume, it may see no significant increase in senior employees’ numbers while overlooking the more complex architecture work that requires their judgment. The change in productivity may be a shift in the capability structure, not linear growth for everyone.
Redefining Productivity: A Decision Matrix From Output to Outcomes
Word count and ticket count are not precise enough on their own. That does not mean quantification must be abandoned. Different work requires different criteria.
| Type of work | Not suitable as the sole core metric | Recommended metrics | Why |
|---|---|---|---|
| Software development | Lines of code (LOC) | Delivery cycle time, escaped-defect rate (with severity weighting and an observation window defined) | AI can generate large volumes of code, but maintainability and system stability are what matter. |
| Content creation | Word count, article count | Reading completion rate, conversion rate, editorial review time | Low-value content can be produced without limit, while the distribution efficiency of useful information remains finite. |
| Customer support | Tickets handled | First-contact resolution (FCR), customer satisfaction (CSAT) | Fast responses that do not address the core need only create more follow-up tickets and customer friction. |
| Strategic analysis | Report length | Predefined post-decision outcomes, speed of hypothesis validation | No matter how substantial the report, its value is limited if it does not influence action or accelerate a decision. |
Productivity is not simply “doing things faster.” It is achieving the same outcome with fewer resources, or achieving a better outcome with the same resources. When AI helps generate tests, documentation, or refactoring, total word count may fall while system stability and maintainability rise.
Output volume is therefore not inherently a risky proxy metric. It becomes especially easy to distort when quality thresholds are absent. During an exploration phase, it can still provide clues about workflow changes. It is simply not suitable for carrying the full judgment of value on its own.
Addressing the Counterargument: The Practical Boundaries of Outcome Orientation
The strongest counterargument is: “If we do not use word count or ticket count, how can a team evaluate performance objectively? Outcomes are too subjective.”
This is a practical concern. Abandoning quantification entirely can make evaluation vague and may even invite disputes shaped by personal relationships. When managers do not have enough time to review every deliverable in depth, output volume, although incomplete, may still be a second-best but workable choice.
Outcome orientation is not feasible in every situation. When business goals are unclear, quality standards have not been established, or team scale makes management costs too high, word count and ticket count can still serve as transitional metrics. They provide basic order and comparability. The tradeoff is that they lower the cost of measurement while increasing the risk of distorted behavior. A more robust path is to improve gradually toward outcome orientation as quality standards and workflows become clearer.
The Manager’s New Challenge: How Do You Evaluate Fairly?
The real management question is not “how do we confirm who is working seriously?” It is how to judge whether a team is turning its effort into verifiable value. Quantitative metrics provide certainty and a sense of control, but excessive reliance on easily measured output may come at the expense of activities that are harder to quantify yet important.
This principle has boundaries too. In healthcare, financial compliance, or high-risk infrastructure, the cost of an error is extremely high. Intensive process monitoring and standardized operations remain necessary. Trust cannot replace rigorous verification. Monitoring strategies need to adjust to the level of risk.
Rework rate can serve as a reverse indicator. First-draft acceptance rate or the number of revision rounds usually comes closer to real efficiency than word count. But if reviewers define quality inconsistently, rework rate can become a projection of preference. It should therefore be used only alongside clear quality definitions and final outcome metrics. Teams can first create a checklist for what an acceptable first draft must include and which standards it must meet, then review whether rework comes from a gap in generated quality or a difference in human judgment.
For analytical work, AI’s value often lies in shortening the time from a core need to an insight. Measuring the time to first present a viable option usually comes closer to the purpose than measuring the option’s length. During the early stage of adoption, teams can distinguish between exploration and execution: establish a baseline, adjust the workflow, then evaluate the tool’s effect. This avoids applying old metrics directly to a new process.
Finding the Balance: Output and Outcomes in Motion
AI lowers the marginal cost of generating a first draft. It does not remove the work of review, fact-checking, or system integration. When content volume rises sharply, these downstream costs may even increase.
AI therefore amplifies the incentive structure already in place. When quality thresholds, review ownership, and outcome metrics are absent, low-quality output can expand more easily. When criteria reward efficiency in responding to the core need, concision and accuracy are more likely to become rational choices.
This is not a call to abandon numbers completely. It is an acknowledgment that metric selection reflects risk preference: between measurement certainty and behavioral distortion, there is no context-free standard answer. Output volume can be a clue during exploration and part of process control. But without quality and outcomes, it is not suitable as the conclusion of productivity. Once that boundary is clear, teams can make their own tradeoffs according to their work and risk profile.
Sources
- NBER: Generative AI at Work — Empirical research on generative AI and customer-support productivity