Microsoft researchers published a study on July 1 examining how tens of thousands of the company’s engineers adopted and used command-line AI coding agents during the early 2026 rollout of Anthropic’s Claude Code and GitHub’s Copilot CLI. The paper, authored by Emerson Murphy-Hill, Jenna Butler, and Alexandra Savelieva, provides the first large-scale look inside a major organization at who actually uses these tools, who sticks with them, and what happens to output.
The headline number: engineers who adopted either tool merged roughly 24% more pull requests than they would have otherwise. The researchers used merged pull requests as their proxy for output, a choice they acknowledge is imperfect. A merged PR is not the same as value delivered. But the lift persisted across the four-month observation window, suggesting the effect is not a novelty bounce.
The more interesting finding is about adoption patterns. First use spread primarily through social networks. Engineers tried Claude Code and Copilot CLI because they saw peers using them, not because of company-wide mandates or manager directives. The researchers found that retention correlated more strongly with an engineer’s baseline coding activity than with demographics like tenure, role, or team. Heavy coders kept using the tools. Light coders tried them and drifted away.
This matters for any organization considering a rollout at scale. Token spend for agentic coding tools at Microsoft’s size runs into millions of dollars annually, the paper notes. Misreading adoption or retention can make a rollout expensive without changing engineering velocity. The study suggests that visible peer use is central to strategy. A top-down push that does not create social proof will struggle to gain traction.
The study does not compare Claude Code against Copilot CLI directly. It treats the two tools as a category, CLI coding agents, and measures aggregate behavior. That is a limitation. The two products have different strengths and different pricing models. Anthropic charges per token for Claude Code, while GitHub bundles Copilot CLI into its existing Copilot subscription. A direct comparison would be useful but would require a different experimental design.
What the study captures well is the heterogeneity of adoption. Not everyone benefits equally. The 24% lift is an average across a population that includes engineers who barely used the tools and engineers who integrated them into daily workflows. The paper does not report the distribution of the lift, only the mean. That average likely masks a wide range of outcomes, from engineers who saw no benefit to engineers who saw a much larger gain.
The social-network finding aligns with what other research on developer tool adoption has shown. Engineers are skeptical of new tools. They trust peer recommendations over marketing or mandates. Microsoft’s rollout appears to have respected this dynamic, letting adoption grow organically rather than forcing it. The paper does not describe the specific rollout tactics, but the data suggests a light-touch approach worked.
For AI builders, the study reinforces a few lessons. First, agentic coding tools produce measurable output gains at scale, not just in controlled experiments. The 24% lift is real, even if it is an average. Second, adoption is not automatic. Engineers need to see peers using a tool before they try it themselves. Third, retention is self-selecting. Engineers who already write a lot of code find the most value. Engineers who write less code do not stick around.
The study also raises questions the authors do not fully answer. Does the 24% lift hold for code quality, not just quantity? Merged PRs can be buggy or poorly designed. The paper does not measure defect rates, code review outcomes, or downstream maintenance costs. A 24% increase in output that comes with a 10% increase in defects is a different story than a 24% increase with no quality degradation.
There is also the question of cost. Microsoft likely negotiated enterprise pricing, so the per-engineer token cost is not public. But the paper flags that token spend can run into millions of dollars annually at organizational scale. A 24% lift in PRs needs to be weighed against that cost. For some teams, the math works. For others, it may not.
The researchers plan to continue tracking the rollout. The four-month window is short. Retention patterns may shift as the novelty fully wears off and as the tools themselves evolve. Claude Code and Copilot CLI are both receiving regular updates. The capabilities that drove adoption in early 2026 may look different by late 2026.
What is clear from the study is that CLI coding agents are not a fad. They produce measurable output gains, and adoption spreads through real social networks, not through PowerPoint decks. The open question is whether the gains compound or plateau. The answer will determine whether these tools become standard infrastructure or remain a niche for heavy coders.