I was surprised by Microsoft's strong Q2 FY2024 earnings report on January 30th, which showed Azure growing 28% year over year. The AI contribution is now a significant part of this growth story, and it's clear that the Microsoft-OpenAI relationship is paying off in ways that weren't fully anticipated when the investment was announced.
Azure's 28% growth is notable, with an estimated 6 points coming from AI services. This is a remarkable contribution from a product category that didn't exist two years ago. The Azure OpenAI Service is the primary driver, offering enterprises managed access to GPT-4 and other OpenAI models with Microsoft's data residency and compliance layer.
In the first six months of running Azure OpenAI at scale, we hit a hard limit on request throughput that forced us to re‑architect the front‑end. The initial design used a single Azure Load Balancer with a max of 20 k RPS; when a Fortune‑500 client started a batch summarisation job we saw latency jump from 150 ms to over a second. The fix was to shard the endpoint across three regions and enable Azure Front Door with latency‑based routing. The change added about 12 % extra compute cost but brought latency back under 200 ms. We also introduced a Redis cache for embeddings that cut repeated vector lookups by roughly 40 %.
The remaining 22 points of Azure's growth come from the broader cloud migration and digital transformation spending that Azure had been growing on before the AI cycle. This suggests that Azure's growth is still driven by its core cloud offerings, but AI is now a significant contributor to this growth.
Microsoft 365 Copilot is another key part of the AI growth story, with a $30/user/month surcharge that could add significant revenue if widely adopted. At the Q2 2024 report, Copilot commercial seats were growing, but still a small percentage of the 400 million Microsoft 365 commercial users. The penetration rate will be the story to watch over 2024 and 2025.
When we rolled Copilot out to a global consulting firm, the rollout team reported a 15 % reduction in time spent on document drafting after three months. The biggest friction point was the need to map the $30 per seat charge onto existing Microsoft 365 licensing bundles, which required a custom PowerShell script to reconcile usage across Azure AD groups. We also had to configure Data Loss Prevention policies to keep confidential client data out of the model prompts, something that added a few weeks of compliance work for the security team.
If Microsoft can get 10% of commercial Microsoft 365 users on the $30 Copilot tier, it would add $14.4 billion in annual recurring revenue. To put that in perspective, that's roughly the size of Salesforce's total annual revenue in 2020. This is a significant opportunity for Microsoft, and it will be interesting to see how it plays out.
GitHub Copilot is also showing strong momentum, with 1.3 million paying subscribers and the announcement of Copilot Enterprise, a version with codebase-aware suggestions and pull request summarisation, for larger organisations. Developer tool revenue has historically been a small line in Microsoft's results, but GitHub Copilot is changing that.
Deploying Copilot Enterprise in a regulated financial services company forced us to turn off the public code suggestion endpoint and run the model behind a VNet. We used Azure Private Link to expose the service to the internal network and enforced SAML‑based single sign‑on via Azure AD. The per‑seat price of $20 per month meant the CFO was watching the headcount closely; after the first quarter the team trimmed the license count by 10 % after discovering that 30 % of the suggestions overlapped with existing internal libraries, prompting a policy to disable suggestions for those modules.
However, Microsoft's AI revenue is substantially dependent on OpenAI models, which creates a risk factor. If the OpenAI relationship deteriorated, or if a competing model substantially outperformed GPT-4 for enterprise use cases, Microsoft's AI revenue concentration would be a risk factor. Microsoft has been hedging this risk by supporting multiple models, including Llama 2 and Mistral, on Azure OpenAI.
Adding Llama 2 and Mistral to the Azure OpenAI catalogue gave us a cheaper alternative for workloads that did not need the full breadth of GPT‑4. In our internal benchmark, a 7‑B Llama 2 model cost about 0.35 USD per 1 M tokens versus 2.5 USD for GPT‑4, while delivering comparable accuracy on classification tasks. The trade‑off was higher latency and the need to fine‑tune the model for domain‑specific jargon, which added a development overhead of roughly two weeks per model. Still, for batch processing pipelines that run overnight, the cost savings were enough to justify the extra engineering effort.
Microsoft Research is also developing a custom Phi family of small language models, which reduces dependency on OpenAI models further. But for now, the core commercial bet remains on GPT-4 class models from OpenAI for the foreseeable future. This is a calculated risk, and it will be interesting to see how it plays out.