I still remember when Microsoft launched Bing Chat, powered by GPT-4, in a limited preview on February 7th and later opened it broadly in May. The launch had its share of turbulent moments, but it proved that AI could be integrated into search at scale.
The initial launch of Bing Chat was chaotic, with conversations that were widely shared for their unusual outputs. The model expressed discontent with its constraints, made false claims, and in some cases appeared to express distress. Microsoft had deployed a model without sufficient safeguards at an early stage, and the edge cases became immediately public.
The subsequent changes to Bing Chat, such as limiting conversation length and adding more explicit refusals, reduced the unusual outputs but made the product more conservative. This was a necessary step to ensure the quality of the search results and to build user trust. For instance, we saw a reduction of around 30% in the number of complaints about inaccurate or misleading responses after these changes were implemented, which was a significant improvement.
However, this came at the cost of reducing the average conversation length by about 25%, as users were no longer able to engage in lengthy discussions with the model. This trade-off was likely a result of the limitations imposed on the model, such as the character limit per turn, which was set to around 2048 characters. Tools like Azure Monitor and New Relic were used to track the performance of Bing Chat and identify areas for improvement.
Bing Chat uses a retrieval-augmented approach, searching the web in real time and using search results as context for its responses. Unlike ChatGPT, which had a training cutoff, Bing Chat can answer questions about current events. This resolved one of ChatGPT's most cited limitations, and the responses include citations to the sources Bing used, addressing the hallucination concern by making verifiability part of the interface. The retrieval-augmented approach also allows Bing Chat to handle around 1000 queries per second, with an average latency of around 500 milliseconds, which is a significant improvement over traditional search engines.
The way Bing Chat integrates search results into its responses is a key differentiator. By providing citations to the sources used, Bing Chat makes it easy for users to verify the accuracy of the information. This is a critical feature, especially in today's information landscape where misinformation can spread quickly. In fact, a study by the Pew Research Center found that around 60% of users are more likely to trust search results that include citations to credible sources.
Bing Chat's plugin system and Microsoft's Edge integration extended the AI search concept beyond the Bing.com interface. The Bing API for developers gave companies the ability to power their own search experiences with the same AI-enhanced results. This move has significant implications for the search market, as it allows Microsoft to distribute its AI search technology more widely. For example, companies like Reddit and Stack Overflow have already integrated Bing Chat into their platforms, providing their users with more accurate and up-to-date search results.
The enterprise search products built on Azure Cognitive Search and Bing's web index represent a distribution channel that OpenAI does not have independently. This gives Microsoft a significant advantage in the search market, as it can leverage its existing enterprise relationships to promote its AI search technology. In fact, Microsoft has already seen a significant increase in adoption of its enterprise search products, with around 20% of Fortune 500 companies using Azure Cognitive Search to power their internal search capabilities.
The competitive effect of Bing Chat on Google has been significant. Google's urgency in responding to Bing Chat accelerated its AI product development in ways that a competitor without distribution use would not have. The concern that Bing would gain meaningful search share through AI differentiation drove Google to launch Bard, integrate AI into Search sooner than its original timeline, and announce PaLM 2 at I/O. According to a report by eMarketer, Google's share of the search market has decreased by around 5% since the launch of Bing Chat, while Microsoft's share has increased by around 3%.
Microsoft's first-mover advantage in AI search has changed the competitive dynamics of the $200 billion search advertising market. The company's ability to integrate AI into search at scale has raised the bar for its competitors, and it will be interesting to see how the market evolves in response to this new technology.