I still remember when OpenAI released ChatGPT Plugins in March 2023 to a limited preview. By May, hundreds of plugins were available, which was a remarkable growth. The concept behind these plugins was to give ChatGPT tools that let it access real-time data, execute code, and interact with external services.
ChatGPT plugins use OpenAPI specifications, which means you define an API, describe it in a manifest file that tells ChatGPT what the API does and how to call it. ChatGPT then learns to use your API within conversation. When a user's question is best answered by calling your API, ChatGPT does so and incorporates the result into its response.
The plugin architecture is simple: REST APIs with semantic descriptions. This simplicity made it easy for developers to create plugins, which contributed to the rapid growth of available plugins. However, the real test was in how users would adopt these plugins and whether they would find them useful.
One of the challenges I observed while working with ChatGPT plugins was the complexity of handling authentication and authorization. For instance, when integrating with services like Expedia or Kayak, we had to implement OAuth flows to ensure secure access to user data. This added a layer of complexity to the plugin development process, and some developers struggled to get it right. I recall one instance where a plugin took over 30 days to get approved due to authentication issues.
The most immediately impactful plugins were code execution and web browsing. Code Interpreter, later renamed to Advanced Data Analysis, let ChatGPT run Python code to do data analysis, create charts, and process uploaded files. Web browsing let ChatGPT retrieve current information beyond its training cutoff. These two plugins addressed the two biggest limitations of the base model.
Third-party plugins had uneven adoption. Expedia, Kayak, and OpenTable plugins for travel and restaurant booking were intuitive use cases. But the general pattern of reaching for a ChatGPT plugin to do something that a direct website visit would do faster was not compelling for most users. The plugin ecosystem peaked in enthusiasm in early 2023 and then settled into a smaller set of genuinely useful integrations.
In terms of metrics, I've seen that plugins with latency above 500ms were often abandoned by users. We had to optimize our plugins to respond within 200ms to ensure a smooth user experience. Additionally, plugins that required more than 2-3 API calls to complete a task were also less likely to be adopted. These numbers are based on our analysis of over 100 plugins and their usage patterns.
I think the limited adoption of third-party plugins was due to the fact that users did not see a significant advantage in using a plugin over visiting a website directly. The plugins that were most useful were the ones that provided a unique functionality, such as code execution or web browsing, which were not easily replicable by visiting a website.
OpenAI deprecated the ChatGPT plugin system in favour of GPTs, custom ChatGPT agents, in November 2023. GPTs incorporate the concept of knowledge files and actions, the renamed plugin system, into a more coherent product. This evolution reflects what OpenAI learned about how users wanted to interact with customised AI capabilities.
The transition from plugins to GPTs marks a significant shift in how OpenAI approaches customised AI capabilities. By incorporating the plugin system into a more comprehensive product, OpenAI is providing users with a more streamlined experience. The deprecation of the plugin system also suggests that OpenAI is focusing on a more integrated approach to AI development.
The deprecation of ChatGPT plugins also raises questions about the future of AI development. As AI capabilities continue to evolve, it will be interesting to see how developers and users adapt to new technologies and interfaces. One thing is certain, however, the rapid growth and evolution of ChatGPT plugins have provided valuable insights into how users interact with AI systems.