What are Chat GPT Plugins and Why do they Matter?

Since the beginning of 2023, an increasing number of people have been utilizing Chat GPT to do incredible things. From helping people with computer programming, accounting, and other digital transformation tasks it has been a go to source of information and a massive productivity booster for those who are taking advantage of it.

However, one of the biggest drawbacks to Chat GPT is that it was only trained on data prior to September of 2021. Meaning that if you wanted to ask questions about current events, you would have to leverage a different service such as Bing Chat, which is powered by ChatGPT-4 but modified by Microsoft to produce shorter answers. Fortunately, OpenAI, the developers behind ChatGPT have recently announced the availability of ChatGPT plugins.

What are ChatGPT Plugins?

ChatGPT Plugins are a way for ChatGPT to connect to data that it was not initially trained on and interact with other services that are not natively part of ChatGPT’s base data.

If you have ever used services like Microsoft Power Automate Flows, or Zapier the concept is similar. These services allow you to create automations between different pieces of cloud software. Want to automatically update QuickBooks online with CRM data from Pipedrive CRM or vice-versa? Historically, you would need to have technical knowledge of how to setup API connections. With automation services, companies like Zapier have already done the hard work for you and you can setup automations using drag and drop workflow builders.

In a similar way, ChatGPT plugins will let people connect to it to other sources of information without needing any technical knowledge. This will greatly extend the usefulness of ChatGPT beyond its already amazing capabilities.

What Plugins are Available for ChatGPT?

When plugins were first announced by OpenAI, they also announced a curated list of companies and developers that they were allowing to access ChatGPT. The new plug-ins fit into four broad categories: Browsing, Code Interpreter, Retrieval, and Third-Party Plugins.

Browsing

The browsing plugin for ChatGPT allows ChatGPT to access the internet giving it the ability to respond to questions with up to date information. The connection is established using the Microsoft Bing API and goes beyond what you would expect a language model to be able to do with website data.

Instead of scraping the internet to add more information to it’s base model, ChatGPT is capable of actually browsing the internet. It can search for websites that might have the information that you need, click on additional links of a website and it responds to your question with how it found the information and can cite its sources.

Screenshot of Chat GPT Browsing the Internet
Example of Chat GPT Browsing from OpenAI

Code Interpreter

The ChatGPT Code Interpreter plugin is the one that we are most excited about. OpenAI has integrated the capability for ChatGPT to run some of the computer programming code that it generates. Specifically they have given ChatGPT the capability to run Python Scripts that it generates itself.

At the moment, the use cases are somewhat limited. The Python scripts run in a sandboxed environment, meaning that it doesn’t have access to the outside world. When we attempted to create our own scripts with no prior computer programming knowledge using ChatGPT 3.5 it was a struggle. The model has improved significantly with ChatGPT-4 and we’ve gotten a little better ourselves, but the idea of having ChatGPT write its own code, execute it, and test it out before showing an end user the results is an even bigger time saver.

One limitation with the Code Interpreter is going to be its current inability to interact with other systems. Python is one of the most widely used programming languages on the planet. It’s free, highly extensible and has a lot of useful libraries that are very useful for business process automation. There are libraries such as openpyxl for working with Excel Spreadsheets or PyPDF for working with PDF documents.

The potential use cases for the technology are basically limitless. ChatGPT is already making it very easy for non-programmers to work their way through creating a computer program. It will become even easier when ChatGPT is writing, compiling, and deploying that code on your behalf.

Retrieval

It will be interesting to see how many people take advantage of ChatGPT’s Retrieval plugin. It gives ChatGPT the functionality to connect to private sources of data, after being granted permission to do so. The idea is that many organizations have large repositories of company specific data that is not publicly available on the internet, but people may want to use a ChatGPT style of interface to interact with it.

Example from OpenAI of ChatGPT answering questions on data that's stored on private networks.
Example from OpenAI of ChatGPT Answering Questions Specific to United Nations Data

This reminds us a lot of Microsoft 365 Copilot, which is Microsoft’s initiative to integrate company data that is saved on OneDrive, SharePoint or other internal networks for use in other office applications with a ChatGPT style of interface.

On the surface it seems like it would be incredibly helpful for scenarios like customer service, or HR where they would want to be able to reference the most recent version of internal documents or an employee handbook, but we wonder how data access will be controlled.

Many businesses will also have documents that are drafts, works in progress, or were saved and never used because the company decided to go in a different direction. While they could be interesting reference material for people at the company, or the people that wrote them, we would not want to see them taken into a large language model like ChatGPT as a source of truth.

Third-party Plugins

Third-party plugins are extensions of ChatGPT that are created by companies other than OpenAI. The interesting point of third party plugins is that they are going beyond opening up ChatGPT to being able to provide additional information and becoming interactive.

This means that you could potentially ask ChatGPT what the least expensive flight is from Portland, Oregon to Phoenix, Arizona is. ChatGPT will reply with the lowest prices from a partner with an enabled plugin, like Expedia, and also allow you to purchase it without leaving ChatGPT.

In one of the most impressive demonstrations of the potential of the technology, OpenAI published a demonstration video on their blog that shows ChatGPT being asked to provide a recipe for vegan chickpea salad, then reaches out to WolframAlpha to calculate the number of calories that the salad will have, and finally invokes the Instacart connection to order the ingredients that the dish will be made with for home delivery.

In the current implementation of the software, the user is required to turn on the third party connections prior to asking for the requestion as a safety measure. It’s likely that solutions such as Apple’s Siri and Google Assistant on Android will be integrating features such as these into their digital assistants over the next several years making them truly useful and potentially gigantic time savers.

Why do ChatGPT Plugins Matter?

Plugins are the future of Large Language Models (LLMS) like ChatGPT. Even from the examples above, we can see that the features and capabilities are expanding very quickly. ChatGPT has the potential of going beyond being a source of information to a source of completing tasks and doing.

They are not just about being able to provide more up to date information, but they’re a way that people can interact with the applications and services that they are already using in a new way. ChatGPT could become the ultimate virtual assistant, especially when coupled with new advances in AI powered voice to text technologies.

It also means that the possibilities of automating systems and reaching new levels of productivity will be unlocked outside of a walled garden. Microsoft is already creating integrations of ChatGPT across all of their systems, Google has announced similar initiatives but very few companies entirely use a single brand of software solutions. ChatGPT plugins are a way that users can get more out of the systems that they are already using.

When will ChatGPT Plugins be Available?

OpenAI is taking a very cautious approach to deploying ChatGPT plugins. They were initially announced in March 2023, and are slowly rolling out to small subsets of users who have signed up for early access. OpenAI is doing this so they can control and test the user experience to ensure not only that the software is working, but that the software has controls around it to ensure that it’s not being abused or providing inaccurate or possibly dangerous information.

In general, we agree with OpenAI’s approach to delivering responsible AI solutions. It may seem like we are waiting forever to begin taking advantage of these new technologies, but the pace of advancement in the Large Language Model and AI space is incredibly fast. If you want to be an early tester of the plugin integrations, you can sign up for the ChatGPT plugins waitlist here.

The technology behind ChatGPT is already becoming quickly democratized. New open-source alternatives and alternatives that use significantly less compute resources to deploy such as Stanford’s Alpaca Project mean that OpenAI’s ChatGPT is going to become one of many different systems that people will interact with.

It’s comforting to know that OpenAI is doing the best that they can to ensure that AI is deployed in a safe manner and that it provides a beneficial experience to all users. However, we can’t help but wonder what will happen if other AI models hold themselves to a lower standard or skip steps when trying to ship a product quickly.

Conclusion

Plugins are a natural progression of ChatGPT. The software has amazing capabilities of synthesizing large amounts of information from a number of sources. ChatGPT Plugins hold the promise of not only allowing ChatGPT to respond to prompts with near real-time information, but also give it a way to interact with other systems.

The integration of ChatGPT with other systems that allow it to start completing tasks and doing them is incredibly exciting. Microsoft and Google are working on integrating similar solutions into their office suites and business software, but having a system that can complete similar tasks that is platform agnostic is going to be even more transformative for the companies that embrace this new technology.

Related Articles

Scroll to Top