Demonstrating the Practical Utility and Limitations of ChatGPT Through Case Studies
Abstract
In November 2022, OpenAI released ChatGPT, a chatbot powered by a large language model (LLM) called Generative Pretrained Transformer (GPT). Amazingly, ChatGPT reached 100 million users within two months. For comparison, YouTube, Instagram, and Facebook took 1.5, 2.5, and 4.5 years, respectively, to reach the same milestone. Most recently, LLMs have shifted the scientific frontier. GPT is one example. These models use an extremely expressive type of architecture called a deep neural network (DNN) to learn about the likelihood of words appearing in the context of different sentences and paragraphs. LLMs are trained on vast stores of data, comprising a sizeable percentage of the Internet. This gives LLMs great breadth of knowledge. In fact, LLMs are a special example of foundational models general models in AI that form the basis for more specialized ones trained to be experts in a domain. To answer this question, we conducted four in-depth case studies. In each case study, we used a version of GPT-3.5 provided in the ChatGPT web-based application to a complete task based on prompts we provided. The case studies described in this paper span multiple domains and call for vastly different capabilities: data science, training and education, research, and strategic planning.
Document Details
- Document Type
- Technical Report
- Publication Date
- Aug 01, 2023
- Accession Number
- AD1208615
Entities
People
- Alejandro Gomez
- Clarence Worrell
- Dominic A. Ross
- Matthew E. Walsh
Organizations
- Carnegie Mellon University