by Beijun Desai

Hello, my name is Beijun, and today, I am going to discuss the basics of Artificial Intelligence. Everyone always talks about AI this, AI that, but what does all of this even mean? If you are a little more experienced with this field, then maybe this will serve as a quick catch-up onto some of the history and terminology.  

So, what is the field of artificial intelligence? It is the field of study that is concerned with the intelligence of machines like computers. There has been research conducted that investigates what makes a machine intelligent, how to enhance the kind of intelligence a machine can show, the best way to go about creating these kinds of systems, and plenty more. 

People believe that AI may be more recent since all the news has spread around like wildfire, but this field has been accepted as a field of study since 1956. You may have heard of some names related to this field like Alan Turning, Walter Pitts, Warren McCulloch, Noam Chomsky, and others. This field has existed for a while although the coverage recently makes it seem like the newest and biggest trend.  

So why does it seem like AI is so new and is now becoming so big? So back on November 30^th, 2022, OpenAI, an AI research organization, released a service known as ChatGPT. This free chatbot can do so much more than just mimic human conversation causing a big boom in the industry. It can write code, emails, essays, music, think of different ideas, answer questions on topics like love, death, and much more. It is no surprise that it is considered the fastest-growing consumer application to date. Within only a few months since its release, it garnered over 100 million users. With other services like DALL-E and GPT-4, OpenAI still continues to cause waves in the field. Seeing that ChatGPT can do all these amazing things, how does it work? What makes it so impressive? Well for starters, ChatGPT is a chat interface to the GPT large language model which then begs the question, what are these models?  

Large Language Models are machine learning (ML) models that have been trained on a large amount of data that can be used to accomplish a variety of tasks such as text generation, chatbots, information retrieval, and much more. Some of the more famous models include OpenAI’s ChatGPT, Google’s Gemini, Meta’s Llama, LMSYS Org’s Vicuna, etc…. One of the reasons that the GPT model, specifically GPT 3 (Generative Pre-trained Transformer 3), sets itself apart is because of the amount of quality training data  ≈700GB (see note below) and the number of parameters – 175 billion. In its broadest sense, the quality and amount of training data and the number of parameters is what is linked to how well the model can react to input and provide the desired output. Usually, the more parameters that a model has, the better its output without any hallucinations i.e. incorrect and irrelevant responses. Hence, why GPT3 is so famous.  

These models are the basis of the different features that are seen in services like ChatGPT, but AI is used in plenty of places like the Google Search Engine, personal recommendations on sites like YouTube and Amazon, games like chess and Go, and speech assistants like Siri and Alexa.  

This field is going to continue to grow as more innovations are made and the more people develop an interest in it, so I believe that it would be worthwhile to do your own research. Feel free to browse the links below to get a better and more detailed understanding of how LLM’s work and the field of artificial intelligence.  

Note: 700GB is a calculation based upon the assumption outlined in [6] in which Brown et al. state, “…570GB after filtering, roughly equivalent to 400 billion byte-pair-encoded tokens…”. The total token amount based on Figure 2.2 is 499 billion. Using this formula, 570/410=𝑥/499 where x = 693.73 giving us ≈700GB of filtered data.  

Thank you!