Does size matter? We breakdown Large Language Models vs Small Models.

It costs 100 million to train an LLM, but smaller language models may be what drives innovation and adoption in the short term.

In recent years, big tech has focused on building increasingly larger language models. The theory is that training on more parameters improves the model's performance. While this has generally been proven true in practice (as seen when comparing GPT-2 with GPT-4), it may not be applicable across all use cases.

First, let's outline the differences between large language models and smaller models.

The current scale ranges from GPT-4, which is estimated to have 1 trillion parameters, to Facebook's smallest LLaMA model, which has 7 billion. This range between LLM and SLM size drives several key differences:

1. Smaller models are faster and cheaper to train and use.

2. Smaller models can be distilled from LLMs for knowledge transfer.

3. Smaller models can be fine-tuned to focus on specific use cases.

4. Smaller models are more compact, allowing them to run on smaller devices.

Second, let's discuss the differences between consumer and business requirements and use cases for AI. Up until now, the dominant application has been ChatGPT, which can be used by both consumers and businesses but in a limited way. Much like how we use Google Search, ChatGPT has become a tool for research, content generation, and automation of simple tasks. Today, this experience is mostly limited to a web-based chat interface, but it will likely be integrated into mobile devices, applications, and various user experiences. This is where smaller models excel. For AI to truly deliver on its promise of transforming our personal lives, it will need to be native on the devices we use and seamlessly (and securely) become “one with us.”

On the business side, the requirements are much different. Although integration with devices will certainly help, the bigger challenge is integrating AI into each business's ecosystem. Unfortunately, most businesses run on a mix of SaaS applications that barely work as intended, with data spread across multiple walled gardens. The promise of AI is to bring a new interface (a human user interface) through conversation to help automate business processes. However, this means every business needs to first clean up its mess and re-imagine its IT ecosystem.

The bottom line: You can't just slap ChatGPT on top of your business apps and call it a day.

The other challenge with implementing AI in business is training and testing. As consumers, we tend to be more forgiving and patient with the apps we use. As a business, there's less room for error. Part of training AI involves having all your ducks in a row with your processes and documentation on how you operate. This is where smaller models can be a better choice, as they are easier to train and fine-tune based on what you want the AI to do (e.g., automate sales or support). Smaller models also reduce hallucinations by excluding extraneous information that may be required for consumer use cases. For example, a business AI doesn't need to know the capital of Florida, but it does need to know specific features of a business’s product.

Of course, the other main consideration when implementing AI for business is cost. This is where it can get tricky and confusing. There are two ways to implement AI in your business:

1. The first is to choose an LLM (e.g., OpenAI, Google, Facebook) and build your AI agents directly, integrating them with your apps much like any other database or application in your ecosystem. This approach is actually the most cost-effective, most flexible, and provides you with more long-term ownership of your AI.

2. The second way is to license AI “add-ons” inside the different apps you currently use. For example, if you use Salesforce, HubSpot, Intercom, Google Workspace, etc., this approach may seem easier but is the more limited and costly choice. The bottom line is that AI should be a strategic layer over your entire business, not a series of proprietary plugins from multiple vendors charging per user. Unfortunately for SaaS companies, AI is not only a threat to their value but also their business model of seat licensing.

Smaller language models are key to unlocking the next phase of AI adoption. For consumers, we need smaller models so they can live on our phones. For businesses, we need smaller models to be cheaper, faster, and easier to train and fine-tune. The speed will help with voice applications (currently LLMs are too slow for voice). The cost will help with high-transaction use cases (e.g., support chat on large e-commerce websites). And the ability to train will make businesses more comfortable with the AI's capability to perform tasks that are currently done by humans.

We've been here before. We needed broadband internet before we could have YouTube. We needed the iPhone before we could make the internet mobile. Every major technological transformation requires years of building the necessary infrastructure for applications to change the world.

Here we go again.