Choosing the right large language model (LLM) for each task you’re working on can supercharge your workflow. It’s not about using a single model for everything—it’s about using the best model for each specific job. Think of it as choosing the right tool from a toolbox. With Domo.AI, you can mix and match LLMs to perfectly balance speed, cost, accuracy, and security.  

In part 3 of our AI Insights Livestream series, Jeremy Rodgers guided us through choosing the best LLMs for your needs. We’re recapping his insights on the blog, and you can catch the full conversation here (hint: Jeremy’s talk starts at 22:40).  

What’s an LLM and why should you care? 

Let’s start with the basics: What is an LLM? An LLM, or a large language model, is an advanced AI system capable of understanding and generating human-like text based on the data it has been trained with.

Think of it as a supercharged autocorrect that can write, summarize information, answer questions, and even generate code. LLMs can boost your workflow by taking over tasks and giving you useful info fast. 

The key to choosing the right LLM: What do you need it to do? 

The first thing to think about is the type of task you want the LLM to do. Different models excel at different tasks. For example, some LLMs are great at coming up with creative text, while others are better at understanding and summarizing complex information.

Here are a few common tasks: 

• Text generation: Creating new content based on a prompt 
• Text summarization: Condensing large amounts of information into shorter summaries. 
• Question answering: Providing accurate answers to specific questions. 
• Code generation: Writing code based on descriptions. 

Knowing what you need the model to do will help you narrow down your options. 

Accuracy matters: How to compare LLMs 

Accuracy is crucial when choosing an LLM. You want a model that can give you reliable and precise results. Accuracy is often measured using benchmarks—standardized tests that check how well a model performs on various tasks.

For instance, the MMLU benchmark is a diverse set of tests used to measure a language model’s undergraduate-level knowledge across multiple subjects. Reviewing benchmark scores can give you a good idea of how different models compare in terms of performance. 

Budget-friendly AI: Considering price per token 

Cost is another important factor. LLMs charge based on the number of tokens processed, which includes both the input and output text. Tokens are chunks of text—think of them as parts of words. Prices can vary a lot between models.

For example, using a highly advanced model like GPT-4 might cost $75 per million tokens, while a less advanced model might cost just $1.25 per million tokens. Think about your budget and how much text you’ll be processing to choose a model that’s affordable for you. 

Speed it up: Why throughput matters 

Throughput refers to how fast an LLM can process text. If speed is crucial for your application—like in a real-time chatbot—you’ll need a model that can handle a high number of tokens per second. Larger models often process text more slowly than smaller ones.

For instance, a smaller model like Haiku might process 111 tokens per second, while a larger model like Opus might only handle 24 tokens per second. Depending on your needs, you may need to balance speed and complexity. 

Don’t lose the plot: Context window length 

The context window length determines how much text an LLM can handle at once. If you’re working with large documents or need to keep track of long conversations, you’ll need a model with a large context window.

For example, GPT-4 has a context window of 8,000 tokens, which may not be enough for very large texts. Models with larger context windows, like GPT-4Turbo, can handle more text without losing track of the context. 

Keeping your data safe: Prioritizing data security  

Security is a top priority, especially when you’re dealing with sensitive data. Domo offers a unique advantage with its DomoGPT models, which can be hosted in your private Domo cloud. This means your data doesn’t leave the secure environment of Domo, keeping everything safe. If privacy and data security are critical for your organization, using DomoGPT could be your best option. 

Wrapping it up: The power of choice in Domo 

In Domo, you have the flexibility to choose any LLM that fits your needs. Whether you’re focused on accuracy, cost, speed, context handling, or security, there’s a model that will work for you. We’re excited for you to start optimizing your work with AI and see the incredible benefits that the right LLM can bring to your projects. 

By considering these factors and leveraging the powerful capabilities of Domo, you can make an informed decision that enhances your productivity and meets your specific requirements. Happy choosing, and welcome to the future of AI-powered work with Domo. 

If you want to go deeper into the Domo.AI world, you can: 

• Watch the replay of AI Insights: Considerations When Choosing an LLM—part 3 of our summer 2024 AI series.  

• Sign up for part 4 of the series, Universal Models: Forecasting, PII, and Beyond, happening on July 25, 2024 at 1:00pm MT. 

• Check out our AI Readiness Guide, posted on our Community Forums. It’s a checklist for priming your data sets to be ready for any AI use case.