Table of Contents
Do not index
Do not index
There's a ton of hype and uncertainty around AI right now. Most people don’t know how it works and what they can realistically achieve either personally, or in business, with this new technology. After hundreds of conversations with customers, we’ve found that while people are excited and intrigued by this new technology - especially tools like ChatGPT - they often don’t know how to set these systems up to fully maximize their potential value.
Outside of the tech ecosystem, most people don’t know how to effectively prompt a model, utilize the right tools, or switch between models to handle different tasks within ChatGP! and that’s completely understandable. People shouldn’t be burdened by this. In my opinion though, we are not close to AGI yet, where one AI model can do everything, but rather it seems we are on a path where more specialized models that can do certain tasks really well. For example, OpenAI just launched a new family of LLMs that’s specialized at reasoning (o1).
What’s truly transforming the landscape is effective AI orchestration. This shift is taking us from reactive chatbots that merely respond to user queries to powerful AI Agents that can proactively manage tasks and complete workflows from start to finish, with or without human intervention.
Why Orchestration is key to unlocking value in AI
AI Orchestration is the process of configuring multiple AI models to work together, each doing what it does best, to achieve a common goal.
Why is orchestration needed? For two reasons:
- AI Models are limited in different ways. One example is the context window, or the amount of information that can be provided as input to the AI model during inference. AI models are limited in this regard and also they tend to ignore otherwise important information when the context window is “stuffed”. Imagine the AI Agent ignoring instructions on how to handle a refund.
- How would an admin go about setting up their AI Agent if the only way to configure it would be a single prompt input? It would be impossible to manage 10, 20, 100 SOPs for a support AI Agent with multiple scenarios each. How would they select and leverage different LLMs for different tasks, e.g.: knowledge retrieval vs. reasoning? Without a platform for orchestration, this becomes impossible.
Orchestration is needed because both users and LLMs will otherwise fail at managing more complex workflows, and these are the ones that drive more value.
Real-World Example: Handling a Product Return
Let me break down how this works in the real world. We recently set up an AI Agent to handle product returns for an e-commerce site.
- Routing Agent (GPT-4o): The routing agent is responsible for routing messages based on intent. It listens to every incoming message and decides which sub-agent should respond.
- First Contact (GPT4o-mini): A customer messages: "Hey, I need to return these shoes." Our 4omini model kicks in and asks for the order number and why they're returning. Nothing fancy, it’s cheap and fast for the business.
- Deterministic API Call: Once we've got the order number, the orchestration layer will execute an API call to get the order information providing deterministic access to key business systems and infrastructure which makes it reliable, secure and predictable.
- Behind the Scenes (GPT-o1 preview): Once the order information is retrieved, GPT-o1 preview takes over and uses advanced reasoning to evaluate the order and decide if the order can be returned or not. This model is more expensive so it’s only used for a small piece of the process.
- Talking to the Customer (GPT-4o): Finally GPT-4o is used to power the AI Agent in charge of interacting with the end customer. If the return is approved, it might say: "Hey there! Good news – we can definitely process that return for you. Here's what you need to do next...".
This single SOP was handled by multiple LLMs and traditional software (Deterministic API Call) working in conjunction, each doing what they do best to achieve a common goal in the most effective and cost-efficient way.
Why Scaling Orchestration = Scaling your AI Agent
Now we’ve seen how orchestration transforms an LLM from a simple chatbot that only handles Q&A to an AI Agent that can autonomously handle a return, saving valuable time to the customer and money to the business plus providing scalability and a better customer experience.
It’s clear that unlocking more value from your AI Agent lies in orchestration, because orchestration enables your AI Agent to execute more labor intensive workflows, that usually cost time and money.
So scaling your AI Agent will require more orchestration, and we are betting BIG on this with what we call flows (others call it conversational pathways). AI Orchestration is about mapping SOP or standard operating procedures to AI Agent configs. Now imagine being able to create hundreds of these orchestrations based on different triggers or scenarios, e.g.:
- Intent based routing: Trigger a different SOP based on a user’s intent (I want to change my shipping address vs. I want to return an item).
- Event based triggers: Trigger an SOP that sends an Email, SMS or WhatsApp message after a form submission, enabling your Agent to engage with prospects instantly.
- Outbound campaigns: Engage with customers at scale with powerful sequencing when a lead is added to a list, where your AI Agent is both the SDR and the system of action itself (e.g.: Outreach or Salesloft).
AI Orchestration Platforms in a post-AGI world
Our vision is that AI Orchestration Platforms will still be needed in a post-AGI world for a few reasons. First, if AGI is achieved, the importance of leveraging AI in a business and in your personal life will be unquestionable. Anyone not using AGI will become the equivalent of someone trying to build a company with just pen and paper today.
In this scenario having the right tools to set up your AGI system for success by providing SOPs, training, feedback, auditing, analytics and a fluid way for human and AGI-powered employees to communicate will be key.
Humans, which are the benchmark for AGI, fervently need these tools to run businesses today with other humans. Imagine a sales or support organization that has no training program, SOPs, knowledge base, software tooling, auditing capabilities, analytics and communication tools like Slack, Teams or WhatsApp?
AI Agents will become the most valuable digital asset
Digital assets are key components of any company or brand. Through the years we’ve seen these assets increase, from websites or ecommerce sites, to mobile apps, to social media profiles and NFTs. As AI Agents start to replace or enhance, at least to some capacity, human labor, these digital assets are poised to become every company’s most valuable digital assets.
For each one of the previous digital assets, platforms like Wix or Webflow, HootSuite or Sprout Social, etc., have enabled businesses to easily create, manage, and optimize their online presence without needing extensive technical expertise. In a similar way, platforms for AI Agents, like the emerging tools for building conversational AI or AI-powered workflows, will enable businesses to develop, deploy, and manage AI Copilots and AI Employees as integral parts of their operations, empowering them to automate tasks, enhance productivity, and offer personalized customer experiences at scale.
As AI continues to evolve, so too will the tools and strategies we use to manage it. The opportunity is massive, and those who get ahead of it now are poised to lead the charge into a future where AI is not just an add-on but the core of business success.
