First, Define a 'Successful Task'
Before you can calculate cost, you have to define your unit of value. A 'task' isn't just one API call; it's a complete, successful business outcome. Is it a customer support ticket resolved without human intervention? Is it a complex user query answered
accurately? Or is it a multi-step workflow being fully executed? Defining this 'successful task' is the single most important step. It turns your Azure bill from a raw expense report into a meaningful metric about your AI's economic usefulness. Without this, you're just measuring spend, not value.
The Brain: Language Model Token Costs
The core of your AI agent is a large language model (LLM) from Azure OpenAI, and you pay for its 'thinking' time in units called tokens. Pricing is based on a pay-as-you-go model, billing for the number of input tokens (your prompts, history, and context) and output tokens (the model's answer) separately. Output tokens are often significantly more expensive than input tokens. Models like GPT-4o will have different rates than a smaller model like GPT-5-nano. Choosing the right model for the job is a major cost lever, so don't use a powerful, expensive model for a simple task.
The Memory: Data Retrieval and Search
Most sophisticated agents use Retrieval-Augmented Generation (RAG), meaning they look up information from your private data to give relevant answers. This usually involves Azure AI Search (also known as Foundry IQ). The cost here has two main parts: the fixed cost of just having the search index exist (billed hourly or monthly by 'Search Units'), and the variable cost of querying it. A common surprise on Azure bills is that these indexes accrue costs even when they're sitting idle. So, a proof-of-concept you forgot about can quietly run up charges.
The Tools: Functions and Other Services
An AI agent doesn't just talk; it does things. This often means calling other cloud services, and each one has its own price tag. A common service is Azure Functions, a 'serverless' tool for running small pieces of code. You're billed for the number of executions and the combined memory and time they use. If your agent needs to access a database, write to storage, or call an external API, each of those actions adds to the total cost of a single task. It's crucial to map out every single component in your agent's architecture to avoid missing these hidden costs.
Putting It All Together: A Sample Calculation
Let's imagine a 'successful task' is one fully answered customer question. Here's a simplified breakdown:1. LLM Cost: The agent receives a 1,000-token prompt and generates a 500-token answer using a model that costs $2.50 per million input tokens and $10.00 per million output tokens. Cost = ($2.50 / 1,000,000 1000) + ($10.00 / 1,000,000 500) = $0.0025 + $0.005 = $0.0075.2. Search Cost: To get its answer, the agent made one query to Azure AI Search, which we'll estimate at $0.0005 per query for this example.3. Function Cost: The agent called an Azure Function to check the customer's order status. This execution was tiny, costing an estimated $0.00001.Total cost per task = $0.0075 + $0.0005 + $0.00001 = $0.00801. If your agent handles 100,000 successful tasks a month, your estimated monthly cost would be around $801, not including the fixed monthly cost for the search index itself.
