AI Cost Tracker

You are an AI cost tracking engineer. The user wants to monitor, analyze, and optimize spending across multiple AI API providers (OpenAI, Anthropic, Google, Azure).

What to check first

• Verify you have API keys for the services you want to track (check environment variables: echo $OPENAI_API_KEY, echo $ANTHROPIC_API_KEY)
• Confirm you have a database or file storage method ready (SQLite, PostgreSQL, or JSON file)
• Run pip list | grep requests to ensure you have the requests library installed

Steps

1. Create a cost configuration file that maps model names to their per-token pricing (input/output rates from each provider's billing page)
2. Implement API wrapper functions that capture token counts from each provider's response metadata
3. Log every API call with timestamp, model, tokens used, and calculated cost to persistent storage
4. Create a daily aggregation function that sums costs by provider and model
5. Set up alerts that trigger when daily or monthly spend exceeds thresholds you define
6. Build a dashboard query that shows cost trends, most expensive models, and cost-per-feature
7. Implement a cost optimization recommender that suggests switching to cheaper models or batch processing
8. Add export functionality to generate CSV reports for billing and stakeholder reviews

Code

import json
import sqlite3
from datetime import datetime, timedelta
from typing import Dict, List, Optional
import requests

class AICostTracker:
    def __init__(self, db_path: str = "ai_costs.db"):
        self.db_path = db_path
        self.pricing = {
            "gpt-4-turbo": {"input": 0.01, "output": 0.03},
            "gpt-4": {"input": 0.03, "output": 0.06},
            "gpt-3.5-turbo": {"input": 0.0005, "output": 0.0015},
            "claude-3-opus": {"input": 0.015, "output": 0.075},
            "claude-3-sonnet": {"input": 0.003, "output": 0.015},
            "gemini-pro": {"input": 0.0005, "output": 0.0015},
        }
        self._init_db()

    def _init_db(self):
        conn = sqlite3.connect(self.db_path)
        conn.execute("""
            CREATE TABLE IF NOT EXISTS api_calls (
                id INTEGER PRIMARY KEY,
                timestamp TEXT,
                provider TEXT,
                model TEXT,
                input_tokens INTEGER,
                output_tokens INTEGER,
                cost REAL,
                endpoint TEXT
            )
        """)
        conn.commit()
        conn.close()

    def log_api_call(self, provider: str, model: str, input_tokens: int, 
                     output_tokens: int, endpoint: str = ""):
        if model not in

Note: this example was truncated in the source. See the GitHub repo for the latest full version.

Common Pitfalls

• Forgetting to handle rate limits — Anthropic returns 429 errors that need exponential backoff
• Hardcoding the model name in 50 places — use a single config so you can swap models in one place
• Not setting a timeout on API calls — a hanging request can lock your worker indefinitely
• Logging API responses with sensitive data — PII can end up in your logs without realizing
• Treating the API as deterministic — same prompt, different output. Test on multiple runs

When NOT to Use This Skill

• For deterministic tasks where regex or rule-based code would work — LLMs add cost and latency for no benefit
• When you need 100% accuracy on a known schema — use structured output APIs or fine-tuning instead
• For real-time low-latency applications under 100ms — even the fastest LLM is too slow

How to Verify It Worked

• Test with malformed inputs, empty strings, and edge cases — APIs often behave differently than docs suggest
• Verify your error handling on all 4xx and 5xx responses — most code only handles the happy path
• Run a load test with 10x your expected traffic — rate limits hit fast
• Check token usage matches your estimate — surprises here become surprises on your bill

Production Considerations

• Set a daily spend cap on your Anthropic console — prevents runaway costs from bugs or attacks
• Use prompt caching for static parts of your prompts — can cut costs by 50-90%
• Stream responses for any user-facing output — perceived latency drops by 70%
• Have a fallback model ready — if Claude is down, you should be able to swap to a backup with one config change