Cost and Abuse Controls for LLM APIs: Rate Limits, Budgets and Denial-of-Wallet

Calling a language model costs money on every request, and the cost scales with the amount of text sent and generated. That makes LLM-backed endpoints an attractive target. An attacker, or just a buggy client, does not need to break in to hurt you; they only need to call an expensive endpoint many times. This is called a denial-of-wallet attack, and "unbounded consumption" is a risk category in the OWASP Top 10 for LLM Applications. This article describes layered controls to prevent it.

How costs get out of control

• Unauthenticated endpoints. Anyone on the internet can trigger paid model calls.
• No per-user limits. One account can send thousands of requests.
• Unbounded input. A user submits a very long document, multiplying tokens.
• Unbounded output. The model is allowed to generate thousands of tokens.
• Agent loops. An agent repeatedly calls the model and tools with no step limit (agents).
• Retry storms. A client or your own code retries failures aggressively.
• Free-tier abuse. Automated signups farm free quotas.
• Extraction. Large volumes of queries used to copy model behavior.

Layer 1: Authenticate everything

Require authentication (API key, token or session) before any model call. Anonymous access to a paid endpoint is the most common cause of surprise bills. Issue separate keys per customer so a single key can be limited or revoked.

Layer 2: Rate limits and quotas

Use two kinds of limit, enforced at different places:

• Rate limits (requests per second or minute) protect availability and slow down abuse. API gateways provide throttling, for example per-stage and per-key throttling in Amazon API Gateway.
• Quotas (requests or tokens per day or month) cap total consumption per customer or plan.

Apply limits per API key or user, not only globally, so one heavy user cannot consume everyone's capacity. Return clear errors (HTTP 429 with a retry-after hint) when limits are reached.

Layer 3: Cap request size

Limit the maximum size of the request body and the number of tokens you will accept. Truncate or reject oversized input before it reaches the model.

const MAX_INPUT_CHARS = 20_000;
if (typeof input !== 'string' || input.length > MAX_INPUT_CHARS) {
  return { statusCode: 413, body: 'Input too large' };
}

Count tokens, not just characters, for accuracy, and remember that images and documents can be expensive.

Layer 4: Cap output and model choice

Set a sensible max_tokens for each use case. Route simple tasks to smaller, cheaper models and reserve large models for tasks that need them. Do not let clients choose arbitrary models or parameters; the server decides.

Layer 5: Budgets with hard stops

Track spend in real time and enforce limits in your own code, not only in the provider's dashboard:

• A per-user daily token budget that stops requests when exceeded.
• A per-task budget for agents, covering total steps, tokens and tool calls.
• A global circuit breaker that disables expensive features if total spend passes a threshold.
• Provider-side spending limits and alerts, as a backstop.

async function chargeOrReject(userId, estimatedTokens) {
  const used = await usage.incrementDaily(userId, estimatedTokens); // atomic
  if (used > DAILY_TOKEN_LIMIT) throw new QuotaExceeded();
}

Use atomic counters (for example a conditional update in DynamoDB) so concurrent requests cannot slip past the limit; this is the same race-condition issue described in business logic vulnerabilities.

Layer 6: Cache and deduplicate

Identical or near-identical requests can often be served from a cache, which saves cost and improves latency. Cache only where responses do not depend on private user data, or key the cache by user. Some providers also offer prompt caching to reduce the cost of repeated long prefixes.

Layer 7: Bot and abuse defenses

• Require email verification and apply limits to new accounts.
• Use CAPTCHA or proof-of-work on signup and free-tier endpoints where abuse is likely.
• Detect unusual patterns: many accounts from one IP range, identical prompts, high request rates at odd hours.
• Block or slow clients that repeatedly hit limits.

Layer 8: Retries and failure handling

Use exponential backoff with jitter and a maximum retry count. Set timeouts on model calls. Avoid architectures where failures trigger duplicate expensive calls.

Layer 9: Monitor and alert

Dashboards and alerts for:

• Tokens and cost per hour, per user and per feature
• Requests that hit limits
• Unusually long inputs or outputs
• Sudden growth in new accounts

Alerts should page someone, because bills accumulate quickly, and the first sign of an attack may be the invoice.

A serverless example

For an API on AWS, a reasonable baseline is: API Gateway with API keys and usage plans (rate, burst and quota per key), a Lambda function with a reserved concurrency limit so it cannot scale without bound, input size validation in the handler, per-user token counters in DynamoDB, a provider spend cap, and CloudWatch alarms on cost and invocation counts. Grant the function only the permissions it needs (IAM hardening), and test the limits with load before launch.

Checklist

• [ ] Authentication required on every model-calling endpoint
• [ ] Per-user rate limits and quotas
• [ ] Maximum input size and output tokens
• [ ] Server-chosen model and parameters
• [ ] Atomic per-user and global budgets with hard stops
• [ ] Caching where safe
• [ ] Signup abuse defenses
• [ ] Backoff, timeouts and retry caps
• [ ] Cost alerts and a documented response plan