Rate Limiting for AI Coders
The cheap insurance that stops one bad actor from emptying your wallet.
In this chapter
- What rate limiting is and why every app with an AI feature needs it
- The 3 real disasters that happen without rate limits
- How to ask AI to add rate limits: exact prompts you can copy
- What limits to pick for different parts of your app
What Is Rate Limiting?
Imagine you run a coffee shop with one barista. If 200 people walk in at once, the barista can't handle it. Your service crashes. So you put a sign up: "Maximum 5 drinks per customer per hour."
Rate limiting is the same thing for your app. A cap on how many times one person can use a feature in a window of time. After they hit the cap, they're politely told to wait.
Why You Need This (3 Real Stories)
Here are three things that happen to AI coders all the time when they don't set limits:
All three are preventable. Add rate limits and none of these happen.
Where It Matters Most
You don't need limits on every button. You need them on the parts of your app that cost you money or could be abused:
How to Ask AI to Add Rate Limits
You don't write the code. You describe what you want and AI handles the technical setup. The trick is to be specific about three things in your prompt:
The 3 things to tell AI
- What feature to limit: "the AI chat", "the signup form", "the search bar"
- How many uses per time window: "20 times per hour", "5 attempts per minute"
- What to show when the limit is hit: "a friendly message saying try again later"
“Add rate limiting to my app. I want to limit my AI chat feature so each user can only send 20 messages per hour. If they hit the limit, show a friendly message: "You've hit your hourly limit, try again in [X minutes]." Use the simplest setup that works on Vercel: pick whatever free tool you recommend. Walk me through where to get any keys I need.”
Picking the Right Limits
The right number depends on the feature. Too low = annoying. Too high = pointless. Here are sensible starting points you can give AI:
10-20 per hour
Enough for normal use, blocks runaway loops and bots
100-500 per hour
Generous but still bounded
5 per minute
Real users only try once or twice. Anything more is brute force
3 per hour
Stops spamming people's inboxes
3 per hour per IP
Stops spam without blocking real customers
Whatever the API allows, divided by 2
Stay under their limit so you never get blocked
Show a Friendly Message, Not an Error
When someone hits the limit, the worst thing you can do is show a scary error like "Error 429: Too Many Requests." Real users have no idea what that means. They just think your app is broken.
❌ Confusing
- Error 429: Too Many Requests
✅ Friendly
Whoa, slow down! You've hit your hourly limit. Try again in 23 minutes.
“When my app hits a rate limit, show a friendly toast notification instead of a scary error. The message should explain what happened in plain English ("You've used your hourly limit") and tell the user when they can try again. If the user is on a free plan, also offer them an "Upgrade for higher limits" link.”
Common Gotchas
I locked myself out while testing
Tell AI: "skip the rate limit when running on localhost so I don't lock myself out during development."
Limits are too tight, real users complain
Doubling the limit is a one-line change. Tell AI "raise the AI chat limit from 20 to 50 per hour" and you're done.
Limits don't work in production
You probably forgot to add the API keys for the rate-limit service to Vercel. Settings → Environment Variables.
Same user gets a different bucket on each device
AI is using IP address as the key. Tell it to use the user's account ID instead so the limit follows them across devices.
Real World Example: Protecting your AI chat the day before launch
You're launching your AI image generator on Product Hunt tomorrow. You're paying for OpenAI on a credit card with a $50 limit. Without rate limits, one viral hour could blow past it and your card gets declined mid-launch. With limits set to 5 generations per hour for free users, even 10,000 visitors can't break the bank: the math just works out.
“I'm launching my AI image generator tomorrow. Add rate limits to protect my OpenAI bill. Free users (anyone not logged in): 3 image generations per hour, tracked by their IP. Logged-in users: 10 generations per hour, tracked by their account. Show a friendly toast when they hit the limit explaining when they can try again. Use the simplest free setup: pick whatever you recommend for Vercel. Make sure it's skipped on localhost so I don't lock myself out testing it.”
Prompt This
“My app has an AI chat that calls OpenAI. Add a rate limit so each user can only send 20 messages per hour. If they hit the limit, show a friendly message and offer them an upgrade link. Skip the limit on localhost. Use whatever free tool you recommend.”
“My app calls the [API name] API to fetch [data type]. Their free tier allows 1000 requests per day. Add caching so I don't call the API more than necessary, and a rate limit so my app never goes over their daily cap even if traffic spikes.”
“On my login page, add a rate limit: 5 attempts per minute per IP. After 5 failed attempts, show a friendly message saying "Too many tries. Wait a minute and try again." This protects against people guessing passwords.”
Frequently Asked Questions
What is rate limiting in plain English?
It's a cap on how many times someone can use a feature in a window of time. 'Free users can ask the AI 10 questions per hour.' After they hit the limit, they're politely told to wait. It's like a bouncer at a club making sure no one orders 200 drinks at once.
Why can't I just trust users?
Most users are great. But the internet has bots. And sometimes a real user clicks a button 50 times in a panic. And sometimes an honest mistake in your code creates an infinite loop. Rate limits protect you from all three: you don't need to assume bad intent, just to have a safety net.
Will rate limits annoy my real users?
Not if you set them right. A normal user might use your AI chat 5-10 times in an hour. A limit of 30/hour is invisible to them but stops abuse. The trick is to set the limit high enough that 99% of real users never notice it.
Do I need rate limits on day one?
If your app calls a paid API (OpenAI, Anthropic, etc.), yes: set them up before launch. If you're just storing data, you can wait until you have your first 100 users. The rule of thumb: anything that costs you money per use needs a limit before launch.