Your per-user rate limit is probably counting IPs

I found this while fixing something else. An endpoint had no rate limit, I added one, and I wrote a test that made 21 requests as one user and expected the 21st to be rejected.

It was. Then I wrote the sibling test — two different users, same office IP, ten requests each — and expected both to sail through. One of them got a 429.

The limiter was not counting users. It had never been counting users. Sixteen throttles in that file documented themselves as "per user", and every one of them was really one bucket per IP address.

Everyone behind a shared NAT was competing for a single budget, and nobody was isolated from anybody. It had been in production for months and it reads perfectly in code review.

Why it happens

Rate limiting belongs in middleware. That is the whole point — you want to reject a flood before it reaches your app, allocates objects, opens a database connection, or runs a single line of business logic. So the limiter sits early in the stack.

Authentication, in most apps, does not. It runs in a controller filter, or a concern, or a before_action. It runs late, because it needs routing, params, and often a database lookup.

Draw those two facts on the same line and the bug is obvious:

request → [30] rate limiter: "which user is this?"  ← asks here
                     ↓
              ... more middleware ...
                     ↓
          router → controller filter: authenticate!  ← answers here

The limiter asks a question that nothing has answered yet. The honest answer is nil. And almost every rate limiter is written to degrade gracefully:

def discriminator_for(req)
  current_user_id(req) || req.ip
end

That || req.ip is the whole failure. It is a sensible fallback for genuinely anonymous traffic — a login form, a public webhook — and it is why nothing ever looked broken. Requests were being counted. Limits were being enforced. 429s were being returned to people who deserved them. The key was wrong, and a wrong key produces no error, no warning, and no log line.

In the codebase I was working in, the specific culprit was this:

warden.user

Warden's user method reads an already-authenticated session. It does not run strategies — it will not perform the authentication, it only reports one that already happened. Under a cookie session established on a previous request it sometimes returns something. Under a bearer token, which is authenticated fresh in the controller on every request, it returns nil every single time.

The API was almost entirely token-authenticated.

How to check yours in five minutes

Do not read the config. The config is what fooled everyone. Ask the running system.

Log the key. Temporarily, in your limiter, emit the discriminator you actually computed:

key = current_user_id(req) || req.ip
Rails.logger.info("[throttle] #{req.path} key=#{key}")
key

Hit an authenticated endpoint the way your real clients do — with a real token, not a session cookie you made in a browser. If the key is an IP, you found it.

Then write the test that would have caught it. Not "does a flood get rejected", which passes either way:

it "counts two users on one IP in separate buckets" do
  10.times { post "/api/v1/thing", headers: auth_headers(user_a) }
  post "/api/v1/thing", headers: auth_headers(user_b)

  expect(response).not_to have_http_status(:too_many_requests)
end

That is the assertion with teeth. It fails loudly on a collapsed key and it keeps failing if someone reorders the middleware stack two years from now.

Fixing it

You have three options, in descending order of how much I like them.

Move identity resolution into the limiter. Do the minimum lookup the limiter needs — parse the bearer token, or read the session store directly — without booting the full auth stack. It duplicates a little logic, which is the cost of asking a question early. Keep it in one function so there is exactly one place to fix.

Move the limiter after authentication. Now the identity is free. But you have given up the reason the limiter was in middleware: a flood now allocates controllers and touches the database before you reject it. For abuse that is usually still fine. For a denial-of-service it is not.

Accept the IP key and document it. Sometimes honest. Say so in the comment, name the limit per_ip, and stop claiming a property you don't have. A limit that lies in its own name is worse than a limit that admits what it does.

I took the first. The discriminator now does its own session lookup, and every throttle in the file routes through that one function.

The second thing I found in the same file

Once you're auditing keys, audit storage. The counter store was an in-memory store, per process:

Rack::Attack.cache.store = ActiveSupport::Cache::MemoryStore.new

with a comment above it stating that production swapped in a shared store. Nothing swapped it. The comment had been true of an intention, never of the code.

It happened to be correct — one web process, no concurrency setting — which is a very different property from being right. Two things follow from a per-process counter, and both are quiet:

Every deploy resets every bucket. Daily ceilings feel this most. If you deploy twice a day, your "20 per 24 hours" limit is really "20 per deploy interval", and an attacker who has noticed your deploy cadence gets a fresh budget with each release.

The day a second process appears, every limit multiplies. Add a worker, scale to two hosts, and every number in that file silently doubles. No error. No log. The traffic that used to be rejected simply is not.

The fix is a shared store, with one trap on the way. Do not reach for whatever cache is lying around — check that its increment is atomic. A file-backed store typically implements increment as read-modify-write with no lock, so swapping to it trades a correct per-process counter for an incorrect shared one. That is worse, and it looks like progress. The store I moved to does it under SELECT … FOR UPDATE inside a transaction, which I verified by reading the gem rather than trusting the README.

One residual worth writing down rather than discovering later: the first increment of a fresh window has no row to lock yet, so N processes racing the very first request of a window can collapse into one. That is an undercount of N−1, once per window, against limits in the tens or hundreds. Strictly better than multiplying every limit by N — but know it, don't find it.

Finally, encode the premise your correctness rests on. If a per-process store is only safe because you run one process, write a test that fails the build the day someone adds a worker directive:

it "has not scaled out while the counter store is per-process" do
  expect(puma_config).not_to match(/^\s*workers\s+\d+/)
end

An assumption that nobody wrote down is an assumption that expires without telling you.

What this generalises to

Rate limiting is a bundle of four independent claims, and each fails silently and separately:

1. The path matches. A rule that matches nothing throws no error.
2. The key is what you think it is. This post.
3. The window behaves how you think. Fixed windows permit roughly double the nominal rate — I wrote that one up separately in your rate limiter's fixed window is a 2× burst.
4. The counter is shared and atomic. Also this post.

All four of mine were wrong at once, in a file that had been reviewed and shipped and had never once misbehaved in a way anyone could see. The reason is worth sitting with: a rate limiter has no natural failure signal. A broken cache raises. A broken query raises. A broken rate limiter just… allows things. Its failure mode is silence, and silence is indistinguishable from working.

That is why the only real test is behavioural. Not "is the config right", but "have I watched this reject the request it is supposed to reject, and allow the one it is supposed to allow?" Both halves. The allow half is where I found mine.

If you want the story of what happens when all four are wrong at the same time and someone notices before you do: your invite endpoint is a mail relay.