Rate Limiting in Java: How to Protect Your API from Overloads and Bots

Rate Limiting and Throttling in Java: From Theory to Production Code

Imagine: your new microservice in Java has just gotten on its feet, when suddenly an avalanche of requests crashes down on it. Not attackers, no — just a careless client who forgot to add a delay in their script. The result: the database goes down, the server chokes, and users (including conscientious ones) get a 502 Bad Gateway. Sound familiar?

Today we'll talk about how to avoid this. The topic of our discussion is rate limiting and throttling in Java. These are not just trendy terms from the DevOps world, but your first line of defense for any public API. We'll break down how "rate limiting" differs from "throttling," look at popular algorithms, and, most importantly, write real code.

If you code in Java and want your service to stay alive under any load — you've come to the right place. Let's go!

Rate Limiting vs Throttling: What's the Difference?

These concepts are often confused, although they solve different, albeit related, problems.

• Rate Limiting is a hard barrier. "No more than 10 requests per second from one IP." Exceeded the limit? Get a 429 Too Many Requests. Fair and square.
• Throttling is a softer mechanism. The service doesn't block the request but slows down its processing. For example, you put the request in a queue and process it at a fixed rate.

In real projects, throttling in Java is often implemented via thread pools (ThreadPoolExecutor) with limited throughput, while rate limiting is implemented via counters, caches, and intermediate filters.

Rate Limiting Algorithms: Choosing Your Weapon

Before writing code, let's choose a strategy. Here are the three most popular algorithms for request limiting in Java.

1. Token Bucket

A classic. We have a "bucket" into which N new tokens drip every second. To execute a request, you need to take one token. If the bucket is empty, the request is rejected. This algorithm handles sudden traffic spikes well, allowing you to "borrow" tokens in advance (up to the bucket size).

2. Sliding Window

A more precise method. Instead of counting requests per calendar second, we slide a time window. For example, "no more than 100 requests in the last 60 seconds." This prevents a "burst" at the beginning of each interval.

3. Fixed Window

The simplest, but dangerous. Reset the counter every N seconds. Problem: you can make a thousand requests at the end of a window and another thousand at the beginning of the next one. Not suitable for critical systems.

We'll focus on the first two — they offer the best balance between performance and accuracy.

Writing Code: Rate Limiter from Scratch

Let's implement a rate limiter in Java using the Token Bucket algorithm. This is an excellent example for understanding how limiting systems work from the inside.

import java.util.concurrent.ConcurrentHashMap;import java.util.concurrent.TimeUnit;import java.util.concurrent.atomic.AtomicLong;

public class TokenBucketRateLimiter {

    private final long maxTokens;         // Maximum bucket size    private final long refillRate;        // Tokens per second    private final ConcurrentHashMap<String, Bucket> buckets = new ConcurrentHashMap<>();

    public TokenBucketRateLimiter(long maxTokens, long refillRate) {        this.maxTokens = maxTokens;        this.refillRate = refillRate;    }

    public boolean tryAcquire(String clientId) {        Bucket bucket = buckets.computeIfAbsent(clientId, k -> new Bucket(maxTokens));        return bucket.tryConsume();    }

    private class Bucket {        private final AtomicLong tokens;        private volatile long lastRefillTimestamp;

        Bucket(long cap