Throttle your API calls: RateLimitBundle

A web application is not complete without an API nowadays. APIs allow third parties - or just end users - to use the data from the platform for whatever they want. But by allowing applications to make automated calls to your API can result quickly in our systems overloading. Too many times third party applications will be polling your API when they don’t really need too, and maybe you can lighten the load a bit with some heavy-duty caching, but in essence you want that every API call made matters.

For our new startup TechAnalyze, a platform where you can automatically assess the technical skills of (PHP) developers,  we are currently in the process of creating an API for third party users. Most of our calls are pretty lightweight, but some of them aren’t, nor are they easily cacheable. This is why we are limiting the number of calls each client can make to the API. But it wouldn’t be fair to just limit the number of calls in general. Some apps would call the “light” resources, which we could handle easily and others would only call the heavy resources. So the number of calls you can do in a certain timeframe depends on the actual call. Some resources could be called 1000 times an hour, while other calls can only be called 10 times a day. By allowing this flexibility, we allow a fair usage of the API for everybody.

Our platform is written in PHP, based on the Symfony2 framework. There are many different bundles available for symfony2, all adding new functionality, but somehow we couldn’t find a (good) bundle for throttling our API. But after a search, we found a gist by Ruud Kamphuis, which pretty much does what we need. So we decided to set up a similar bundle, and added some flexibility in its usage.

Check out the code here: https://github.com/jaytaph/RateLimitBundle

What can it do?

Easy: Add the @ratelimit annotation to a controller or action, and the calls to this controller or action will be automatically throttled. As a bonus, the bundle will automatically add X-Rate-* headers to your response, giving the user information about how many calls are still available, and when the throttling will be reset. When a user makes more calls than allowed, the bundle will return a 429 HTTP status code, indicating that the user has done too many requests. All the settings can be easily modified through configuration.

Drawbacks

There are a few drawbacks to the bundle, but they are easily overcome if needed. First of all, it uses redis as a storage for the ratings. Since we need to store information about the calls, per call and quite possible per user, we don’t want to save this information into a slower system like MySQL. However, if you don’t want to use Redis, adding a new storage backend is quite easy (you would only need to implement 4 small methods).

A second drawback is that everybody has a different way of interpreting what is a distinct call. For instance, since our platform uses OAuth2, we need to store information per call and per OAuth access token. Somebody else might prefer some basic authentication scheme, and others have even different ways of identifying different users connecting to the API. I’ve setup an event handler that listens to a particular event, which allows you to “modify” the key which is used to save information about the call. In my case, the event listener will just append the OAuth access token behind the current key, making it unique for each call/user.

How does it work?

The bundle consists of a few parts:

• An annotation class.
• A configuration reader that parses the configuration.
• An event listener that does something with the annotations (the business logic)
• An event listener that adds the X-Rate-* headers to the response if needed.
• A rate limit service that pretty much only passes information to a storage backend
• A storage backend to stores and retrieves the information in Redis.
• A service.xml that ties everything together.

The annotation class

Writing annotations in Symfony2 is quite simple and the framework-extra bundle makes it even more simple. The annotation itself is nothing more than a value object which an annotation reader will fill. You don’t even need to extend or implement classes or interfaces, but I extended the ConfigurationAnnotation class (actually, you would be ok by implementing the ConfigurationInterface). This allows another event listener within the Framework-extra bundle to automatically read my annotations. It just saves me the work of creating my own reader (yeah, we’re lazy like that).

A configuration reader that parses the configuration.

Letting users configure your bundle allows much more flexibility in your bundle. For instance, it’s possible to configure the storage backend, the names of the HTTP headers, and even the HTTP response code that will be sent out. The configuration itself is created using the Symfony2 semantic config component which is a neat way of creating very complex configurations. It’s even possible to add complex validation rules to your configurations as well (for instance, we only allow you configure a custom HTTP response code in the 400 range).

But with the configuration alone you can’t do very much. You must use this configuration somewhere. The bundle will parse the configuration, and when valid, it will actually copy some of the values over to the service container parameters. This is needed because other parts of the bundle will be using this information (the event listener that modifies our HTTP response must know which HTTP code it must output, or which header names to use for instance).

An event listener that does something with the annotations (the business logic)

The business logic of the bundle takes place in this event listener and is the most complex class of the whole bundle. This event listener is hooked into the kernel.controller event, which means that it will be called AFTER Symfony2 figured out which controller to call, but BEFORE the actual call to the controller has been made. It’s a point-of-no-return so to speak, where event listeners could decide to go to another controller instead (which is exactly what we want to do when a call is being throttled).

First of all, we will check if we are running on a master request or not. Symfony2 can do internal requests (for instance, when redirecting internally to another controller or something), and we only are interested in throttling the main requests that is instantiated by a user.

Next, we will verify if the controller we just called is either a controller/action pair (which is the default behaviour), or if it’s a closure. Symfony2 is quite happy to call closures as a controller, and we will be even use this later on ourselves.

Obviously, the bundle doesn’t do anything when we run an action that doesn’t have any matching ratelimit annotations. In that case, we just skip the rest of the listener.

At this point, we have a correct controller/action pair, which has one (or more) ratelimit annotations to it, and we can figure out what and how to throttle. But the ratelimit bundle can accept multiple annotations. For instance, one annotation throttles PUT and POST requests, another one throttles GET requests, and a default annotation which throttles every other HTTP method. Which one do we need? This is dealt with a separate method called findBestMethodMatch(). It will decide based on the HTTP method which annotation must be used for throttling.

First up, we need to figure out our unique key. Normally we would be ok by concating the controllername and action and use this as a unique key, but we want to throttle per specific user as well. In our case, we are using OAuth2, so we need to add our access-token to the key as well. The bundle will actually dispatch an event (the ratelimit.generate.key event), which allows you to do whatever you want with the key. In our case, we have hooked in a OauthListener, which just adds the access token to the key.

Once we figured out the actual key to use, we can call our ratelimitservice. The limitRate() method will return a RateLimitInfo object with information about the ratelimit for the current key. If there is no ratelimit returned, we assume that this is the first time we called this key (the first time this user has called this particular controller/action), so we let the ratelimitservice create a new ratelimit object. (maybe it would better to have this done automatically by the ratelimitservice though).

Once we fetched our ratelimitinfo object, we store this inside the current request. This seems like a hackish way to store information that can be used later on by other components, and I would agree with that, but it seems the general way of doing so though.

Finally, we have come to a point where we could check to see if a user has actually exceeded the ratelimit.  It’s a really simple check (check if the number of calls of the ratelimitinfo object exceeds the limit of the ratelimitobject), and if so, we need to throttle the call.

We do this by changing the controller which Symfony2 is about to call. Actually, what we really wanted to do here, is not calling a controller at all, but directly outputting a HTTP response. Unfortunately, this is not possible in the kernel.controller events. But there are two workarounds for it. First off, we could throw an exception. In that case, symfony2 will call the exception handler, where we could check if the exception was really an exception, or just us wanting to throttle the call. This doesn’t seem a good fit for it, as we are literally abusing the exception-system for expected flows.

A better way is to change the controller into something else. We could create a dedicated controller/action that just outputs our throttle-response, and change to that controller in our event, but again doesn’t seem the right way to do so. A simpler way is to create a closure that does nothing more than returning a Response object with the correct HTTP status code and message.  Because we change the controller in the event to this closure, we let Symfony2 do its normal thing (including calling other event listeners that might listen to kernel.controller). When completed, Symfony2 will call our closure, and returning directly our response. Much more simpler and elegant.

An event listener that adds the X-Rate-* headers to the response if needed.

So we either throttle the call, or not. But either way, we would want to display some information to the client. We do this in a kernel.response listener, which is called when the response has been created by Symfony2. At this point, event listeners could add, remove or change things in the response or obviously the HTTP response headers as well.

Remember that we temporarily stored the ratelimitinfo object inside the request? At this point, we could fetch the request, extract the ratelimitinfo object, and add this information to the HTTP headers. If there is no ratelimitinfo object available in the request, it must mean that the controller we’ve called wasn’t throttled to begin with, and thus we don’t have to show anything.

A rate limit service that pretty much only passes information to a storage backend

I think the ratelimitservice is a bit of a misnomer. Since most of the business logic is inside the event listeners, this service is nothing really more than a proxy to the configured backend. This makes the bundle independent from the storage so we could use all kind of different storage backends, provided they implement the StorageInterface.

A storage backend to stores and retrieves the information in Redis.

We use Redis as our storage engine. It’s a neat key/value store and much faster as storing it in MySQL for instance. As you can see, the storage backend is quite simple, and I expect any other backend would be equally similar in setup. But if you need complex things, it’s possible too.

A service.xml that ties everything together.

Finally, we need to tie everything together. In the service.xml file we add our listeners and services and their dependencies. One thing to note is that the “noxlogic_rate_limit.rate_limit_annotation_listener” service, which is our main listener that takes care of the throttling, MUST have a lower priority than 0. This is because we are using the event-listener of the framework-extra bundle that reads our annotations and this listener runs on priority 0. If we have our listener in priority 0 (or higher), Symfony2 cannot guarantee that it will read the annotations before it will actually use the annotations. By having it -1 or lower, we are certain that the annotations are read before using them.

Conclusion

I hope you’ve gained some insight in creating a simple bundle and inspired you to write your own. Obviously, there are still things missing, not quite right and most probably a bug or two. But it shows how easy it is to setup a simple bundle. If you like to fix something, add a storage backend or something else, pull requests are always welcome!