ruby on rails - Nginx rate limiting for unique IPs

Question

we've been dealing with constant attacks on our authentication url, we're talking millions of requests per day, my guess is they are trying to brute force passwords.

Whenever we would block the IP with the server firewall, few seconds later the attacks would start again from a different IP.

we ended up implementing a combination of throttling through rack-attack plus custom code to dynamically block the IPs in the firewall. But as we improved our software's security, so did the attackers, and now we are seeing every request they make is done from a different IP, one call per IP, still several per seconds, not as many but still an issue.

Now i'm trying to figure out what else can i do to prevent this, we tried recaptcha but quickly ran out of the monthly quota and then nobody can login.

I'm looking into Nginx rate limiter but from what I can see it also uses the IP, considering they now rotate IPs for each request, is there a way that this would work?

Any other suggestions on how to handle this, maybe one of you went through the same thing?

Stack: Nginx and Rails 4, Ubuntu 16.

Answer 1

For your condition, the most effective way to prevent this attack is using captcha. You mentioned that you have used recaptcha, and that can be run out of soon, but if you develeop the captcha yourself and you would have unlimited captcha images.

As for the other prevent method, like lock the IPs, this is always useless when the attackers use IP pool, there are so many IPs(including some IoT devices' IPs) that you can not identify/lock them all even if you use the commercial Threat Intelligence Pool. So the suggestion like this

1. Develop the captcha yourself,and implement this on your api,
2. Identify and lock the IPs that you think malicious
3. Set some rules to identify the UA and Cookie of the http request (usually the normal request is deferent from the attack)
4. Use WAF (if you have enough budget)