#rate limiting

DDoS attacks are a serious threat to organizations of all sizes, and their impact can be significant and costly. Read more...

Distributed Denial of Service (DDoS) is a type of cyber attack that aims to overload and overwhelm a website, network, or server with a massive amount of traffic. In a DDoS attack, the attacker typically uses a network of compromised computers (known as a botnet) to flood the target with traffic, rendering it inaccessible to legitimate users. DDoS attacks can be extremely disruptive and costly,…

WhatsApp, Meta’s instant messaging and Voice over IP service, was recently the focus of a research study through the University of Vienna, and its results were astounding. 3.5 billion mobile phone numbers – and the associated personal information connected to them – were compiled using the app’s lack of rate limiting to abuse contact-discovery API.

Application Programming Interface (API) is widely used for sharing information, especially on social media platforms. It’s in that hyperlinked ‘people you may know’ suggestion used almost everywhere that networking is. As a tool, it’s invaluable to those looking for their professional peers (like on LinkedIn), or expanding their social circle (like on Facebook, X, Instagram, Tumblr, etc). But it is an overlooked security gap.

The study, run from a single university server and five authenticated sessions, presumed they would be halted by the app’s security measures. They weren’t. The accounts weren’t blocked, the traffic wasn’t throttled, the IP address wasn’t restricted and perhaps most concerning, WhatsApp never attempted contact to see how and why one device was doing all this scraping. The researchers were able to collect 63 billion mobile numbers from around the globe, and then tested them against the API, which returned with their results. Aside from just generating data on which numbers are associated with WhatsApp accounts, the researchers were also able to gather data from other API endpoints, including GetUserInfo, GetPrekeys, and FetchPicture.

The research team reached out to WhatsApp themselves to report their findings, which then began the process of plugging this enormous opportunity for abuse by rate limiting API queries. As stated in the paper written about this study: “The dataset contains phone numbers, timestamps, about text, profile pictures, and public keys for E2EE encryption, and its release would entail adverse implications to the included users.” The team said this would qualify as the largest data leak in history if it hadn’t been conducted under purely empirical conditions. No kidding. 3.5 billion users is, without hyperbole, half the world. Imagine if this had been run by a threat actor using LummaStealer.

WhatsApp isn’t the only place where contact-discovery API is not adequately protected from abuse. In 2021, Facebook's ‘Add Friend’ feature was exploited to allow uploaded contact lists to check whether those numbers were on the platform. In the end, threat actors were able to create profiles for 533 million users that included their numbers, Facebook IDs, names, and more. No rate limiting was in place. Twitter suffered a similar incident in 2022, and Dell in 2024. What’s highlighted here is a need for better foundational security for information sharing since this is becoming a popular vector for cybercrime. Rate limiting will help, but doesn’t address the underlying problem in online privacy: the lack thereof.

Modern APIs power the digital economy, but rate limits create artificial ceilings on business growth. Whether consuming financial data feeds, integrating multiple SaaS platforms, or building data-driven applications, rate limiting forces developers into frustrating compromises. Residential proxies offer elegant solutions that transform API constraints into manageable scaling challenges.

Understanding Rate Limit Architecture

API providers implement rate limiting to ensure fair resource distribution and prevent system overload. Common strategies include requests per second, daily quotas, and concurrent connection limits. These restrictions often assume single-user scenarios, failing to accommodate legitimate business needs for high-volume data access.

Traditional workarounds prove inadequate at scale. Exponential backoff strategies slow data collection to crawling speeds. Multiple API keys require complex account management and often violate terms of service. Datacenter proxies face immediate detection and blocking by sophisticated API providers monitoring for distributed access patterns.

Residential Proxies as Scaling Infrastructure

Residential proxies distribute API requests across thousands of authentic IP addresses, each appearing as individual users to rate limiting systems. This distribution allows organizations to scale API consumption linearly with proxy pool size while respecting per-IP limitations.

Implementation requires careful orchestration. Assign specific residential proxies to handle defined request quotas, rotating through available IPs before limits trigger. A pool of 100 residential proxies, each handling 1,000 daily requests, enables 100,000 API calls while staying within standard rate limits.

Geographic distribution adds another scaling dimension. Many APIs apply regional rate limits, allowing higher aggregate consumption when requests originate from diverse locations. Residential proxies naturally provide this geographic diversity, maximizing available API quotas across regions.

Architectural Best Practices

Design proxy rotation systems that track per-IP usage against API limits. Implement circuit breakers that automatically switch proxies before hitting rate thresholds. This proactive approach prevents limit violations that trigger temporary bans or require manual intervention.

Session persistence matters for stateful APIs. Configure sticky sessions ensuring API authentication and subsequent requests route through consistent residential IPs. This consistency maintains API session integrity while distributing load across multiple proxies for different sessions.

Queue management becomes critical at scale. Implement intelligent request routing that distributes API calls across available proxies based on current usage, remaining quotas, and request priority. High-priority requests route to proxies with maximum remaining capacity.

Monitoring and Optimization

Deploy comprehensive monitoring tracking request success rates, response times, and quota consumption per proxy. Identify underperforming proxies or geographic regions with stricter enforcement. This data drives optimization decisions and capacity planning.

Cost optimization requires balancing proxy expenses against API overage charges or business opportunity costs from data delays. Calculate break-even points where residential proxy investments deliver positive ROI through increased API throughput.

Scaling Beyond Limits

Introduction:In today’s API-driven world, ensuring that your API is secure and efficient is crucial. One way to achieve this is by implementing throttling and rate limiting in your Django REST Framework (DRF) applications. Throttling controls the rate at which clients can make requests to your API, helping to prevent abuse and ensuring that your server resources are not overwhelmed by excessive…

Please note: The content in this blog post is no longer applicable/has been deprecated. Feel free to contact us if you have any questions. 

Over the years our legacy APIs have not had rate-limiting built into them, other than the implicit, informal rate limiting caused by performance bottlenecks. Most of the time, for most users of our public APIs, this has been sufficient. As the registry grows, however, we’ve seen heavier use of our APIs, some of which has been heavy enough to prompt us to take action. If we can identify the user and suspect it’s a bug, we reach out. Sometimes we simply block IPs when the usage is at levels that affect other people’s ability to use the APIs.

This isn’t ideal, and we’d rather give clear signals to API users what the allowed rates are and when they are being rate-limited. We’re therefore rolling out more explicit rate-limiting for all registry apis. Your tools should be ready to handle responses with http status code 429, which means that you have exceeded the allowed limit.

We will be allowing logged-in users to make requests at a higher rate than anonymous users.

For some services we have already begun limiting the amount of data that can be requested at a time. For example, we limit package search requests to queries that are at least three characters long. We have also taken steps to prevent API users from exploiting bugs to work around that limit.

We’ve also re-instituted limits in our downloads API. Our previous implementation of this API limited data ranges to well under a year for performance reasons. Our current implementation performs much better but turned out to have its breaking point as well,. You may now request at most 18 months of data at a time for single-package queries. Bulk package data queries are limited to at most 128 packages and 365 days of data.

We reserve the right to further limit API usage without warning when we see a pattern of requests causing the API to be unusable for most callers. We’ll follow up with documentation in these cases. Our primary goal is to prevent API use from either deliberately or accidentally making the service unresponsive for other users.

How to enable Rate limiting for Streaming media in Checkpoint GAIA

How to enable Rate limiting for Streaming media in Checkpoint GAIA :- Gaia is Check Point’s next generation operating system for security applications. In Greek mythology, Gaia is the mother of all, representing closely integrated parts to form a single, efficient system. The Gaia Operating System supports the full portfolio of…