Difference between revisions of "Rate limiting"

From Halon, SMTP software for hosting providers
Jump to: navigation, search
(Rate limit page in Web UI)
Line 28: Line 28:
  
 
== Rate limit page in Web UI ==
 
== Rate limit page in Web UI ==
The rate limit page in the Webui may display the rate limit database both in clustered mode (<tt>/cluster/ratelimit/</tt>) and standalone per unit (<tt>/cluster/ratelimit/</tt>), in both layouts different information is presented. Values may be alphabetically paged and filtered by name and minimal count (threshold).
+
The rate limit page in the Web UI may display different fields depending on how it is configured. Entries may be alphabetically paged and filtered by name and by a minimal count (threshold).
  
 
{| class="wikitable"
 
{| class="wikitable"
 
! Result !! Description
 
! Result !! Description
 
|-
 
|-
| Namespace || The namespace or "bucket"
+
| Namespace || The <tt>namespace</tt> or "bucket".
 
|-
 
|-
| Entry || The name of the rate limit item
+
| Entry || The name of the rate limit <tt>entry</tt>.
 
|-
 
|-
| Count || The maximum value a single node has, this should be the same on all units.  
+
| <i>See table below.</i>
 
|-
 
|-
| <i>See table below</i>
+
| Interval || The time between first and last <tt>count</tt>.
 
|-
 
|-
| Interval || The time between first and last count
+
| Most recent || When a <tt>count</tt> was last added to this entry.
|-
 
| Most recent || When the rate limit was last hit
 
 
|}
 
|}
  
 
=== Cluster ===
 
=== Cluster ===
  
If viewed in a cluster (<tt>/cluster/ratelimit/</tt>).
+
If viewed in a cluster (<tt>/cluster/ratelimit/</tt>). The rate limit may or may not be synchronized.
  
 
==== Synchronized ====
 
==== Synchronized ====
Line 55: Line 53:
 
! Result !! Description
 
! Result !! Description
 
|-
 
|-
| Count || The maximum value a single node has, this should be the same on all units.  
+
| Count || The maximum <tt>count</tt> a single unit has; this should preferable be the same on all units.  
 
|-
 
|-
| Min || If shown, it is the lowest value a single node has that is not count.
+
| Min || If shown, it is the lowest <tt>count</tt> a single node has.
 
|}
 
|}
  
Line 67: Line 65:
 
| <i>Same as for synchronized</i>
 
| <i>Same as for synchronized</i>
 
|-
 
|-
| Sum || The total sum of all counts on all units.
+
| Sum || The total sum of all <tt>count</tt>s on all units.
 
|}
 
|}
  
Line 77: Line 75:
 
! Result !! Description
 
! Result !! Description
 
|-
 
|-
| Count || The count this node has.
+
| Count || The <tt>count</tt> this unit has.
 
|-
 
|-
| Local count || The count created on this node (may be same as Count).
+
| Local count || The <tt>count</tt> created on this unit (may be same as Count).
 
|}
 
|}

Revision as of 21:48, 15 May 2014

Rate limiting is a crucial defense method against service misuse. We have implemented a global rate controlling service called rated. It handles rate limiting for all processes, based on an "items per time interval" algorithm. Our implementation features a sliding rate limit (in contrast to leaking bucket; which has a fixed output rate regardless of exactly when the rate was exceeded).

Implementation

Rate limits may be "hit" using the HSL rate() function, and sometimes using the graphical blocks in the flows. It returns true as long as the rate() call interval is less than its rate limit, but false when exceeded. If the count parameter is zero, the current rate is returned instead. When a "hit" is counted, its expiration time is the fourth parameter (interval).

// get the rate limit number, without increasing
if (rate("failed-login:saslusername"$saslusername060) >= 3)
    
Reject("Too many failed logins");

// increase, and reject if exceeded
if ($is_spam)
    if (
rate("spammers"$senderip10086400) == false)
        
Reject("You may only send 100 suspicious messages per day"); 

Rate limits may be read and cleared using the SOAP API.

Clustering

Rate limits are synchronized in the cluster if a HMAC-SHA1 key is specified on the system settings page. The protocol is based on UDP and uses port 13131. In terms of security it may be wise to only activate this feature on local networks, as the HMAC-SHA1 only serves as a packet-authenticity firewall. It does not provide encryption nor protect against reply attacks. It should probably be protected by other means such as a VPN tunnel or psychical security (DMZ).

Performance

The rate limits performance is roughly O(log N + log M) and lookups are currently implemented as a C++ std::multimap.

Rate limit page in Web UI

The rate limit page in the Web UI may display different fields depending on how it is configured. Entries may be alphabetically paged and filtered by name and by a minimal count (threshold).

Result Description
Namespace The namespace or "bucket".
Entry The name of the rate limit entry.
See table below.
Interval The time between first and last count.
Most recent When a count was last added to this entry.

Cluster

If viewed in a cluster (/cluster/ratelimit/). The rate limit may or may not be synchronized.

Synchronized

Result Description
Count The maximum count a single unit has; this should preferable be the same on all units.
Min  If shown, it is the lowest count a single node has.

Non-synchronized

Result Description
Same as for synchronized
Sum The total sum of all counts on all units.

Single unit

If viewed on a single unit (/ratelimit/).

Result Description
Count The count this unit has.
Local count The count created on this unit (may be same as Count).