Webshield Detectors
Common Sense๐
In each iteration, the detectors fetch database access log data and validate the model. If a detector detects an unusual rise in traffic, the corresponding users should be blocked. Since multiple detectors are available, all of them can be used to analyze traffic in different ways.
Model โ Aggressive Rise๐
The model defines the algorithm detectors use to identify aggressive users who are likely to be blocked.
The Aggressive Rise model works by comparing user access logs over different time periodsโfor example, in one-hour stepsโto
detect new groups of users generating the highest traffic. Each detector has a configuration variable
*[DETECTOR_NAME]_INTERSECTION_PERCENT*, which specifies the overlap (in percent) between new and old groups.
If the intersection percent is greater than the configured value, we assume the groups represent the same users and the situation is normal. If the intersection percent is lower than the configured value, we assume this indicates unusual traffic and block the entire new group of users.
Additionally, the BLOCKING_WINDOW_DURATION_SEC parameter defines the time interval over which users are fetched.
Example๐
Assume the current time is 2025-01-01 02:00:00, and we have:
BLOCKING_WINDOW_DURATION_SEC = 3600(1 hour)DETECTOR_TFT_RPS_INTERSECTION_PERCENT = 10
In this case, the TFT_DETECTOR should fetch the top active users that exceed the detectorโs threshold from the following two intervals:
- Group A: [2025-01-01 00:00:00 โ 2025-01-01 01:00:00)
- Group B: [2025-01-01 01:00:00 โ 2025-01-01 02:00:00)
The detector then calculates how many users from GroupB also exist in GroupA. If the percentage of overlapping users is less than 10%, the detector blocks all users from GroupB.
Currently, Aggressive Rise is the only model, and all detectors use it.
Floating Thresholds๐
The thresholds of detectors can be initialized with default values. WebShield is able to automatically adapt to the current situation. At each iteration, a detector updates its thresholds. The main idea is to calculate the standard deviation of the accumulated access log data.
For example, if we have 3 users with RPS values of 1, 2, and 3 respectively, the arithmetic mean is 2, and the standard deviation (1ฯ) is 0.82. The updated threshold is therefore 2 + 0.82 = 2.82. This means users with RPS greater than 2.82 fall into the risky group.
Detector IP_RPS๐
Aggregate users by IP address and calculate their RPS
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_IP_RPS_DEFAULT_THRESHOLD | 10 | Installs the default RPS threshold |
| DETECTOR_IP_RPS_INTERSECTION_PERCENT | 10 | Defines, in percent, how many users from Group B also persist in Group A. |
| DETECTOR_IP_RPS_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
Detector IP_TIME๐
Aggregate users by IP address and calculate their cumulative response time.
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_IP_TIME_DEFAULT_THRESHOLD | 10 | Installs the default accumulative time threshold |
| DETECTOR_IP_TIME_INTERSECTION_PERCENT | 10 | Defines, in percent, how many users from Group B also persist in Group A. |
| DETECTOR_IP_TIME_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
Detector IP_ERRORS๐
Aggregate users by IP address and calculate the number of responses finished with errors
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_IP_ERRORS_DEFAULT_THRESHOLD | 10 | Installs the default responses error threshold |
| DETECTOR_IP_ERRORS_INTERSECTION_PERCENT | 10 | Defines, in percent, how many users from Group B also persist in Group A. |
| DETECTOR_IP_ERRORS_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
| DETECTOR_IP_ERRORS_ALLOWED_STATUSES | [100, 101, …] | Defines the list of response status codes ignored by WebShield |
Detector TFT_RPS๐
Aggregate users by TFT-hash and calculate their RPS
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_TFT_RPS_DEFAULT_THRESHOLD | 10 | Installs the default RPS threshold |
| DETECTOR_TFT_RPS_INTERSECTION_PERCENT | 10 | Defines, in percent, how many users from Group B also persist in Group A. |
| DETECTOR_TFT_RPS_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
Detector TFT_TIME๐
Aggregate users by TFT-hash and calculate their cumulative response time.
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_TFT_TIME_DEFAULT_THRESHOLD | 10 | Installs the default accumulative time threshold |
| DETECTOR_TFT_TIME_INTERSECTION_PERCENT | 10 | Defines, in percent, how many users from Group B also persist in Group A. |
| DETECTOR_TFT_TIME_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
Detector TFT_ERRORS๐
Aggregate users by TFT-hash and calculate the number of responses finished with errors
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_TFT_ERRORS_DEFAULT_THRESHOLD | 10 | Installs the default responses error threshold |
| DETECTOR_TFT_ERRORS_INTERSECTION_PERCENT | 10 | Defines, in percent, how many users from Group B also persist in Group A. |
| DETECTOR_TFT_ERRORS_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
| DETECTOR_TFT_ERRORS_ALLOWED_STATUSES | [100, 101, …] | Defines the list of response status codes ignored by WebShield |
Detector TFH_RPS๐
Aggregate users by TFH-hash and calculate their RPS
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_TFH_RPS_DEFAULT_THRESHOLD | 10 | Installs the default RPS threshold |
| DETECTOR_TFH_RPS_INTERSECTION_PERCENT | 10 | Defines, in percent, how many users from Group B also persist in Group A. |
| DETECTOR_TFH_RPS_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
Detector TFH_TIME๐
Aggregate users by TFH-hash and calculate their cumulative response time.
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_TFH_TIME_DEFAULT_THRESHOLD | 10 | Installs the default accumulative time threshold |
| DETECTOR_TFH_TIME_INTERSECTION_PERCENT | 10 | Defines, in percent, how many users from Group B also persist in Group A. |
| DETECTOR_TFH_TIME_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
Detector TFH_ERRORS๐
Aggregate users by TFH-hash and calculate the number of responses finished with errors
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_TFH_ERRORS_DEFAULT_THRESHOLD | 10 | Installs the default responses error threshold |
| DETECTOR_TFH_ERRORS_INTERSECTION_PERCENT | 10 | Defines, in percent, how many users from Group B also persist in Group A. |
| DETECTOR_TFH_ERRORS_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
| DETECTOR_TFH_ERRORS_ALLOWED_STATUSES | [100, 101, …] | Defines the list of response status codes ignored by WebShield |
Detector GeoIP๐
Aggregate users by city and calculate their total RPS. All users from cities with unusual traffic should be blocked. It is also possible to define a list of whitelisted cities that will be ignored by the filter.
| NAME | VALUE | DESCRIPTION |
|---|---|---|
| DETECTOR_GEOIP_RPS_DEFAULT_THRESHOLD | 10 | Installs the default RPS threshold |
| DETECTOR_GEOIP_INTERSECTION_PERCENT | 10 | Defines, in percent, how many cities from Group B also persist in Group A. |
| DETECTOR_GEOIP_BLOCK_USERS_PER_ITERATION | 100 | Defines the number of users that can be blocked per check. |
| DETECTOR_GEOIP_PATH_TO_DB | /etc/tempesta/webshield/city.db | Defines the path to the MaxMind City GeoIP database. |
| DETECTOR_GEOIP_PATH_ALLOWED_CITIES_LIST | /etc/tempesta/webshield/allowed_cities.db | Defines the path to the MaxMind City GeoIP database. |