Tempesta Technologies
  • Home
  • Tempesta FW
    • Features
      • Web acceleration
      • Load balancing
      • Application performance monitoring
    • Performance
    • How it works
    • Deployment
    • Support
    • Knowledge base
  • Services
    • Software development
      • High performance
      • Networking
      • Databases
      • Linux kernel
      • Machine learning
      • How we work
      • Case studies
    • Performance analysis
    • Network security
      • DDoS protection
      • Application security
      • Cryptography
      • Security assessment
      • How we work
      • Case Studies
  • Solutions
    • DDoS Protection
    • Web Acceleration
  • Blog
  • Company
    • Research
    • Careers
    • Contact
Tempesta Technologies

Webshield Detectors

Common Sense๐Ÿ”—

In each iteration, the detectors fetch database access log data and validate the model. If a detector detects an unusual rise in traffic, the corresponding users should be blocked. Since multiple detectors are available, all of them can be used to analyze traffic in different ways.

Model โ€“ Aggressive Rise๐Ÿ”—

The model defines the algorithm detectors use to identify aggressive users who are likely to be blocked.

The Aggressive Rise model works by comparing user access logs over different time periodsโ€”for example, in one-hour stepsโ€”to detect new groups of users generating the highest traffic. Each detector has a configuration variable *[DETECTOR_NAME]_INTERSECTION_PERCENT*, which specifies the overlap (in percent) between new and old groups.

If the intersection percent is greater than the configured value, we assume the groups represent the same users and the situation is normal. If the intersection percent is lower than the configured value, we assume this indicates unusual traffic and block the entire new group of users.

Additionally, the BLOCKING_WINDOW_DURATION_SEC parameter defines the time interval over which users are fetched.


Example๐Ÿ”—

Assume the current time is 2025-01-01 02:00:00, and we have:

  • BLOCKING_WINDOW_DURATION_SEC = 3600 (1 hour)
  • DETECTOR_TFT_RPS_INTERSECTION_PERCENT = 10

In this case, the TFT_DETECTOR should fetch the top active users that exceed the detectorโ€™s threshold from the following two intervals:

  • Group A: [2025-01-01 00:00:00 โ€“ 2025-01-01 01:00:00)
  • Group B: [2025-01-01 01:00:00 โ€“ 2025-01-01 02:00:00)

The detector then calculates how many users from GroupB also exist in GroupA. If the percentage of overlapping users is less than 10%, the detector blocks all users from GroupB.


Currently, Aggressive Rise is the only model, and all detectors use it.

Floating Thresholds๐Ÿ”—

The thresholds of detectors can be initialized with default values. WebShield is able to automatically adapt to the current situation. At each iteration, a detector updates its thresholds. The main idea is to calculate the standard deviation of the accumulated access log data.

For example, if we have 3 users with RPS values of 1, 2, and 3 respectively, the arithmetic mean is 2, and the standard deviation (1ฯƒ) is 0.82. The updated threshold is therefore 2 + 0.82 = 2.82. This means users with RPS greater than 2.82 fall into the risky group.

Detector IP_RPS๐Ÿ”—

Aggregate users by IP address and calculate their RPS

NAME VALUE DESCRIPTION
DETECTOR_IP_RPS_DEFAULT_THRESHOLD 10 Installs the default RPS threshold
DETECTOR_IP_RPS_INTERSECTION_PERCENT 10 Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_IP_RPS_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.

Detector IP_TIME๐Ÿ”—

Aggregate users by IP address and calculate their cumulative response time.

NAME VALUE DESCRIPTION
DETECTOR_IP_TIME_DEFAULT_THRESHOLD 10 Installs the default accumulative time threshold
DETECTOR_IP_TIME_INTERSECTION_PERCENT 10 Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_IP_TIME_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.

Detector IP_ERRORS๐Ÿ”—

Aggregate users by IP address and calculate the number of responses finished with errors

NAME VALUE DESCRIPTION
DETECTOR_IP_ERRORS_DEFAULT_THRESHOLD 10 Installs the default responses error threshold
DETECTOR_IP_ERRORS_INTERSECTION_PERCENT 10 Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_IP_ERRORS_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.
DETECTOR_IP_ERRORS_ALLOWED_STATUSES [100, 101, …] Defines the list of response status codes ignored by WebShield

Detector TFT_RPS๐Ÿ”—

Aggregate users by TFT-hash and calculate their RPS

NAME VALUE DESCRIPTION
DETECTOR_TFT_RPS_DEFAULT_THRESHOLD 10 Installs the default RPS threshold
DETECTOR_TFT_RPS_INTERSECTION_PERCENT 10 Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFT_RPS_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.

Detector TFT_TIME๐Ÿ”—

Aggregate users by TFT-hash and calculate their cumulative response time.

NAME VALUE DESCRIPTION
DETECTOR_TFT_TIME_DEFAULT_THRESHOLD 10 Installs the default accumulative time threshold
DETECTOR_TFT_TIME_INTERSECTION_PERCENT 10 Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFT_TIME_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.

Detector TFT_ERRORS๐Ÿ”—

Aggregate users by TFT-hash and calculate the number of responses finished with errors

NAME VALUE DESCRIPTION
DETECTOR_TFT_ERRORS_DEFAULT_THRESHOLD 10 Installs the default responses error threshold
DETECTOR_TFT_ERRORS_INTERSECTION_PERCENT 10 Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFT_ERRORS_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.
DETECTOR_TFT_ERRORS_ALLOWED_STATUSES [100, 101, …] Defines the list of response status codes ignored by WebShield

Detector TFH_RPS๐Ÿ”—

Aggregate users by TFH-hash and calculate their RPS

NAME VALUE DESCRIPTION
DETECTOR_TFH_RPS_DEFAULT_THRESHOLD 10 Installs the default RPS threshold
DETECTOR_TFH_RPS_INTERSECTION_PERCENT 10 Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFH_RPS_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.

Detector TFH_TIME๐Ÿ”—

Aggregate users by TFH-hash and calculate their cumulative response time.

NAME VALUE DESCRIPTION
DETECTOR_TFH_TIME_DEFAULT_THRESHOLD 10 Installs the default accumulative time threshold
DETECTOR_TFH_TIME_INTERSECTION_PERCENT 10 Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFH_TIME_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.

Detector TFH_ERRORS๐Ÿ”—

Aggregate users by TFH-hash and calculate the number of responses finished with errors

NAME VALUE DESCRIPTION
DETECTOR_TFH_ERRORS_DEFAULT_THRESHOLD 10 Installs the default responses error threshold
DETECTOR_TFH_ERRORS_INTERSECTION_PERCENT 10 Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFH_ERRORS_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.
DETECTOR_TFH_ERRORS_ALLOWED_STATUSES [100, 101, …] Defines the list of response status codes ignored by WebShield

Detector GeoIP๐Ÿ”—

Aggregate users by city and calculate their total RPS. All users from cities with unusual traffic should be blocked. It is also possible to define a list of whitelisted cities that will be ignored by the filter.

NAME VALUE DESCRIPTION
DETECTOR_GEOIP_RPS_DEFAULT_THRESHOLD 10 Installs the default RPS threshold
DETECTOR_GEOIP_INTERSECTION_PERCENT 10 Defines, in percent, how many cities from Group B also persist in Group A.
DETECTOR_GEOIP_BLOCK_USERS_PER_ITERATION 100 Defines the number of users that can be blocked per check.
DETECTOR_GEOIP_PATH_TO_DB /etc/tempesta/webshield/city.db Defines the path to the MaxMind City GeoIP database.
DETECTOR_GEOIP_PATH_ALLOWED_CITIES_LIST /etc/tempesta/webshield/allowed_cities.db Defines the path to the MaxMind City GeoIP database.

Share this article
  • Home
  • Requirements
  • Installation
    • Install from packages
    • Install from Sources
  • Configuration
    • Migration from Nginx
    • On the fly Reconfiguration
    • Handling clients
    • Backend servers
    • Scheduling and Load Balancing
    • Caching Responses
    • Non Idempotent Requests
    • Modify HTTP Messages
    • Virtual hosts and locations
    • Sticky Cookie
    • HTTP tables
    • HTTP security
    • Header Via
    • Health monitor
    • Tempesta TLS
    • Vhost Confusion
    • Traffic Filtering by Fingerprints
    • Access Log Analytics
  • Run and stop
  • Application Performance Monitoring
    • Performance statistics
    • Servers statistics
  • Use cases
    • Clouds
    • High availability
    • DDoS mitigation
    • Web security
    • WAF acceleration
    • Best practices
    • WordPress tips and tricks
  • Performance
    • Hardware virtualization performance
    • HTTP cache performance
    • HTTP transactions performance
    • HTTPS performance
    • HTTP2 streams prioritization
  • Bot Protection
    • Tempesta Webshield
    • Setup and Run The Webshield
    • Webshield Configuration
    • Webshield Detectors
    • Webshield Observability
    • Webshield Use Cases
  • Contributing
    • Report issues and send patches
    • Development guidelines
    • Memory safety guideline
    • Debugging and troubleshooting
    • Prepare a new release
    • Testing
    • QTCreator project

Powered by Tempesta FW

Stay up to date with our latest developments

Useful Links

Home
Blog

Tempestaยฎ FW

Features
Performance
Deployment
Support
Knowledge Base

Services

Software Development
Performance analysis
Network Security

Solutions

DDoS Protection

Web Acceleration

Company

Research
Careers
Contact