Operate

Automatic remediation

Turn a firing security-threshold alert into an automatic IP-list block. Configure the condition and expiry, let Tiyi process eligible sources in batches, then verify the alert receipt, list entries and serving-node results. Stop new selections and remove existing blocks separately when recovery is needed.

Security counters → threshold and hold satisfied → alert fires
                  → eligible source IPs → global deny list → verify node application
                  → fixed-expiry cleanup, or manual entry removal

The current automatic action is global IP blocking. A site filter limits the observations and candidate sources; the resulting list applies to all sites. It is not a site-only block or arbitrary custom-rule generation. Only security-threshold alerts support automatic remediation.

1. Verify client IPs and the affected sites

Check client IPs in Logs → Security Logs. Behind a CDN, load balancer or Nginx, configure trusted proxies and verify the resulting IP with actual requests. A proxy address or shared network exit is not necessarily an individual attacker.

The existing Nginx guide uses loopback addresses to rehearse forwarding; it does not validate production client-IP trust. Do not enable automatic blocking directly on that rehearsal. Use an isolated environment, controlled test sources and an independent management path; do not generate block tests from your only management exit.

Use an account allowed to edit alert rules and IP lists. Confirm that all sites can accept this global action. For site-only remediation, use reviewed site-scoped protection configuration instead.

2. Create a security-threshold rule and review its condition

Open Alerts → Rules, create a rule and choose Security threshold. The following values illustrate the configuration; adjust them to your own traffic:

Field Example Meaning
Name Website SQL injection alert Identify the business scope
Site Select the monitored site Limits observations, not the subsequent global block
Window, seconds 300 Count matching events in a rolling five-minute window
Minimum count 100 Trigger at 100 matching events across that population
Outcome Blocked only Count security events interrupted by the WAF
Attack type SQL injection (SQLI) Candidate sources must match this filter too
Hold, seconds 60 Require a sustained match before firing
Automatic remediation Off initially Review the condition and historical replay first

The threshold aggregates the selected population: each IP does not need to reach 100 events. After the alert qualifies, Tiyi selects eligible sources from that same population. Source-IP range, attack type and minimum event severity are mutually exclusive threshold filters. Site, outcome and window still constrain the condition together. The alert's own severity classifies the alert and notifications; it is separate from minimum security-event severity.

Passed-only outcomes still represent security-rule matches that were not interrupted by the WAF, not all ordinary traffic. Routine IP-list denials do not become new CRS events feeding this security threshold, avoiding a block → alert → block feedback loop.

Review the editor's last 24 hours replay estimate to understand possible trigger volume. Replay does not save the rule, notify anyone or execute blocks. It is a historical estimate, not a guarantee of future alerts or deliveries. Investigate missing coverage or query errors rather than treating an empty result as proof of safety.

3. Enable remediation and choose an expiry

After reviewing the condition, enable Automatic remediation and select its target:

Expiry options are one hour, 24 hours, seven days or permanent. A short expiry can help during an initial evaluation; it is not a universal production policy.

Each new source gets a fixed expiry when first reserved for a remediation batch. Retries, repeated requests and continued firing do not renew an effective entry. Periodic cleanup removes expired entries and reapplies configuration; do not assume every node unblocks at the exact expiry second. Permanent entries require manual removal.

Bind notification channels as needed, review recipients, then save and enable the rule. Channels control notification delivery. A silence or failed delivery does not stop remediation. A channel's Test button sends a real message.

4. Understand firing and batch limits

When the condition stays true for the configured hold and enters firing, selection starts. While that occurrence remains firing, each evaluation can append at most five new IPs, with at most 50 distinct IPs per occurrence. This is not the list's total capacity or an immediate block for every observed attack.

Selection retains the same site, time window, outcome and chosen filter. If complete source statistics are unavailable, the alert can still fire while remediation shows Waiting for complete source statistics. Tiyi does not guess candidates from sampled logs or start that batch's expiry early.

An already-reserved IP is not queued again within that occurrence. A later occurrence can select it again: an effective entry retains its original expiry; an expired entry can receive a new fixed expiry.

5. Verify storage, execution and real requests

After saving, check in this order:

  1. Alerts → Active alerts: find the rule's occurrence and verify its firing state, condition and source evidence. Inspect remediation details for the target list, batch IPs, expiry, waiting/processing/success/failure results and any failure reason.
  2. Protection → IP Lists: open that exact list and check the added IPs, expiry and global Block binding. Saving a rule alone does not prove enforcement.
  3. Nodes: verify application results on the built-in runtime and remote nodes serving those sites. A saved list or successful controller result does not prove an offline node has updated.
  4. Logs → Enforcement: inspect IP-list denials. Routine list enforcement is not fabricated as another CRS Security Log entry. Use a separate enforcement-threshold alert to monitor enforcement pressure; it has no automatic-remediation switch.
  5. Actual requests: in an authorized test environment, send an ordinary request from a selected source and confirm IP-layer blocking, then confirm service from an unblocked source. With custom responses, inspect the enforcement layer, action and node results rather than assuming a fixed status code.

For CLI inspection, run these read-only commands on the Tiyi host using your actual IDs:

sudo tiyi alert rule list
sudo tiyi alert rule get YOUR_RULE_ID
sudo tiyi alert list --status active --rule-id YOUR_RULE_ID
sudo tiyi alert get YOUR_ALERT_ID

A custom instance needs its corresponding --admin-socket to avoid querying the default instance. Lists may be paginated. See the CLI and API reference for full fields and interfaces.

6. Stop new selections and remove existing blocks

Handle these separately:

  1. Edit the rule, turn automatic remediation off and save; or disable the entire rule to stop subsequent evaluation and new candidate selection. Already queued batches may still execute. Review remediation details and list changes first.
  2. To restore a source promptly, remove its entry in Protection → IP Lists, preserving other required entries and bindings. Confirm no in-flight batch adds it again. When retries remain, follow the results; one deletion alone does not prove recovery is complete.
  3. Check serving-node application, then request again from that source to verify the IP block is gone. Other WAF, Bot or rate-limit controls can still reject requests; inspect the actual enforcement reason.

Silence controls notifications; acknowledging or resolving an alert is not an unblock operation. Disabling a rule does not empty its list. Deleting a rule retains the list and entries as ordinary operator-managed resources, so deletion is not an unblock shortcut. If immediate restoration is unnecessary, wait for non-permanent entries to expire and verify cleanup.

7. Common symptoms

Symptom Check first
No automatic-remediation switch Rule kind is Security threshold; the account has edit permission
Alert exists but no IP was added Firing state, complete source coverage, the 50-source occurrence limit, already-processed sources
Existing-list selector is empty Eligible list has a global Block binding
IP is listed but requests still pass Actual client IP, expiry, global binding, node application and other allow configuration
New blocks continue during silence Silence does not stop detection or remediation; change the rule
Source remains blocked after resolution Entry expiry is independent of alert state; review manual removal

The example thresholds explain configuration and need adjustment for your traffic. Before production use, verify remediation and unblocking in your own controlled environment using the steps above.