Reduce Alert Fatigue and Optimize Decision-Making with CrowdSec CTI Helpers

One of the fundamentals of the CrowdSec project is that, while our users contribute signals by providing details on blocked attacks, we generate a blocklist (often called fire) and redistribute it to the users.

However, the fire blocklist is the tip of the CrowdSec CTI iceberg. Our community users are allowed one CTI API key with up to 50 calls per day, and we just added CTI helpers, a new feature to make CTI use easier from within the CrowdSec Security Engine, so let’s take a look. I will also show you how to leverage the CTI at runtime to alter the decision that CrowdSec would take, or add context to your notifications.

Introducing CTI helpers

With CrowdSec Security Engine 1.5, we added a CrowdsecCTI  helper that allows you to fetch the CrowdSec CTI info on an IP. Providing all our users (including free console users) with access to CTI API keys, we do believe it was the right thing to do. 🙂

The CTI helper can have many uses, and in this article, I will only be scratching the surface of a nearly infinite number of possibilities and applications. Despite the limited scope of this tutorial, I will do my best to demonstrate how direct and easier access to the CTI through the Security Engine can have a great impact on your security processes. 

• Reduce alert fatigue & optimize triage: Use CTI data to alter notifications, or even hide them when they are not interesting. By relying on CTI information, we can decide which information should be visible to certain teams, what is worth pushing to SOC workflows, etc.
• Increase decision-making speed: Enrich your security information set, and allow for faster and better decision-making. The typical threat intelligence (TI) workflow — receiving an alert, checking IP attributes, and making a decision — is unnecessarily lengthy. By embedding the TI in the Security Engine, the TI information is bundled in the alert itself, skipping step two, and helping humans save time!
• Increase decision-making accuracy: As different scenarios trigger different remediation actions, you can rely on the CTI to create advanced scenarios, such as detecting successful authentication from VPN or proxies. Consider having payment errors on your e-commerce website triggered by an IP that is a well-known public proxy. Well, you might not want to treat this incident in the same way you would treat a real customer having actual issues with your payment forms!

Requirements

To take advantage of CTI helpers and to follow along with this tutorial, you’ll need the following:

• CrowdSec Security Engine 1.5.0. Download it here.
• A medium to advanced level of understanding of how CrowdSec works. If you need a CrowdSec crash course, our Academy is always available!

Enhanced notifications: CTI context

Before we get started, you need to make sure you have notifications turned on in /etc/crowdsec/profiles.yaml. Here I decided to pick Slack notifications, and I’m making a custom profile so that manual decisions (as we are going to test by using cscli directly) are passed to the notification plugins:

name: manual_ip_remediation
#debug: true
filters:
 - Alert.GetScope() == "Ip"
notifications:
  - slack_default  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
---
name: default_ip_remediation
#debug: true
filters:
 - Alert.Remediation == true && Alert.GetScope() == "Ip"
decisions:

Note: Here, I added the section manual_ip_remediation. It is a bit different from the default profile because I removed the Alert.Remediation == true condition, that is only valid when a scenario is being triggered, but not when you issue a manual decision with cscli.

Configure your Slack notification plugin

Here, you should have nothing more to do than slapping your Slack webhook URI in your /etc/crowdsec/notifications/slack.yaml file.

Test that it works

With this set, if I emit a manual decision, I’m going to get a Slack notification:

# cscli decisions add -i 6.7.8.14
INFO[27-07-2023 17:26:38] Decision successfully added

You can now use the new CTI helpers, so let’s take the first step by enriching our notifications with contextual IP information.

Configuring CTI API key

First, create your CTI API key from the CrowdSec Console through the Settings > API Keys section.

You now need to put your API key in the API section of your /etc/crowdsec/config.yaml file. Check out the documentation for more information.

api:
  cti:
    #The API key you got from the console
    key: ...
    #How long should CTI lookups be kept in cache
    cache_timeout: 60m
    #How many items can we keep in cache
    cache_size: 50
    enabled: true
    log_level: trace
  client:
    insecure_skip_verify: false
    

You can now use the CrowdsecCTI helper in your profiles and notification templates.

Enriching notifications with CTI context

This is how the template looks like by default:

format: |
  {{range . -}}
  {{$alert := . -}}
  {{ /* this part sets the flag if available, or defaults to pirate flag */ }} 
  {{range .Decisions -}}
  {{ $country_flag := ":pirate_flag:" }}
  {{ if $alert.Source.Cn -}}
  {{ $country_flag = print ":flag-"  $alert.Source.Cn  ":" }}
  {{ end }}
  {{ /* this is the actual part that we're printing */ }}
  {{$country_flag}}  [https://www.whois.com/whois/{{.Value}}|{{.Value}}] will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}} on machine '{{$alert.MachineID}}'. [https://app.crowdsec.net/cti/{{.Value}}|CrowdSec CTI]
   {{end -}}
  {{end -}}
  

Now, I’m going to extend it with some CTI information. First of all, I’m interested in the categories associated with the IP. To make this visible, I rely on the CTI helpers I mentioned above. The returned object contains a classification object, holding both false positive categories and normal categories, and this is what I want to look into:

{{- $cti := $alert.Source.IP | CrowdsecCTI  -}}
{{ if $cti.Classifications }} 
{{ range $cat := $cti.Classifications.Classifications }} 
*{{$cat.Name}}*
{{ end -}}
{{ end }}

This will allow me to display the categories associated with the reported IP(s). Here is what the template looks like:

format: |
  {{range . -}}
  {{$alert := . -}}
  {{range .Decisions -}}
  {{/*this sets the emoji flag if available*/}}
  {{ $country_flag := ":pirate_flag:" }} 
  {{ if $alert.Source.Cn -}}
  {{ $country_flag = print ":flag-"  $alert.Source.Cn  ":" }}
  {{ end }}
  
  {{$country_flag}}  [https://www.whois.com/whois/{{.Value}}|{{.Value}}] will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}} on machine '{{$alert.MachineID}}'. [https://app.crowdsec.net/cti/{{.Value}}|CrowdSec CTI]
  {{/*this fetches cti data for the IP, and display classification names*/}}
  {{- $cti := $alert.Source.IP | CrowdsecCTI  -}}
  {{ if $cti.Classifications }} {{ range $cat := $cti.Classifications.Classifications }} *{{$cat.Name}}* {{ end -}} {{ end }}
  {{end -}}
  {{end -}}
  

With your updated notification template, you can restart CrowdSec and have a look:

# cscli decisions add -i 72.167.42.59
INFO[27-07-2023 17:46:28] Decision successfully added

Here, as you can see, contextual information is displayed about the originating IP, which can help a lot in decision-making. The information returned by the CTI is either available via methods or directly in the object returned by the helper.

Enhanced notifications: Alert on false positives

Now that you have the whole CTI part set, you can leverage it further, for example, to avoid or mitigate the risk of false positives. For instance, a web server without proper x-forwarded-for configuration might see all attacks as coming from its CDN, which would then lead to banning the CDN IPs. This would be both inefficient and damaging, ultimately leading to a self-Denial of Service. This is where the CTI helpers can be useful, by discarding this dangerous ban decision and instead emitting a notification that would alert your teams of the situation.

I am going to create a special profile for that purpose in the /etc/crowdsec/profiles.yaml file:

name: known_fp_ban
#debug: true
filters:
 - Alert.GetScope() == "Ip" && CrowdsecCTI(Alert.GetValue()).IsFalsePositive()
notifications:
  - slack_fp  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
on_success: break
---

This profile looks for alerts targeting IPs (Alert.GetScope() == "Ip") that are known false positives (CrowdsecCTI(Alert.GetValue()).IsFalsePositive()). When this happens, the notification slack_fp is executed (notice the lack of decision), and other profiles (on_success: break) aren’t evaluated.

What is considered a false-positive can be found here, and is usually known IPs of CDNs, DNS, mainstream SEO crawlers, known security vendors, etc.

Now let’s create the slack_fp notification template, that will report the false positive and highlight the associated categories of the IP. This is intended to notify the team in charge so they can fix the scenarios or the associated service configuration.

type: slack           # Don't change
name: slack_fp   # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: trace

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
#group_threshold: 3    # Amount of alerts that triggers a message before [group_wait] has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the slack message
format: |
  {{range . -}}
  {{$alert := . -}}
  {{- $cti := $alert.Source.IP | CrowdsecCTI  -}}
  :warning: Discarded ban on FP :warning:
  [https://www.whois.com/whois/{{$alert.Source.IP}}|{{$alert.Source.IP}}] triggered *{{.Scenario}}* on [{{substr 0 7 $alert.MachineID}}]
  [ip:{{$alert.Source.IP -}}
  {{if $alert.Source.Cn }} cn:{{$alert.Source.Cn}}{{end -}}
  {{if $cti.AsName }} as:{{$cti.AsName}}{{end -}}
  {{if $cti.ReverseDNS }} dns:{{$cti.ReverseDNS}}{{end -}}]
  {{ range $cat := $cti.GetFalsePositives }} - [https://doc.crowdsec.net/docs/next/cti_api/taxonomy#false-positives|{{$cat}}] {{ end }}
  {{end -}}

webhook: https://hooks.slack.com/services/...

Note: This is a copy of the existing slack.yaml file. I changed the name of the template and the format to reflect our purpose:

• The offending IP and the scenario + machine
• The originating country, AS name, and reverse DNS when available
• The false positive categories with a link to the taxonomy

You can decide to send this notification differently due to its importance by, for example, switching the Slack channel, or by using another notification plugin, such as “mail”, to make traceability easier.

Other ideas and future development

This first introduction to the new CTI helpers showcases only a few basic usages, but a lot more is, of course, possible. In this tutorial, I showed CTI usage in profiles and notifications, but as you might have guessed already, it works in scenarios too. Over the years, we’ve seen CrowdSec users being brilliantly creative and innovative with Security Engine applications, so, by all means, be creative with the new CTI helpers as well! 

And don’t hesitate to share your use cases with us (Discord | Discourse) or let us know how we can improve things.