Sending pfSense logs to QRadar

QRadar Community Edition offers a great way to better understand the product.  I have decided to install QRadar CE on a VM at home to give it a try.  Now, I do not run any of the big firewalls that they support but I do run pfSense, which i believe is one of the best solutions for the home network.

After doing a couple of searches on the Internet, I found that there is no DSM for pfSense logs available for QRadar.  I had decided to give a try on my own custom DSM to parse these logs.  After some time and effort, I have been able to properly parse the following logs from pfSense in QRadar:

  • Firewall Logs
  • DHCP Events
  • DNS Queries

I have also been able to run Snort and softflowd (Netflow) on pfSense and send the IDS logs and flow information to QRadar.

In this article, we will be showing how to send the pfSense Firewall Logs into QRadar and use the custom log source extension I am providing to help parse the logs correctly.  Note that this is a work in progress and there are events that are not correctly parsed, or not parsed at all.  Also,  the regular expressions that I am using may not be the most efficient, but at least they will get the job done.  Any recommendations on improving them will be appreciated in the comments.  This LSE works for pfSense 2.2 and above.

You can download the XML file here.

Creating the Custom Log Source Extension in QRadar:

The following steps go through the process of creating a log source extension for QRadar and uploading the XML file.

  1. In QRadar, go to Admin page and click DSM Editor under the Data Sources / Events section.
  2. In the DSM Editor screen you will be prompted to select a Log Source Type,  click “Create New”.
  3. Enter the name “pfSense” for the new Log Source Type and then click Save.
  4. Close the DSM Editor and then click on Log Source Extensions in the Admin page.
  5. In the Log Source Extension screen, click “Add”
  6. In the Name Field type:  “pfSenseCustom_ext” (without the quotes).
  7. Type a description in the Description field.
  8. In the log source types section, look for the one you created called “pfSense”  and click the button to move it to the right to set this Log Source Extension as default for the pfSense Log source Type.
  9. Use the Upload Extension to upload the XML provided in the link above.
  10. Once the XML file has been uploaded click on Save.

You will now be able to parse logs for pfSense by using this new log source extension.

Configure Event Mappings

Once you’re able to correctly parse the information in QRadar, you will still need to map the events to existing QIDs so that QRadar can know what type of event it is.  For example, a Firewall Deny or a Firewall allow are two different type of events and may generate different offenses according to your rules.

There are a couple of tools that I have tried to use to export the QIDMAPs including the qidmap_cli.sh tool. However, I have not yet had any luck  exporting the event mappings, therfore, they need to be created manually.

Mapping of pfSense Events

  1. In the QRadar Admin page, select DSM Editor
  2. Selct the pfSense Log Source Type
  3. Click on the Event Mappings tab and then click the “+” button to create a new mapping.
  4. Use the tables below to enter the values for each event.  You will need to enter the event ID and category, then click the Choose Event link to search for the QID/Event
  5. Type in the event name as shown in the tables below, select the matching event and click OK.
  6. Click create to finish creating the mapping.

Firewall Mappings:

There are only two main event mappings used for Firewall Events which are the most common events that you will be getting from pfSense.

Event Event ID Category QID/Name
Firewall Permit pass filterlog Firewall Permit – Event CRE
Firewall Deny block filterlog Firewall Deny – Event CRE

DNS Mappings:

In order to get DNS Queries you will need to be using unbound as your DNS service in pfSense.  Unbound is the default DNS service in pfSense 2.3 and above.  You will also need to configure unbound to send DNS query events to the system logs.  This is described in the Configuring pfSense to Send Logs to QRadar section below.  The mappings below correspond to DNS Query events.

Event Event ID Category QID/Name
A Query A unbound DNS Query – A record
Quad-A Query AAAA unbound DNS Query – AAAA record
CNAME Query CNAME unbound DNS Query – CNAME record
MX Query MX unbound DNS Query – MX record
NS Query NS unbound DNS Query – NS record
PTR Query PTR unbound DNS Query – PTR record
SOA Query SOA unbound DNS Query – SOA record
SRV Query SRV unbound DNS Query – SRV record
TXT Query TXT unbound DNS Query – TXT record

DHCP Events Mappings:

DHCP Events also include identity fields.  This will allow QRadar to update the discovered assets using DHCP event information.  In order to obtain the identity information, you will need to override the Identity Fields for events.  To do this, once the mappings have been created and saved,  click on the Override default identity fields for this event.  Then click Select in the Identity Fields section.  Select all available properties and then click OK  The following table includes the DHCP events that are mapped.

Event Event ID Category QID/Name
DHCPACK DHCPACK dhcpd DHCPACK
DHCPDISCOVER DHCPDISCOVER dhcpd DHCPDISCOVER
DHCPOFFER DHCPOFFER dhcpd DHCPOFFER
DHCPREQUEST DHCPREQUEST dhcpd DHCPREQUEST

Configuring pfSense to Send Logs to QRadar

Now that we have created the custom DSM and custom mappings, we move on into pfSense to configure it to send the logs to QRadar via syslog.  The following steps need to be performed on the pfSense administration console.

  1. Go to Status -> System Logs.  Then click on Settings on the far right side.
  2. In the System Logs / Settings page, scroll down to the remote logging options and enter the following:
  • Enable remote logging:  Check
  • Source Address:  Select the interface whose address you want to use as the source address.  I recommend that you select the interface where your QRadar system is connected to.  For example,  if QRadar has an IP address in the LAN Segment, then use the LAN interface as source address.  You will need to make note of the the IP address for pfSense on this interface since you will be using it when you configure the Log Source identifier.
  • Remote log servers:  <QRADAR IP>:<PORT>.  For example, if QRadar is 172.31.250.10, then you type in 172.31.250.50:514.  Most cases will use the default 514 port.
  • Remote SYSLOG Contents.  Here I selected everything,  however, this setting allows you the option of being more granular as to what information is sent to QRadar.

3. Once you have all the settings you need, click save.

Enable DNS Query Events in pfSense:

By default, pfSense does not log all DNS Queries.  I had tried enabling this logging in the unbound log settings but did not work.  In order to enable the logging of DNS queries you will need to configure it in the custom options:

  1. In the pfSense menu, go to Services -> DNS Resolver.
  2. Scroll down and click on the Display Custom Options button.
  3. In the Custom Options text box, type in the following:log-queries: yes

Edit:  A reader pointed out that the correct unbound custom options should also contain the line “server:” above the “log-queries: yes” line.  If not, unbound will not send the DNS logs via syslog.  Thank you for the heads up!

Creating the pfSense Log Source in QRadar

Finally, we need to configure the log source in QRadar.

  1. In the QRadar Admin Page, Click Log Sources.
  2. Click Add
  3. Use the following parameters:

Log Source Name : Type in a name for the log source
Log Source Description : Type in a descryption
Log Source Type : pfSense
Protocol Configuration : Syslog
Log Source Identifier :  IP of the interface used above.
Enabled: Check
Credibility: 5
Target Event Collector: Leave as is
Coalescing Events: Unchecked
Incoming Payload Encoding: UTF-8
Store Event Payload: Checked
Log Source Language: Leave as is
Log Source Extension: pfSenseCustom_ext

4. Click Save

This concludes the configuration of the pfSense log source.  Once finished, you can go to the log activity tab in QRadar and filter for the pfSense log source.  You should now be able to view pfSense Events.

Note that not all events from pfSense are parsed correctly.  I may not need to add all of the different event types that pfSense generates since I do not use all of them.  However, feel free to create your own and share them with the community.

17 thoughts on “Sending pfSense logs to QRadar”

  1. Hello.

    Great article, it helped me a lot. Did you investigate how to the VPN event mappings? It’s a lot of work and it would help a lot.

    Best regards.

  2. The additional field log source time should be mapped as well. Below are the changes that can map this. It will parse the correct time into QRadar for the logs.

    Log Source Time:
    Expression Type: Regex
    Expression: (\w+)\s+(\d+)\s+([\d:]+)\s
    Format String: 2018-$1-$2 $3
    Date Format: yyyy-MMM-dd hh:mm:ss

  3. How did you manage to send logs from snort to qradar ? I got qrce on vm and snort on win 10 and I have no clue how to do it , the manual on there web site( ibm )refer to Linux distribution , . Can you tell me the steps plz.

  4. Hello,

    I am also trying to test QRadar and am looking to use PFSense. Do you have any additional info on to get Snort and softflowd (Netflow) in QRadar?

    1. Sounds great! I have been a subscriber for your YouTube channel for a while and used your videos for my original installation of QRadar CE. Looking forward to your DSM video.

      1. Carlos, What is your last name as I would like to mention you by first and Last name in the video.
        Thanks

    2. Hi Carlos,

      I’m the guy collaborating with Jose on the QRadar+pfSense video series.

      I would like to share with you that, based on my tests on pfSense 2.4.4, in order to enable DNS Query Events, you would need to add a “server:” clause to the DNS Resolver Custom options, otherwise unbound will send this error:

      —– pfSense unbound config error message —-BEGIN

      The following input errors were detected:

      The generated config file cannot be parsed by unbound. Please correct the following errors:
      /var/unbound/test/unbound.conf:104: error: syntax error
      read /var/unbound/test/unbound.conf failed: 1 errors in configuration file

      —– pfSense error message —-END

      So, the correct configuration would be adding these two lines:

      server:
      log-queries: yes

      I just tested it on my home network and now my QRadarCE is parsing all the DNS Queries properly.

      Hope it helps.

      Saludos!

  5. In order to export qid map you need to use: /opt/qradar/bin/contentManagement.pl, but you will have to create new qid record for every entry mapping to make it custom…. because this script exports only custom content.

  6. Hi Carlos,

    Thanks for the guide. It has been very useful for the videos that Jose Bravo and I have been working on.

    Currently, I’m facing an issue with the unbound DNS queries logs, apparently on the latest pfSense firmware 2.4.4_3 the custom option “log-queries: yes” is not working. I just tested it and I received this error message:
    ——–
    The following input errors were detected:

    The generated config file cannot be parsed by unbound. Please correct the following errors:
    /var/unbound/test/unbound.conf:104: error: syntax error
    read /var/unbound/test/unbound.conf failed: 1 errors in configuration file
    ——–

    The only way I have been able to log the DNS queries is by increasing the verbosity to level 2 from “Services -> DNS Server -> Advanced Settings -> Log Level -> Level 2: Detailed operational information”. However, the parsing for those queries is not working as expected, all unbound events are idenfied as PfSenseCustom Message

    Have you experienced the same on the latest version of PfSense?

    Thanks in advance.

    1. Hi Polo, Thank you for your comment. Sorry for my late response. You are correct, you also need to add “server:” before the “log queries: yes” line. I completely missed this and have added an update in the article.

Leave a Reply

Your email address will not be published. Required fields are marked *