Difference between revisions of "Reporting False Positives"

Revision as of 08:41, 27 August 2012

Thank you for taking the time to report a false positive or false negative to us. We appreciate that you want to have this resolved as quickly as possible, and be following the process below, and providing us with the information requested we can normally resolve and release an update within a few hours of you report. Real time and ASL customers can expect to see the update the same day they report this issue, and delayed users will have access to the update within approximately 90 days. If you use the delayed rules, and you require a fix earlier, please upgrade to the real time rules for same day service.

General Questions and Answers

False Positives/Negatives with Atomic Secured Linux (ASL)

If you are an ASL customer, our goal is to release an update the same day you report the issue. So please report any concerns, bugs or feature requests you may have to us. We're here to help!

False Positives/Negatives in ASL-Lite and the Real Time Rules

If you are an ASL-Lite or real time rules customer, our goal is to release an update the same day you report the issue. So please report any concerns, bugs or feature requests you may have to us. We're here to help!

False Positives/Negatives in the Delayed Rules

If you are using the delayed rules, and are reporting a false positive or false negative, an update will be available in the real time rules generally the same time you report the issue, and may already be resolved in the real time rules. The update will be reflected in the delayed rules in approximately 90 days when they catch up to the real time rules. If you require a fix earlier, please upgrade to the real time rules.

WAF/Modsecurity rules False Positives

Reporting False Positives when Running Atomic Secured Linux (ASL)

If you have purchased Atomic Secured Linux (ASL) then follow this procedure. If you are not running ASL then please use the procedure below this.

If you report a false positive we generally release a fix for the issue the same business day. If you aren't sure if something ASL has detected is malicious or may just be a false positive, report it to us. Consider us your Security Gurus - we'll figure it out for you, and let you know.

Please make sure you are running the most up to date version of the real time rules before reporting a false positive. We publish updates several times a day, and its possible your issue may have already been resolved.

If ASL blocks something it shouldn't you can report a False Positive to our support team by simply clicking the "False Positive" button in the GUI. You will find this button on the right side in the event details windows. You can pull up an event details window by clicking on the the specific event you wish to view in the main ASL GUI screen.

This will open up a case in the support portal, and if you have setup a support portal account your False Positive will be added to your account for review. If you have ASL configured to send alerts to one of the email addresses associated with your account then you will see your False Positives show up in real time in the support portal. If not, then a member of our support team will have to manually associate your reports with your account and this can take some time.

If you can not use the GUI to report a false positive or you are a rules only member (and therefore do not have the ASL GUI), you can report false positives from the command line. For example, if you have an event like this in your /var/log/httpd/audit_log file:

[modsecurity] [client 1.2.3.4] [domain yourdomain.com] [403] [/20091115/20091115-1635/20091115-163542-rM-wwlKl8i4AACHwQ70AAAAa] [file "/etc/httpd/modsecurity.d/10_asl_rules.conf"] [line "224"] [id "340026"] [rev "49"] [msg "Atomicorp.com WAF Rules: PHP Injection attempt in URI"] [data ""] [severity "CRITICAL"] Access denied with code 403 (phase 2). Match of "beginsWith http://%{SERVER_NAME}/" against "MATCHED_VAR" required.

The fourth variable, highlighted above as [/20091115/20091115-1635/20091115-163542-rM-wwlKl8i4AACHwQ70AAAAa], is the unique token for the event. If you have ASL installed, you can report it with this command:

asl --report-false-positive /20091115/20091115-1635/20091115-163542-rM-wwlKl8i4AACHwQ70AAAAa

This method uses e-mail to send the false positive, so make sure your system can send email out and check your mail logs to ensure that email is being delivered to our servers. ASL does not control or manage email on your system.

Reporting False Negatives when Running ASL

If you discover an attack or method that is not detected or prevented by ASL, we want to know. Please provide as much information as possible about how the attack or method works, and if you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please provide the following information for any False Negatives:

1. Your account name
2. Access logs for the attack or method
3. Location of the logs (For example, /var/log/something/log_file)
4. Your active response log file /var/ossec/logs/active-responses.log
5. Your OSSEC logs, tar up the entire contents of this directory (this may be a large file): /var/ossec/logs/alerts/ (You may need to provide a URL for this file if it is particularly large)
6. If this was the result of a vulnerability scan, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.
7. If this was the result of a vulnerability scan, we will need the vulnerability scan report.
8. If this involves a malicious file, please provide any uploaded or malicious software in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.
9. How it was discovered (uploaded to system, found on desktop, log analysis etc.)

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

Reporting False Positives when not running ASL

If you are running the Atomic ModSecurity Rules and you are not running ASL then you will need to report a false positive manually. modsecurity does not come with a simple tool for reporting false positives like ASL does. ASL will do all of this for you, so if you are running ASL you do not need to follow this procedure, just click on the Report False Positive in the ASL GUI and you are all set.

1. Make sure you are running the latest rules first

Please make sure you are running the most up to date version of the real time rules before reporting a false positive. We publish updates several times a day, and its possible your issue may have already been resolved.

2. Make sure modsecurity is setup correctly

If you are not running ASL, first you must ensure that you have setup modsecurity properly to log events in enough detail that we can help you. Please ensure that you have modsecurity configured according to the instructions on our Atomic ModSecurity Rules page.

3. Get the audit_log event ID for this event

Note: The Apache error_log is not your audit_log, and apache error log events do not contain the information needed to diagnose a false positive. Please follow this process to send us your audit_log entry for this event.

If you followed the instructions on the Atomic ModSecurity Rules page you should have audit logging setup correctly and will have a directory called:

 /var/asl/data/audit

If you do not have this directory stop here and go back to step 2. Make sure you have modsecurity setup according to the instructions on the Atomic ModSecurity Rules page.

This directory will create a special directory for each event that is blocked on your system by first creating a unique directory for that day. The format of the directory is YEARMONTHDATE. For example, if an event occurred on November 25th, 2009, there would a directory of the following format:

 /var/asl/data/audit/20091125

Within that directory will be subdirectories that correspond to events that happen at specific time. The format is YEARMONTHDAY-HOURMINUTE. So if an event occured at 17:23 hours on November 25th, 2009, the corresponding directory would be:

 /var/asl/data/audit/20091125/20091125-1723

Inside this directory will be the audit payloads of all the events that occured in that time period. If many events occured within that minute there will be multiple files. The /var/log/http/audit_log file will contain a running log of the events so you can look up the directories they are stored in. For example this audit_log entry:

[modsecurity] [client 1.2.3.4] [domain example.com] [403] [/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK] [file "/etc/httpd/modsecurity.d/10_asl_antimalware.conf"] [line "42"] [id "360000"] [rev "2"] [msg "Atomicorp.com Malware Blacklist: Malware Site detected in Argument (AE)"] [data "example.com/"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Matched phrase "example.com/" at ARGS:sourcedir.

The audit_log event ID for this event is highlighted above, and is:

/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK

You will need this ID to get the actual audit_log event payload, which will actually include the whole event, its headers, payload and so on. This is the information that is needed to diagnose and fix a false positive.

4. Get the audit_log entry for this event

The payload for this event can be found by appending the event ID, in the example above "/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK" to the path "/var/asl/data/audit". For example:

 cat    /var/asl/data/audit/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK

This command, when run as root, will print out the actual payload of the attack. In this case the example attack looks like this:

--907db756-A--
[25/Nov/2009:17:23:10 --0500] gPA9rUrQYacAACJpX@UAAAAK 1.2.3.4 55732 74.208.97.167 80
--907db756-B--
GET /delayed/rules/modsec/?sourcedir=http://example.com/mambots/idxx.txt?%20? HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: example.com
User-Agent: Mozilla/5.0

--907db756-F--
HTTP/1.1 403 Forbidden
Content-Length: 303
Connection: close
Content-Type: text/html; charset=iso-8859-1

--907db756-H--
Message:  [file "/etc/httpd/modsecurity.d/10_asl_antimalware.conf"] [line "42"] [id "360000"] [rev "2"] [msg "Atomicorp.com Malware Blacklist: Malware Site detected in Argument (AE)"] [data "example.com/"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Matched phrase "example.com/" at ARGS:sourcedir.
Action: Intercepted (phase 2)
Stopwatch: 1259187790167469 27763 (4424 27390 -)
Producer: ModSecurity for Apache/2.5.11 (http://www.modsecurity.org/); 200911221938.
Server: Apache/2.2.3 (CentOS)

--907db756-Z--

And now you have the information from the event that we need to diagnose your issue.

5. Report the false positive

To report this as a false positive, you will need to include:

1. Your account name (if this is for the real time rules)
2. The version of the rules you are running
3. The version of mod_security you are running (not the version of the rules)
4. The webserver you are running and the version (e.g. Apache 2.1.23)
5. The output of the audit log entry for this event (in the example above /var/asl/data/audit/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAA or whatever it is for your system) - This is critical, without this information we can not help you. Please note that apache error_log messages are not audit log entries, and the apache error log messages do not contain the information needed to determine the cause of false positive. Please do not send apache error_log entries, they are of no use to us.
6. Source of the mod_security module (Atomicorp RPM, built from source, third party, etc.)
7. The platform you are running on (Operating system, distribution and apache version).
8. System memory and CPU

The apache error_log file is of no use to us, so please do not send it. It does not contain any of the information we need to investigate a false positive.

Then e-mail this to support@atomicorp.com. Our CRM will automatically open a case for you to track the incident.

If you are RealTime feed customer, you are encouraged to open a support portal account.

Reporting False Negatives when not running ASL

If you discover an attack or method that is not detected or prevented by ASL, we want to know. Please provide as much information as possible about how the attack or method works, and if you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please provide the following information for any False Negatives:

1. Your account name
2. Web access logs for the attack or method
3. If this was the result of a vulnerability scan, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.
4. If this was the result of a vulnerability scan, we will need the vulnerability scan report.
5. Any uploaded or malicious software in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.
6. How it discovered (uploaded to system, found on desktop, etc.)

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

CLAMAV False Positives/Negatives

To report a false positive

Please send us the following information with your false positive report:

1. Your account name
2. The name of the signature that was triggered
3. How it was triggered (uploaded via FTP/Web, blocked locally via dazuko, etc.)
4. The specific error message(s) your application produced when the file was rejected.
5. The log entry in /var/log/clamav/clamd.log for this event.
6. And the file that was incorrect detected in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.

To report the false positive, please open either a case in the support portal or email us the false positive report to support AT atomicorp DOT com.

To report a false negative

Please send us the following information with your false negative report:

1. Your account name
2. The name of the malware as detected by other scanners (if this information exists, if this is a zero day, please state that)
3. How it discovered (uploaded to system, found on desktop, etc.)
4. And the malicious software in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

Atomic Secured Linux (ASL) HIPS/KIPS/WIPS False Positives/Negatives

To report a false positive

Just click on the "False Positive" button in the ASL GUI for that event. That will send us all the information we need.

To report a false negative

If you discover an attack or method that is not detected or prevented by ASL, we want to know. Please provide as much information as possible about how the attack or method works, and if you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please send us the following information with your false negative report:

1. Your account name
2. The version of ASL you have installed, and the output of the commands "asl -ck" and "asl -s".
3. Full logs for the attack or method
4. If this was the result of a vulnerability scan, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.
5. If this was the result of a vulnerability scan, we will need the vulnerability scan report.
6. How it discovered (uploaded to system, found on desktop, etc.)

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

Vulnerability Scanner False Positives/Negatives

To report a false positive

Please send us the following information with your false positive report:

1. Your account name
2. The version of ASL you have installed
3. Please cut and paste the ASL scanner alert and any supporting configuration files that show the vulnerability does not exist.

To report the false positive, please open either a case in the support portal or email us the false positive report to support AT atomicorp DOT com.

To report a false negative

If you discover a vulnerability is not detected or prevented by ASL, we want to know. Please provide as much information as possible about the vulnerability - for example if you used a tool to find, please let us know what that tool is and how you used it. If you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please send us the following information with your false negative report:

1. Your account name
2. CVE reference
3. As many details as you can provide of the vulnerability, such as links to the publication of the vulnerability such as vendor and security researcher reports.
4. If this was the result of a vulnerability scan with another tool, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.
5. If this was the result of a vulnerability scan, we will need the vulnerability scan report.

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.