Difference between revisions of "Reporting False Positives"

Revision as of 18:05, 21 January 2011

Thank you for taking the time to report a false positive or false negative to us. We appreciate that you want to have this resolved as quickly as possible, and with the information described below we can normally resolve and release an update within a few hours for our ASL and real time feed customers. Delayed users can expect to see the update in approximately 90 days when the delayed rules catch up with the real time rules. If you use the delayed rules, and you require a fix earlier, please upgrade to the real time rules.

General Questions and Answers

False Positives/Negatives with Atomic Secured Linux (ASL)

If you are an ASL customer, our goal is to release an update the same day you report the issue. So please report any concerns, bugs or feature requests you may have to us. We're here to help!

False Positives/Negatives in ASL-Lite and the Real Time Rules

If you are an ASL-Lite or real time rules customer, our goal is to release an update the same day you report the issue. So please report any concerns, bugs or feature requests you may have to us. We're here to help!

False Positives/Negatives in the Delayed Rules

If you are using the delayed rules, and are reporting a false positive or false negative, an update will be available in the real time rules generally the same time you report the issue, and may already be resolved in the real time rules. The update will be reflected in the delayed rules in approximately 90 days when they catch up to the real time rules. If you require a fix earlier, please upgrade to the real time rules.

WAF/Modsecurity rules False Positives

Reporting False Positives when Running Atomic Secured Linux (ASL)

If you have purchased Atomic Secured Linux (ASL) then follow this procedure. If you are not running ASL then please use the procedure below this.

If you report a false positive we generally release a fix for the issue the same business day. If you aren't sure if something ASL has detected is malicious or may just be a false positive, report it to us. Consider us your Security Gurus - we'll figure it out for you, and let you know.

Please make sure you are running the most up to date version of the real time rules before reporting a false positive. We publish updates several times a day, and its possible your issue may have already been resolved.

If ASL blocks something it shouldn't you can report a False Positive to our support team by simply clicking the "False Positive" button in the GUI. You will find this button on the right side in the event details windows. You can pull up an event details window by clicking on the the specific event you wish to view in the main ASL GUI screen.

This will open up a case in the support portal, and if you have setup a support portal account your False Positive will be added to your account for review. If you have ASL configured to send alerts to one of the email addresses associated with your account then you will see your False Positives show up in real time in the support portal. If not, then a member of our support team will have to manually associate your reports with your account and this can take some time.

If you can not use the GUI to report a false positive or you are a rules only member (and therefore do not have the ASL GUI), you can report false positives from the command line. For example, if you have an event like this in your /var/log/httpd/audit_log file:

[modsecurity] [client 1.2.3.4] [domain yourdomain.com] [403] [/20091115/20091115-1635/20091115-163542-rM-wwlKl8i4AACHwQ70AAAAa] [file "/etc/httpd/modsecurity.d/10_asl_rules.conf"] [line "224"] [id "340026"] [rev "49"] [msg "Atomicorp.com WAF Rules: PHP Injection attempt in URI"] [data ""] [severity "CRITICAL"] Access denied with code 403 (phase 2). Match of "beginsWith http://%{SERVER_NAME}/" against "MATCHED_VAR" required.

The fourth variable, highlighted above as [/20091115/20091115-1635/20091115-163542-rM-wwlKl8i4AACHwQ70AAAAa], is the unique token for the event. If you have ASL installed, you can report it with this command:

asl --report-false-positive /20091115/20091115-1635/20091115-163542-rM-wwlKl8i4AACHwQ70AAAAa

This method uses e-mail to send the false positive, so make sure your system can send email out and check your mail logs to ensure that email is being delivered to our servers. ASL does not control or manage email on your system.

Reporting False Negatives when Running ASL

If you discover an attack or method that is not detected or prevented by ASL, we want to know. Please provide as much information as possible about how the attack or method works, and if you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please provide the following information for any False Negatives:

1. Your account name
2. Web access logs for the attack or method
3. If this was the result of a vulnerability scan, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.
4. If this was the result of a vulnerability scan, we will need the vulnerability scan report.
5. Any uploaded or malicious software in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.
6. How it discovered (uploaded to system, found on desktop, etc.)

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

Reporting False Positives when not running ASL

If you are running the Atomic ModSecurity Rules and you are not running ASL then you will need to report a false positive manually. modsecurity does not come with a simple tool for reporting false positives like ASL does. ASL will do all of this for you, so if you are running ASL you do not need to follow this procedure, just click on the Report False Positive in the ASL GUI and you are all set.

Please make sure you are running the most up to date version of the real time rules before reporting a false positive. We publish updates several times a day, and its possible your issue may have already been resolved.

If you are not running ASL, first you must ensure that you have setup modsecurity properly to log events in enough detail that we can help you. If you followed the instructions on the Atomic ModSecurity Rules page you should have audit logging setup correctly and will have a directory called:

 /var/asl/data/audit

This directory will create a special directory for each event that is blocked on your system by first creating a unique directory for today. The format of the directory is YEARMONTHDATE. For example, if an event occurred on November 25th, 2009, there would a directory of the following format:

 /var/asl/data/audit/20091125

Within that directory will be subdirectories that correspond to events that happen at specific time. The format is YEARMONTHDAY-HOURMINUTE. So if an event occured at 17:23 hours on November 25th, 2009, the corresponding directory would be:

 /var/asl/data/audit/20091125/20091125-1723

Inside this directory will be the audit payloads of all the events that occured in that time period. If many events occured within that minute there will be multiple files. The /var/log/http/audit_log file will contain a running log of the events so you can look up the directories they are stored in. For example this audit_log entry:

[modsecurity] [client 1.2.3.4] [domain example.com] [403] [/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK] [file "/etc/httpd/modsecurity.d/10_asl_antimalware.conf"] [line "42"] [id "360000"] [rev "2"] [msg "Atomicorp.com Malware Blacklist: Malware Site detected in Argument (AE)"] [data "example.com/"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Matched phrase "example.com/" at ARGS:sourcedir.

The payload for this event can be found by looking at the highlighted line above, and appending /var/asl/data/audit. For example:

 cat    /var/asl/data/audit/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAAK

Would produce the actual payload of the attack. In this case the example attack looks like this:

--907db756-A--
[25/Nov/2009:17:23:10 --0500] gPA9rUrQYacAACJpX@UAAAAK 1.2.3.4 55732 74.208.97.167 80
--907db756-B--
GET /delayed/rules/modsec/?sourcedir=http://example.com/mambots/idxx.txt?%20? HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: example.com
User-Agent: Mozilla/5.0

--907db756-F--
HTTP/1.1 403 Forbidden
Content-Length: 303
Connection: close
Content-Type: text/html; charset=iso-8859-1

--907db756-H--
Message:  [file "/etc/httpd/modsecurity.d/10_asl_antimalware.conf"] [line "42"] [id "360000"] [rev "2"] [msg "Atomicorp.com Malware Blacklist: Malware Site detected in Argument (AE)"] [data "example.com/"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Matched phrase "example.com/" at ARGS:sourcedir.
Action: Intercepted (phase 2)
Stopwatch: 1259187790167469 27763 (4424 27390 -)
Producer: ModSecurity for Apache/2.5.11 (http://www.modsecurity.org/); 200911221938.
Server: Apache/2.2.3 (CentOS)

--907db756-Z--

To report this as a false positive, you will need to include:

1. Your account name (if this is for the real time rules)
2. The version of the rules you are running
3. The version of mod_security you are running
4. Source of the mod_security module (Atomicorp RPM, built from source, third party, etc.)
5. The platform you are running on (Operating system, distribution and apache version).
6. System memory and CPU
7. The output of /var/asl/data/audit/20091125/20091125-1723/20091125-172310-gPA9rUrQYacAACJpX@UAAAA (or whatever it is for your system)

Then e-mail this to support@atomicorp.com. Our CRM will automatically open a case for you to track the incident. The apache error_log file is of no use to us, so please do not send it. It does not contain any of the information we need to investigate a false positive.

If you are RealTime feed customer, you are encouraged to open a support portal account.

Reporting False Negatives when not running ASL

If you discover an attack or method that is not detected or prevented by ASL, we want to know. Please provide as much information as possible about how the attack or method works, and if you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please provide the following information for any False Negatives:

1. Your account name
2. Web access logs for the attack or method
3. If this was the result of a vulnerability scan, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.
4. If this was the result of a vulnerability scan, we will need the vulnerability scan report.
5. Any uploaded or malicious software in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.
6. How it discovered (uploaded to system, found on desktop, etc.)

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

CLAMAV False Positives/Negatives

To report a false positive

Please send us the following information with your false positive report:

1. Your account name
2. The name of the signature that was triggered
3. How it was triggered (uploaded via FTP/Web, blocked locally via dazuko, etc.)
4. The specific error message(s) your application produced when the file was rejected.
5. The log entry in /var/log/clamav/clamd.log for this event.
6. And the file that was incorrect detected in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.

To report the false positive, please open either a case in the support portal or email us the false positive report to support AT atomicorp DOT com.

To report a false negative

Please send us the following information with your false negative report:

1. Your account name
2. The name of the malware as detected by other scanners (if this information exists, if this is a zero day, please state that)
3. How it discovered (uploaded to system, found on desktop, etc.)
4. And the malicious software in a zip, tar, rar or bz archive format. Please put a password on the archive to ensure that it is not rejected by any email antivirus scanners in route to us.

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

ASL HIPS/KIPS/WIPS False Positives/Negatives

To report a false positive

Please send us the following information with your false positive report:

1. Your account name
2. The version of ASL you have installed
3. Please cut and paste the whole alert you received from ASL
4. Please provide the full log entry if the ASL alert truncated the log event (large log events are truncated to prevent DOS attacks and other undesirable behaviour)
5. The Rule ID and payload/syslog message

To report the false positive, please open either a case in the support portal or email us the false positive report to support AT atomicorp DOT com.

To report a false negative

If you discover an attack or method that is not detected or prevented by ASL, we want to know. Please provide as much information as possible about how the attack or method works, and if you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please send us the following information with your false negative report:

1. Your account name
2. The version of ASL you have installed, and the output of the commands "asl -ck" and "asl -s".
3. Full logs for the attack or method
4. If this was the result of a vulnerability scan, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.
5. If this was the result of a vulnerability scan, we will need the vulnerability scan report.
6. How it discovered (uploaded to system, found on desktop, etc.)

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.

Vulnerability Scanner False Positives/Negatives

To report a false positive

Please send us the following information with your false positive report:

1. Your account name
2. The version of ASL you have installed
3. Please cut and paste the ASL scanner alert and any supporting configuration files that show the vulnerability does not exist.

To report the false positive, please open either a case in the support portal or email us the false positive report to support AT atomicorp DOT com.

To report a false negative

If you discover a vulnerability is not detected or prevented by ASL, we want to know. Please provide as much information as possible about the vulnerability - for example if you used a tool to find, please let us know what that tool is and how you used it. If you can not do this we would ask that you provide us with access to your system. The process for providing us access to your system is documented here.

Please send us the following information with your false negative report:

1. Your account name
2. CVE reference
3. As many details as you can provide of the vulnerability, such as links to the publication of the vulnerability such as vendor and security researcher reports.
4. If this was the result of a vulnerability scan with another tool, please confirm that this is not a false positive of the scanner first. Many PCI-DSS scanners are prone to false positives, ensure that this is not the case before reporting it as a false negative.
5. If this was the result of a vulnerability scan, we will need the vulnerability scan report.

To report the false negative, please open either a case in the support portal or email us the false negative report to support AT atomicorp DOT com.