Atomicorp WAF Rules Troubleshooting

Specific Errors

Invalid command 'SecRemoteRulesFailAction', perhaps misspelled or defined by a module not included in the server configuration

This means that your system is using an out of date version of modsecurity that does not support 2.9.1 and higher rule syntax.

Solutions:

1) Upgrade to ASL. ASL will automatically ensure you have the latest version of modsecurity installed on your system.

2) Use aum, which will automatically ensure you have the latest version of modsecurity installed on your system.

3) use our rpms which are built using 2.9.1, and support remote rules.

4) If you do not wish to upgrade to ASL, use aum, or use our rpms then you will need to manually upgrade modsecurity to at least version 2.9.1, and enable remote rules support when you compile modsecurity.

Invalid command 'SecTmpSaveUploadedFiles'

This means that your system is using an out of date version of modsecurity that does not support 2.9.0 and higher rule syntax.

Solutions:

1) Upgrade to ASL. ASL will automatically ensure you have the latest version of modsecurity installed on your system.

2) Use aum, which will automatically ensure you have the latest version of modsecurity installed on your system.

3) If you do not wish to upgrade to ASL, or use aum and can not upgrade to 2.9.0, you will need to remove the ruleset(s) that are generating this error.

Error creating rule: Unknown variable: FILES_TMP_CONTENT

This means that your system is using an out of date version of modsecurity that does not support 2.9.0 and higher rule syntax.

Solutions:

1) Upgrade to ASL. ASL will automatically ensure you have the latest version of modsecurity installed on your system.

2) If you do not wish to upgrade to ASL, and can not upgrade to 2.9.0, you will need to remove the ruleset(s) that are generating this error.

Error creating rule: Failed to resolve operator: fuzzyHash

This means that your system is an out of date version of modsecurity that does not support 2.9.0 and higher rule syntax.

Solutions:

1) Upgrade to ASL. ASL will automatically ensure you have the latest version of modsecurity installed on your system.

2) If you do not wish to upgrade to ASL, and can not upgrade to 2.9.0, you will need to remove the ruleset(s) that are generating this error.

Error creating rule: Could not open phrase file "/etc/asl/custom-domain-blocks": No such file or directory

This means that you have not setup this ruleset correctly. Please see the link below for installation instructions:

https://www.atomicorp.com/wiki/index.php/WAF_rule_families#01_asl_domain_blocks.conf

SecReadStateLimit is depricated, use SecConnReadStateLimit instead.

This means your system is using the very buggy 2.8.0 modsecurity release. Do not use 2.8.0. Please see the article below for more information on 2.8.0.

If you must use 2.8.0, this error is benign and you can ignore it.

Error creating rule: Could not add entry

See below if you are using mod_security 2.8.0.

ModSecurity: IPmatch: bad IPv4 specification

This error is not caused by the rule. It is caused by a buggy version of modsecurity, specifically 2.8.0. 2.8.0 has a known critical bug in its IP matching code. It can not properly use IP addresses with rules, as documented in the github bug report below:

https://github.com/SpiderLabs/ModSecurity/issues/706

Do not use 2.8.0. Users should only use the currently supported version of modsecurity documented at the URL below:

https://www.atomicorp.com/wiki/index.php/Atomic_ModSecurity_Rules#Minimum_Version_of_Modsecurity_Required_to_use_the_rules

Solutions

cpanel

Step 1) Remove 2.8.0 from your system

Disable modsecurity in cpanel and uninstall it (dont worry, the next step will show you how to install a bug free version)

Step 2) Install aum on your system

aum will install modsecurity and keep both modsecurity and your rules up to date, automatically, with stable bug free versions that will never conflict with our rules. Please see the URL below for installation instructions for aum:

https://www.atomicorp.com/wiki/index.php/Downloading_Rules#Just_a_downloader

Note: If you are using an older modsecurity management tool that does not support the faster concurrent logging method, you need to manually configure modsecurity to use the slower serial method as documented at the URL below:

https://www.atomicorp.com/wiki/index.php/Atomic_ModSecurity_Rules#Serial_logging

Plesk

Step 1) Disable buggy modsecurity from the plesk interface under server settings

Note: Run these commands in the steps below as the root user.

Step 2) Remove the buggy modsecurity packages

yum remove plesk-modsecurity*

Step 3) Change the rules path to the correct systemwide path in /etc/asl/config

MODSEC_RULES_PATH="/etc/httpd/modsecurity.d"

Step 4) Make a directory for the rules

mkdir /etc/httpd/modsecurity.d

Step 5) Run the Atomicorp Update Manager

aum -u

Custom

If you have installed modsecurity either from source, or via some other third party method remove 2.8.0 and use 2.7.7 or 2.9.0. This bug is not fixed in 2.8.0 or in subversion.

httpd: ModSecurity: WARNING Using transformations in SecDefaultAction is deprecated

This means that you are using out of date rules. If you are using Atomicorp rules, then this means you are not using the latest real time rules. The latest real time rules do not use transformations in the SecDefaultAction.

If you are using third party rules, this means that the rules contain a transform function in the SecDefaultAction setting. This is a deprecated setting.

ModSecurity: Failed to access DBM file "/var/asl/data/msa/

This means that you do not have the correct permissions setup for modsecurity to work correctly. Please make sure you have followed all the setup and installation instructions in the Atomic ModSecurity Rules wiki article. Specific guidance is provided in this section of the Setting Up Modsecurity guide:

https://www.atomicorp.com/wiki/index.php/Atomic_ModSecurity_Rules#Step_3:_Set_permissions_for_directories

Failed to create subdirectories

This means that the permissions on those subdirectories do not match the user that apache is running as. For example, if apache is running as the user "apache" those directories must be owned by apache.

Important note: If you are using an apache module that changes the user that apache runs as, on the fly, then those directories must be writable by all users. The use of modules that require you to set these directories as world writable and world readable is a serious security vulnerability, and should not be done. This will allow all users on the system to both change these logs, as well as to see both the logs and the payloads for all other users on the system.

Some modules, for example mod_ruid2, write logs as the user for that instance, but will set permissions on subdirectories such that other users can not write or read from them.

The most secure option is for these directories to be writable and readable only by a trusted user, and not by general users on the system.

Please follow this guidance at the URL below:

https://wiki.atomicorp.com/wiki/index.php/Atomic_ModSecurity_Rules#Step_3:_Set_permissions_for_directories

Error creating rule: Unknown variable: MATCHED_VARS

Causes:

If you are getting this error, it means that you are using an old version of modsecurity that does not support the modern rule language and you are not running ASL. We recommend you install ASL.

ASL will not allow incompatible versions of the modsecurity rules to be installed. It will also automatically upgrade mod_security provided you have UPDATE_TYPE set to "all", which is the default. If you have updates disabled in ASL, then you will need to manually upgrade modsecurity:

yum upgrade mod_security

If you are not using ASL, then you will need to upgrade modsecurity using whatever method you used to install it. You will need to be running at least version 2.6.3.

Exec: Execution failed while reading output: /usr/bin/modsec-clamscan.pl (End of file found)

Without ASL

This error occurs if you install the 05_asl_scanner.conf file and/or the 99_asl_scanner.conf ruleset. These rulesets are not supported without ASL, as malware scanning is not included in the rules only subscription. ASL comes with malware upload scanning for HTTP, SSH, FTP and other protocols, including real time malware protection and much more. If you want malware upload protection, upgrade to ASL.

Solution:

Option 1. Update to ASL

Option 2: disable this setting in /etc/asl/config

https://www.atomicorp.com/wiki/index.php?title=ASL_WAF#MODSEC_99_SCANNER

Option 3. Remove the file 05_asl_scanner.conf and/or 99_asl_scanner.conf and restart apache.

With ASL

This means someone or something has disabled the malware protection system on your system. ASL can not and will not do this.

/usr/bin/modsec-clamscan.pl is not installed on the server.

Malware scanning is not included in the rules only subscription. ASL comes with malware upload scanning for HTTP, SSH, FTP and other protocols, including real time malware protection and much more. If you want malware upload protection, upgrade to ASL.

We also don't include that file or use the methods demonstrated in it because it doesn't scale very well. ASL has a binary streaming system that scales.

No action id present within the rule

This means that you using out of date rules. If you are using Atomicorp rules, then this means you are not using the latest real time rules. The latest real time rules have an id action for every rule.

httpd: ModSecurity: WARNING Using transformations in SecDefaultAction is deprecated

This means that you are using out of date rules. If you are using Atomicorp rules, then this means you are not using the latest real time rules. The latest real time rules do not use transformations in the SecDefaultAction.

If you are using third party rules, this means that the rules contain a transform function in the SecDefaultAction setting. This is a deprecated setting.

ModSecurity: Found another rule with the same id

This means that you have a modsecurity rule with the same id. All of our rules have unique id's. This error can only occur for one of two reasons:

1) You have loaded our rules twice. Please follow this installation guide and ensure that you have modsecurity configured as described in this document. These installation procedures will only load the rules once.

2) You have rules that are using id's already assigned to other rules.

If you are using third party or custom rules, check to make sure they have unique id's. Modsecurity requires a unique id for each rule. The range 300000-399999 is used by our rules, do not use this range for any custom rules, and if you have third party rules with these id's be sure to remove these rules.

Rule execution error - PCRE limits exceeded (-8): (null).

This is an internal limit to prevent a special type of DOS attack on the WAF itself. This is not caused by any of the rules. This is caused by the content the rules are inspecting. In certain cases, the content may be so complex that the WAF is stopping itself from doing too much work which could lead to a DOS attack on the system itself. If your system is generating these kinds of errors, it means you need to set the limits higher on your system, while it is beyond the scope of this article, another solution is to reduce the complexity of the content you are inspecting.

It is also possible this is occurring due to an actual DOS attack on your system. If you are certain this is not a DOS attack, simply increase these limits accordingly for your system. we recommend a minimum of 250000 for a modern system.

 SecPcreMatchLimit 250000
 SecPcreMatchLimitRecursion 250000

You may have to increase these limits for your system if you continue to get PCRE limit errors.

High memory usage

If apache is growing and growing in memory usage, and does not stablize please see the checklist below. If apache is simply using more memory, but has stablized (for example, Apache is using 100MB of memory, as opposed to 40MB of memory without modsecurity) this is normal. modsecurity caches the rules, and certain activities to speed up processing. Memory is infinitely faster than your hard drives, and loading the rules from the hard drive would be terrifyingly slow, too slow to be usable.

Ensure that modsecurity is setup correctly

An incorrectly configured modsecurity setup can cause this to happen. Some vendors that install modsecurity with their control panels do so incorrectly, and configure it in a manner that will lead to memory leaks with advanced security rules. The correct solution is to setup modsecurity correctly. There are two ways to do this:

Solution 1) Install ASL.

This will setup modsecurity correctly, and automatically.

Solution 2) Manually configure and setup modsecurity yourself.

Documentation for setting up modsecurity is available on the Atomic ModSecurity Rules page.

Ensure that modsecurity is built correctly

Solution 1) Install ASL

ASL will setup a correctly built version of modsecurity.

Solution 2) Install a modsecurity build from our rpm repositories.

You can download modsecurity here.

Solution 3) Install asl-lite

Please see the ASL Lite page for details. ASL Lite is unsupported software.

Check for leaking web applications

Web applications can and do leak memory. Check to make sure your memory leak isnt from a web application. Or a web application isnt amplifying a memory leak in something else.

Ensure that apache is built correctly

The major OS vendors, Redhat, Suse, Ubuntu and others do an excellent job of building Apache correctly. However, all software has bugs, and you should make sure that you are both running the latest updates from your vendor for a currently supported version of operating system from that vendor, and that you have ruled out an actual bug in apache. modsecurity is a complex piece of software, that relies of an even more complex piece of software, Apache, to work correctly. To improve performance, modsecurity will use apaches built in memory caching capabilities, and if there is a bug in these Apache supplied capabilities memory leaks can occur. We always recommend you discuss Apache memory leaks with your OS vendor first.

Please be especially diligent with source builds and Apache builds form non OS vendors. These builds are generally not as well tested, and so far are the only ones we have seen with memory management problems. If you use an Apache build from a non-OS vendor, you should discuss any memory management issues with them first if you have rules out the previous causes, as they are most likely a byproduct of a build error, or a bug in Apache.

Site loads slowly when the rules are loaded

There can be a number of reasons for this. Below is a list of the more common issues:

1. DNS based rules are activated, but there is no local DNS server on the system

Check to see if you have any DNS based rules loaded. Please see the mod_security page for a list of rules and which ones are DNS based.

These rules use DNS lookups to check for various characteristics of the connecting system, for example is it on a blacklist, is the IP address known to be used by a trusted source (Google for example). This means that for every connection to your system, a DNS lookup is done against the IP to see if the source IP is listed in some zone or blacklist. If you do not have a local DNS server, this can take a very long time depending on how busy the DNS server is that you are communicating with, and because the request is not cached, your system is asking the DNS server the same question several times in a row. This is because of the way HTTP works. A typical user request may need to open 100 or more connections to your site, and each time a DNS request is made to the RBL.

The simple fast solution is to make sure you are running a local DNS server on your system and that it caches requests. That way your RBL lookups are only to your local system, and if it already has the answer it will not have to ask the RBLs DNS server across the Internet. The difference in performance is night and day with a local DNS server, so we highly recommend you use on if you are going to use any RBL with any product.

2. The system is low on resources

As with any piece of software, the WAF will consume some resources on the system. If the system is short on resources the WAF has to compete with other parts of the system for those resources. Check to make sure you have free memory and CPU cycles for the WAF.

3. Custom rules may be inefficient

If you are using custom rules, try disabling them. If a rule is written in an inefficient manner it can kill performance on the system.

4. Rules are loaded twice

If you have setup modsecurity yourself, check to make sure you arent loading the rules more than once. One several cpanel installations we've seen cases where users inadvertently setup modsecurity to load the same rules twice, and in one case a user set them up to load three times. Apache is a literal animal, if you load the same rule twice it will process that same rule twice, doubling the work on the system! And because our rules use advanced branching logic, if you load them multiple times that can have a very adverse effect on performance as the branching tree logic won't work correctly and you lose our on all the performance benefits a modern ruleset provides.

5. Suboptimal custom configuration

If you setup modsecurity yourself, your configuration could be suboptimal. You may have serial logging enabled, debugging enabled, pour inspection options selected, or any number of things could be wrong. We recommend you use our ASL product to setup modsecurity, its automatic and it will always configure the optimal settings for your system.

If you prefer to do it yourself, then please make sure you have read the modsecurity documentation and understand the options for modsecurity before you enable them. modsecurity is a powerful piece of software that can easily be misconfigured. Care should be taken when setting up this software yourself.

For do it yourselfers, we have put together a guide to help you on the Atomic ModSecurity Rules page.