Atomicorp WAF Rules Troubleshooting

High memory usage

If apache is growing and growing in memory usage, and does not stablize please see the checklist below. If apache is simply using more memory, but has stablized (for example, Apache is using 100MB of memory, as opposed to 40MB of memory without modsecurity) this is normal. modsecurity caches the rules, and certain activities to speed up processing. Memory is infinitely faster than your hard drives, and loading the rules from the hard drive would be terrifyingly slow, too slow to be usable.

Ensure that modsecurity is setup correctly

An incorrectly configured modsecurity setup can cause this to happen. Some vendors that install modsecurity with their control panels do so incorrectly, and configure it in a manner that will lead to memory leaks with advanced security rules. The correct solution is to setup modsecurity correctly. There are two ways to do this:

Solution 1) Install ASL.

This will setup modsecurity correctly, and automatically.

Solution 2) Manually configure and setup modsecurity yourself.

Documentation for setting up modsecurity is available on the Atomic ModSecurity Rules page.

Ensure that modsecurity is built correctly

Solution 1) Install ASL

ASL will setup a correctly built version of modsecurity.

Solution 2) Install a modsecurity build from our rpm repositories.

You can download modsecurity here.

Solution 3) Install asl-lite

Please see the ASL Lite page for details. ASL Lite is unsupported software.

Check for leaking web applications

Web applications can and do leak memory. Check to make sure your memory leak isnt from a web application. Or a web application isnt amplifying a memory leak in something else.

Ensure that apache is built correctly

The major OS vendors, Redhat, Suse, Ubuntu and others do an excellent job of building Apache correctly. However, all software has bugs, and you should make sure that you are both running the latest updates from your vendor for a currently supported version of operating system from that vendor, and that you have ruled out an actual bug in apache. modsecurity is a complex piece of software, that relies of an even more complex piece of software, Apache, to work correctly. To improve performance, modsecurity will use apaches built in memory caching capabilities, and if there is a bug in these Apache supplied capabilities memory leaks can occur. We always recommend you discuss Apache memory leaks with your OS vendor first.

Please be especially diligent with source builds and Apache builds form non OS vendors. These builds are generally not as well tested, and so far are the only ones we have seen with memory management problems. If you use an Apache build from a non-OS vendor, you should discuss any memory management issues with them first if you have rules out the previous causes, as they are most likely a byproduct of a build error, or a bug in Apache.

Site loads slowly when the rules are loaded

There can be a number of reasons for this. Below is a list of the more common issues:

1. DNS based rules are activated, but there is no local DNS server on the system

Check to see if you have any DNS based rules loaded. Please see the mod_security page for a list of rules and which ones are DNS based.

These rules use DNS lookups to check for various characteristics of the connecting system, for example is it on a blacklist, is the IP address known to be used by a trusted source (Google for example). This means that for every connection to your system, a DNS lookup is done against the IP to see if the source IP is listed in some zone or blacklist. If you do not have a local DNS server, this can take a very long time depending on how busy the DNS server is that you are communicating with, and because the request is not cached, your system is asking the DNS server the same question several times in a row. This is because of the way HTTP works. A typical user request may need to open 100 or more connections to your site, and each time a DNS request is made to the RBL.

The simple fast solution is to make sure you are running a local DNS server on your system and that it caches requests. That way your RBL lookups are only to your local system, and if it already has the answer it will not have to ask the RBLs DNS server across the Internet. The difference in performance is night and day with a local DNS server, so we highly recommend you use on if you are going to use any RBL with any product.

2. The system is low on resources

As with any piece of software, the WAF will consume some resources on the system. If the system is short on resources the WAF has to compete with other parts of the system for those resources. Check to make sure you do not have any process that is using up RAM or cycles on the system.

3. Custom rules may be inefficient

If you are using custom rules, try disabling them. If a rule is written in an inefficient manner it can kill performance on the system.

4. Rules are loaded twice

If you have setup modsecurity yourself, check to make sure you arent loading the rules more than once. One several cpanel installations we've seen cases where users inadvertently setup modsecurity to load the same rules twice, and in one case a user set them up to load three times. Apache is a literal animal, if you load the same rule twice it will process that same rule twice, doubling the work on the system! And because our rules use advanced branching logic, if you load them multiple times that can have a very adverse effect on performance as the branching tree logic won't work correctly and you lose our on all the performance benefits a modern ruleset provides.

5. Suboptimal custom configuration

If you setup modsecurity yourself, your configuration could be suboptimal. You may have serial logging enabled, debugging enabled, pour inspection options selected, or any number of things could be wrong. We recommend you use our ASL product to setup modsecurity, its automatic and it will always configure the optimal settings for your system.

If you prefer to do it yourself, then please make sure you have read the modsecurity documentation and understand the options for modsecurity before you enable them. modsecurity is a powerful piece of software that can easily be misconfigured. Care should be taken when setting up this software yourself.

For do it yourselfers, we have put together a guide to help you on the Atomic ModSecurity Rules page.