https://wiki.atomicorp.com/wiki/index.php?title=Apache&feed=atom&action=historyApache - Revision history2024-03-29T06:20:56ZRevision history for this page on the wikiMediaWiki 1.20.2https://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=4761&oldid=prevMshinn: /* Backtracing */2014-04-10T17:01:21Z<p><span dir="auto"><span class="autocomment">Backtracing</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 17:01, 10 April 2014</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 83:</td>
<td colspan="2" class="diff-lineno">Line 83:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>gdb /path/to/httpd /path/to/core --batch --quiet -ex "thread apply all bt full" > backtrace.log  </div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>gdb /path/to/httpd /path/to/core --batch --quiet -ex "thread apply all bt full" > backtrace.log  </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>And look at the backtrace for the cause.  Most often the cause is a bug in Apache, or one of its supporting libraries.  '''Please make sure you share your backtrace with your Apache vendor first so they can rule out if this an Apache bug.'''  So far, we've only see apache bugs that cause segfaults, so if you want your segfault resolved quickly contact your Apache vendor first.  They can tell you if the issue <del class="diffchange diffchange-inline">if </del>with Apache, or with mod_security.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>And look at the backtrace for the cause.  Most often the cause is a bug in Apache, or one of its supporting libraries.  '''Please make sure you share your backtrace with your Apache vendor first so they can rule out if this an Apache bug.'''  So far, we've only see apache bugs that cause segfaults, so if you want your segfault resolved quickly contact your Apache vendor first.  They can tell you if the issue <ins class="diffchange diffchange-inline">is </ins>with Apache, or with mod_security<ins class="diffchange diffchange-inline">.  If the issue is with Apache, we can't help you fix that</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Keep in mind that memory use changes do not cause segfaults.  So if you have a webapp, for example, that uses a lot of memory and you get segfaults unfortunately the memory use isnt the cause. Its just correlational, not causational.  More memory in use means more opportunities for the fault to occur. It is not the cause, it just increases the probability not the possibility. So when you have more memory in use and you segfault, and less and you don't segfault - the higher use of memory is not the cause of the segfault.  Think of that as a warning that you have something else wrong.  You can rule memory use out.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Keep in mind that memory use changes do not cause segfaults.  So if you have a webapp, for example, that uses a lot of memory and you get segfaults unfortunately the memory use isnt the cause. Its just correlational, not causational.  More memory in use means more opportunities for the fault to occur. It is not the cause, it just increases the probability not the possibility. So when you have more memory in use and you segfault, and less and you don't segfault - the higher use of memory is not the cause of the segfault.  Think of that as a warning that you have something else wrong.  You can rule memory use out.</div></td></tr>
</table>Mshinnhttps://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=4760&oldid=prevMshinn: /* Step 3 */2014-04-10T17:00:22Z<p><span dir="auto"><span class="autocomment">Step 3</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 17:00, 10 April 2014</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 53:</td>
<td colspan="2" class="diff-lineno">Line 53:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Here is an example for a package managed apache install:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Here is an example for a package managed apache install:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>   yum install httpd-debuginfo</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>   yum install httpd-debuginfo  </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>'''If your OS is missing a debuginfo package for the application you want to backtrace, file a bug report with them.'''   </div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>'''If your OS is missing a debuginfo package for the application you want to backtrace, file a bug report with them.'''   </div></td></tr>
</table>Mshinnhttps://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=4758&oldid=prevMshinn: /* Step 3 */2014-04-10T16:53:22Z<p><span dir="auto"><span class="autocomment">Step 3</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 16:53, 10 April 2014</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 49:</td>
<td colspan="2" class="diff-lineno">Line 49:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 3===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 3===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Install the debuginfo packages for the applications you want to backtrace.  This is so your core file will contain the debug information needed to assist you in tracking down the source of the segfault.  For example, if you are backtracing just apache, then you need to install its debuginfo packages.  If you are debugging something else, like PHP, you need to install its debug symbols and so on.  Here is an example for a package managed apache install:</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Install the debuginfo packages for the applications you want to backtrace.  This is so your core file will contain the debug information needed to assist you in tracking down the source of the segfault.  For example, if you are backtracing just apache, then you need to install its debuginfo packages.  If you are debugging something else, like PHP, you need to install its debug symbols and so on.   </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Here is an example for a package managed apache install:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>   yum install httpd-debuginfo</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>   yum install httpd-debuginfo</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>If your OS is missing debuginfo, file a bug report with them.  <del class="diffchange diffchange-inline">Although we do make debuginfo rpms available in the atomic repository for our Apache builds, unless you use our free Atomic rpms you will want to install the correct debuginfo file for your system.</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">'''</ins>If your OS is missing <ins class="diffchange diffchange-inline">a </ins>debuginfo <ins class="diffchange diffchange-inline">package for the application you want to backtrace</ins>, file a bug report with them.<ins class="diffchange diffchange-inline">''' </ins>  </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Note: if you have a source built apache, such as with cpanel, you will need to rebuild apache with debugging symbols.  For cpanel, <del class="diffchange diffchange-inline">you can do </del>the following:</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Note: if you have a source built apache, such as with cpanel, you will need to rebuild apache with debugging symbols.  For cpanel, <ins class="diffchange diffchange-inline"> </ins>the following <ins class="diffchange diffchange-inline">may work, but contact cpanel for assistance</ins>:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>1) set the appropriate CFLAGS</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>1) set the appropriate CFLAGS</div></td></tr>
</table>Mshinnhttps://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=4757&oldid=prevMshinn: /* Step 3 */2014-04-10T16:51:23Z<p><span dir="auto"><span class="autocomment">Step 3</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 16:51, 10 April 2014</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 66:</td>
<td colspan="2" class="diff-lineno">Line 66:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If this does not work for you, please contact your apache vendor for assistance.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If this does not work for you, please contact your apache vendor for assistance.</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">Note:  If you have apache modules loaded, please check to make sure they have the debug symbols installed as well.  Contact your OS or control panel vendor for assistance if you are unsure.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 4===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 4===</div></td></tr>
</table>Mshinnhttps://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=4503&oldid=prevMshinn: /* Step 3 */2014-02-28T15:38:14Z<p><span dir="auto"><span class="autocomment">Step 3</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 15:38, 28 February 2014</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 55:</td>
<td colspan="2" class="diff-lineno">Line 55:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If your OS is missing debuginfo, file a bug report with them.  Although we do make debuginfo rpms available in the atomic repository for our Apache builds, unless you use our free Atomic rpms you will want to install the correct debuginfo file for your system.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If your OS is missing debuginfo, file a bug report with them.  Although we do make debuginfo rpms available in the atomic repository for our Apache builds, unless you use our free Atomic rpms you will want to install the correct debuginfo file for your system.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Note: if you have a source built apache, such as with cpanel, you will need to do the following:</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Note: if you have a source built apache, such as with cpanel, you will need to <ins class="diffchange diffchange-inline">rebuild apache with debugging symbols.  For cpanel, you can </ins>do the following:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>1) set the appropriate CFLAGS</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>1) set the appropriate CFLAGS</div></td></tr>
</table>Mshinnhttps://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=4502&oldid=prevMshinn: /* Step 3 */2014-02-28T15:37:29Z<p><span dir="auto"><span class="autocomment">Step 3</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 15:37, 28 February 2014</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 55:</td>
<td colspan="2" class="diff-lineno">Line 55:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If your OS is missing debuginfo, file a bug report with them.  Although we do make debuginfo rpms available in the atomic repository for our Apache builds, unless you use our free Atomic rpms you will want to install the correct debuginfo file for your system.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If your OS is missing debuginfo, file a bug report with them.  Although we do make debuginfo rpms available in the atomic repository for our Apache builds, unless you use our free Atomic rpms you will want to install the correct debuginfo file for your system.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">Note: if you have a source built apache, such as with cpanel, you will need to do the following:</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">1) set the appropriate CFLAGS</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">export CFLAGS="-g"\x{2028}</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">2) Run easyapache</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">/scripts/easyapache --build </ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">If this does not work for you, please contact your apache vendor for assistance.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 4===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>===Step 4===</div></td></tr>
</table>Mshinnhttps://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=3924&oldid=prevMshinn: /* Apache Segmentation Faults */2013-09-04T18:20:00Z<p><span dir="auto"><span class="autocomment">Apache Segmentation Faults</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 18:20, 4 September 2013</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 11:</td>
<td colspan="2" class="diff-lineno">Line 11:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>''kernel: grsec: From 1.2.3.4: Segmentation fault occurred at 0000003000003dbc in /usr/sbin/httpd[httpd:15804]''</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>''kernel: grsec: From 1.2.3.4: Segmentation fault occurred at 0000003000003dbc in /usr/sbin/httpd[httpd:15804]''</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>You are experiencing a Segmentation Fault.  A Segmentation Fault, also referred to as a segfault or a "bus error" occurs when the systems hardware notifies the Operating system that a memory access violation has occurred, and the OS then notifies the application (via a signal) about this condition.  Most processes, by default, will then terminate the process and, if so configured, will also "dump core".  One symptom of this with apache is a "blank screen" or an empty response.  The child process or thread was killed, and it didnt finish whatever it was doing.   </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>You are experiencing a Segmentation Fault<ins class="diffchange diffchange-inline">.  This is a generic error in all Linux systems, and is not caused by [[ASL]]</ins>.  A Segmentation Fault, also referred to as a segfault or a "bus error" occurs when the systems hardware notifies the Operating system that a memory access violation has occurred, and the OS then notifies the application (via a signal) about this condition.  Most processes, by default, will then terminate the process and, if so configured, will also "dump core".  One symptom of this with apache is a "blank screen" or an empty response.  The child process or thread was killed, and it didnt finish whatever it was doing.   </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Lots of things can cause a sefault.  For example, with Apache it could be a bug in Apache itself, one of its core elements (like APR), PHP, a web application, an external application or even a library.  To find the cause of a segfault you need to generate a core file and perform what is called a backtrace.  Disabling pieces of apache isn't a good way to find the cause.  segfaults are memory errors, and simple "I disabled this and it stopped" cause and effect is extremely misleading.  This article will provide some guidance on one method you can use to do that and to find the actual cause of the segfault.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Lots of things can cause a sefault.  For example, with Apache it could be a bug in Apache itself, one of its core elements (like APR), PHP, a web application, an external application or even a library.  To find the cause of a segfault you need to generate a core file and perform what is called a backtrace.  Disabling pieces of apache isn't a good way to find the cause.  segfaults are memory errors, and simple "I disabled this and it stopped" cause and effect is extremely misleading.  This article will provide some guidance on one method you can use to do that and to find the actual cause of the segfault.</div></td></tr>
</table>Mshinnhttps://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=3454&oldid=prevMshinn: /* Core Files */2013-05-12T21:50:56Z<p><span dir="auto"><span class="autocomment">Core Files</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 21:50, 12 May 2013</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 21:</td>
<td colspan="2" class="diff-lineno">Line 21:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Core files allow you to diagnose what '''actually''' caused the memory fault on your system.  Think of them as application logs, they will tell you exactly what was going on and when it stopped working correctly.  To determine the cause, with a corer file, you must first configure your system to allow the creation of core files (this is usually disabled, as core files can use up a considerable amount of disk space).</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Core files allow you to diagnose what '''actually''' caused the memory fault on your system.  Think of them as application logs, they will tell you exactly what was going on and when it stopped working correctly.  To determine the cause, with a corer file, you must first configure your system to allow the creation of core files (this is usually disabled, as core files can use up a considerable amount of disk space).</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Step 1<del class="diffchange diffchange-inline">:</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">===</ins>Step 1<ins class="diffchange diffchange-inline">===</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Check that your system allows cores:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Check that your system allows cores:</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 31:</td>
<td colspan="2" class="diff-lineno">Line 31:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>ulimit -c unlimited</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>ulimit -c unlimited</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Step 2<del class="diffchange diffchange-inline">:</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">===</ins>Step 2<ins class="diffchange diffchange-inline">===</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Then you need to configure the application to "dump core" when it encounters a segmentation faulty.  For example, if the segfaults are occurring with Apache, configure Apache to dump core:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Then you need to configure the application to "dump core" when it encounters a segmentation faulty.  For example, if the segfaults are occurring with Apache, configure Apache to dump core:</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 47:</td>
<td colspan="2" class="diff-lineno">Line 47:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>   chmod og+rwx /tmp/apache2-gdb-dump</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>   chmod og+rwx /tmp/apache2-gdb-dump</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Step 3<del class="diffchange diffchange-inline">:</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">===</ins>Step 3<ins class="diffchange diffchange-inline">===</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Install the debuginfo packages for the applications you want to backtrace.  This is so your core file will contain the debug information needed to assist you in tracking down the source of the segfault.  For example, if you are backtracing just apache, then you need to install its debuginfo packages.  If you are debugging something else, like PHP, you need to install its debug symbols and so on.  Here is an example for a package managed apache install:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Install the debuginfo packages for the applications you want to backtrace.  This is so your core file will contain the debug information needed to assist you in tracking down the source of the segfault.  For example, if you are backtracing just apache, then you need to install its debuginfo packages.  If you are debugging something else, like PHP, you need to install its debug symbols and so on.  Here is an example for a package managed apache install:</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 56:</td>
<td colspan="2" class="diff-lineno">Line 56:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Step 4<del class="diffchange diffchange-inline">:</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">===</ins>Step 4<ins class="diffchange diffchange-inline">===</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Restart apache</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Restart apache</div></td></tr>
</table>Mshinnhttps://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=2303&oldid=prevMshinn: /* Backtracing */2012-04-14T03:28:42Z<p><span dir="auto"><span class="autocomment">Backtracing</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 03:28, 14 April 2012</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 68:</td>
<td colspan="2" class="diff-lineno">Line 68:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>gdb /path/to/httpd /path/to/core --batch --quiet -ex "thread apply all bt full" > backtrace.log  </div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>gdb /path/to/httpd /path/to/core --batch --quiet -ex "thread apply all bt full" > backtrace.log  </div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>And look at the backtrace for the cause.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>And look at the backtrace for the cause<ins class="diffchange diffchange-inline">.  Most often the cause is a bug in Apache, or one of its supporting libraries.  '''Please make sure you share your backtrace with your Apache vendor first so they can rule out if this an Apache bug.'''  So far, we've only see apache bugs that cause segfaults, so if you want your segfault resolved quickly contact your Apache vendor first.  They can tell you if the issue if with Apache, or with mod_security</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Keep in mind that memory use changes <del class="diffchange diffchange-inline">are </del>not <del class="diffchange diffchange-inline">causes, so </del>if you have a webapp <del class="diffchange diffchange-inline">or something </del>that uses a lot of memory and you get segfaults unfortunately the memory use isnt the cause. Its just correlational, <del class="diffchange diffchange-inline">more </del>memory in use means more opportunities for the fault to occur. It is not the cause. So <del class="diffchange diffchange-inline">if </del>you have more memory in use and you segfault, and less and you don't <del class="diffchange diffchange-inline">thats </del>not the cause of the <del class="diffchange diffchange-inline">problem</del>. You can rule <del class="diffchange diffchange-inline">that </del>out.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Keep in mind that memory use changes <ins class="diffchange diffchange-inline">do </ins>not <ins class="diffchange diffchange-inline">cause segfaults.  So </ins>if you have a webapp<ins class="diffchange diffchange-inline">, for example, </ins>that uses a lot of memory and you get segfaults unfortunately the memory use isnt the cause. Its just correlational, <ins class="diffchange diffchange-inline">not causational.  More </ins>memory in use means more opportunities for the fault to occur. It is not the cause<ins class="diffchange diffchange-inline">, it just increases the probability not the possibility</ins>. So <ins class="diffchange diffchange-inline">when </ins>you have more memory in use and you segfault, and less and you don't <ins class="diffchange diffchange-inline">segfault - the higher use of memory is </ins>not the cause of the <ins class="diffchange diffchange-inline">segfault</ins>. <ins class="diffchange diffchange-inline"> Think of that as a warning that you have something else wrong.  </ins>You can rule <ins class="diffchange diffchange-inline">memory use </ins>out.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== What changed? ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== What changed? ==</div></td></tr>
</table>Mshinnhttps://wiki.atomicorp.com/wiki/index.php?title=Apache&diff=1944&oldid=prevMshinn: /* Core Files */2011-10-06T17:03:32Z<p><span dir="auto"><span class="autocomment">Core Files</span></span></p>
<table class='diff diff-contentalign-left'>
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 17:03, 6 October 2011</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 49:</td>
<td colspan="2" class="diff-lineno">Line 49:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Step 3:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Step 3:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Install the debuginfo packages so your core file will contain the debug information needed to assist you in tracking down the source of the segfault:</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Install the debuginfo packages <ins class="diffchange diffchange-inline">for the applications you want to backtrace.  This is </ins>so your core file will contain the debug information needed to assist you in tracking down the source of the segfault<ins class="diffchange diffchange-inline">.  For example, if you are backtracing just apache, then you need to install its debuginfo packages.  If you are debugging something else, like PHP, you need to install its debug symbols and so on.  Here is an example for a package managed apache install</ins>:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>   yum install httpd-debuginfo</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>   yum install httpd-debuginfo</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If your OS is missing debuginfo, file a bug report with them.  Although we do make debuginfo rpms available in the atomic repository for our Apache builds, unless you use our free Atomic rpms you will want to install the correct debuginfo file for your system.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>If your OS is missing debuginfo, file a bug report with them.  Although we do make debuginfo rpms available in the atomic repository for our Apache builds, unless you use our free Atomic rpms you will want to install the correct debuginfo file for your system.</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Step 4:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Step 4:</div></td></tr>
</table>Mshinn