[mdlug] [Apache] High Load Issues... ?

[Jonathan Billings]

On Mon, Apr 13, 2009 at 04:07:10PM -0400, Wojtak, Greg wrote:
>
> I recently updated from RHEL 4 to CentOS 5, taking Apache along for
> the ride (2.0 -> 2.2).  Recently (only in our production
> environment), we've been seeing load randomly shoot up to around 130
> or so.  One of the things I've noticed is that during these
> occurrences, apache is always on top when running top, though it is
> always different PID's, and (this is the one that gets me) memory
> utilization drops tremendously.  I assume there is some kind of
> garbage cleanup going on.   Does anyone know what apache might be
> doing that frees up approximately 200 MB or more of physical RAM?
> Any other insights? 

Often when you see a high load like that is when you have a lot of
processes waiting on a kernel call, like disk wait or a TCP socket.
Check your apache configuration, I suspect the maximum number of
httpds that are to be running is close to 130.

Keep in mind:  high load != lots of CPU operations.  "Load" is defined
by a complex algorithm, basically giving the lenght of the "queue" of
operations to run.  If there are a lot of processes waiting on a
kernel call to complete, each one of those processes increments the
load by one.

Perhaps what you're seeing is a scan on your web server where the
client doesn't close the connection correctly.  Next time you see it,
check out lsof or netstat.  Or maybe you're serving data out with each
connection, and they're waiting on IO?

-- 
Jonathan Billings <billings at negate.org>