[okfn-help] further to load on us0

Wed Oct 7 16:34:31 BST 2009

Martin Keegan wrote:
> On Wed, 7 Oct 2009, James Casbon wrote:
> 
>> 2009/10/7 Martin Keegan <martin at no.ucant.org>:
>>> Today (being Wednesday 7 Oct), check out the changed memory usage profile
>>> visible after 11am when I made the config change (the results may not be
>>> so clearly visible after a few hours):
>> That looks a lot better.  So what was the issue?
> 
> There is some memory leak (or equivalent behaviour) in something loaded in
> the Apache address space, e.g., some data being accumulated in some python
> code. I set a limit on the number of requests any worker process may
> handle. (This necessitated rebalancing the load by changing the maximum
> thread count and the total number of threads and processes). Once the
> limit is reached, the process terminates, freeing the memory, and gets
> replaced with a fresh instance.
> 
> There is still more work to do, but at least the responsiveness of the
> machine is back to tolerable.

Interesting.

FWIW if that Python code is KForge, which uses Django, which recommends 
service execution with pre-fork rather than worker Apache, and you're 
limiting requests to workers, then switching Apache may help.

http://djangoinproduction.com/wiki/Prefork_MPM_or_worker_MPM
http://groups.google.com/group/django-users/browse_thread/thread/e64b80f4ef3d61a9
http://www.mail-archive.com/django-users@googlegroups.com/msg13695.html

But in any case, making sure the processes terminate will refresh the 
parts other garbage collectors don't reach. :-)

Jb

> 
> Mk
> 