[knowledgeforge-user] maintenance work
john.bywater at appropriatesoftware.net
Wed Feb 3 10:13:26 UTC 2010
Mark Longair wrote:
> Rufus Pollock <rufus.pollock at okfn.org> wrote:
>> 2009/5/3 Mark Longair <mark-knowledgeforge at mythic-beasts.com>:
>>> Rufus Pollock <rufus.pollock at okfn.org> wrote:
>>>> 2009/4/20 Mark Longair <mark-knowledgeforge at mythic-beasts.com>:
>>>>> Unfortunately, I still find that the connection is lost during
>>>>> "git svn" updates, sometimes after it has been running for for
>>>>> half an hour, sometimes up to an hour. What are Apache's
>>>>> timeout and keepalive timeout settings set to?
>>>> KeepAlive On
>>>> Timeout 300
>>>> MaxKeepAliveRequests 500
>>>> KeepAliveTimeout 5
>>> Hm. I suppose that KeepAliveTimeout could be a bit higher, but
>>> it seems to me that it's a bit unlikely to be the problem given
>>> the symptoms I'm seeing - I've noticed that when I get a
>>> connection reset it becomes impossible to connect to port 80 of
>>> project.knowledgeforge.net for a couple of minutes afterwards,
>>> either from JANET or from my home ADSL connection.
>>> So that you have a time point to find in the logs when this
>>> definitely happened, I set "git svn clone --stdlayout
>>> http://knowledgeforge.net/ukparse/svn/" going earlier from the
>>> client 184.108.40.206 and got the connection reset by peer error 
>>> Sun May 3 12:57:40 BST 2009
>>> I could connect to port 80 again at 13:01, but there were a
>>> couple of minutes before that wwhere every connection attempt
>>> from each network I tried failed.
>> Monit reboots apache when system overloads and this is what seems to
>> be happening here.
>> My "guess" (from experience) is that because this is "graceful" it can
>> be slow since it waits for existing connections to die/time-out and
>> this may explain the long period
>>> I hope this is of some help in tracking down the problem.
>> Will see what we can do :)
> I've found a workaround for this now, so I thought it would be
> worth sending an update to the list with more details of the
> problem. The quick version is "make sure you have
> 'http-compression = yes' set in ~/.subversion/servers". For more
> details, read on...
> It seems that when I was doing very long subversion operations
> the memory consumption of Apache grew until monit was provoked
> into restarting it, as if there was a memory leak of some kind.
> It seems most likely that this is the same problem as described
> in this long-standing bug report against mod_dav_svn:
> We very briefly tried one of the earlier suggested workarounds
> (SVNAllowBulkUpdates Off) but it caused svn operations to be
> unbearably slow, so that was unacceptable.
> In November last year, someone updated the bug report suggesting
> that it was the same as this one:
> The "massive" memory leak described there can be reproduced when
> the server is set up to use mod_deflate but the client doesn't
> indicate that it can cope with compression. It does indeed seem
> that this was the problem for me - setting "http-compression =
> yes" in my client means that operations that fetch thousands of
> revisions can now run to completion correctly.
> This means that finally I have a "git svn" import of the whole
> of ukparse, which is a joy :) (The repository with complete
> history is 9GB while the working tree is 23GB.)
> I hope that's of some use.
> P.S. Short of actually fixing the mod_dav_svn bug, the other
> possible workaround we discussed was to patch kforge to offer
> subversion over the svn protocol as well as the HTTP-based
> transport. Obviously that's much more work, though.
Very interesting notes. Another option would be to convert your
Subversion repository to Git. http://blog.notahat.com/posts/25
More information about the knowledgeforge-user