[knowledgeforge-user] problems with long svn updates from ukparse

Wed Oct 22 13:46:21 UTC 2008

On Sun, Oct 19, 2008 at 8:18 PM, Mark Longair
<mark-knowledgeforge at longair.net> wrote:
> I wonder if someone could help with the problems I've been
> having with ukparse:
>
> For all the time I've been using the ukparse SVN repository I've
> had lots of problems with long svn updates.  For example, when I
> was using it heavily earlier this year (on a very reliable
> network connection on JANET) I would frequently get errors like
> this:

The repository is pretty massive (GBs ...) and there seems to be quite
a few large docs (pdfs, html files) checked in. Thus a raw svn update
may simply take out the server (any tips on configuring apache/svn to
avoid this gratefully received)

When doing raw checkouts it might be an idea to them incrementally
(e.g. pick particular folders or subfolders and do svn ups on each of
those one at a time). Perhaps Francis may have useful comments here as
he knows the repository structure much better than me.

>  [.. lots of output ...]
>  A    svn/trunk/parldata/cmpages/lordspages/daylord2007-06-20b.html
>  svn: REPORT request failed on '/ukparse/svn/!svn/vcc/default'
>  svn: REPORT of '/ukparse/svn/!svn/vcc/default': Could not read chunk size: connection was closed by server. (http://knowledgeforge.net)

My guess here is that the server is simply being overwhelmed.

> (That was on a new checkout, although I would frequently see it
> on updates as well.)
>
> I tried again today (over ADSL rather than on JANET) to check
> out the whole repository and got something similar - after
> succesfully checking out some small percentage of the files the
> output paused and eventually I got an error indicating that the
> connection had timed out.  I got the same error (after fetching
> some more files) on updating again:
>
>  [.. lots of output ..]
>  A    parldata/cmpages/wrans/answers2001-12-12.html
>  A    parldata/cmpages/wrans/answers2005-03-01.html
>  A    parldata/cmpages/wrans/answers2000-07-20a.html
>  A    parldata/cmpages/wrans/answers2002-07-10.html
>  [.. long pause ..]
>  svn: REPORT request failed on '/ukparse/svn/!svn/vcc/default'
>  svn: REPORT of '/ukparse/svn/!svn/vcc/default': Could not read chunk size: connection timed out. (http://knowledgeforge.net)
>
> This is particularly frustrating because scripts that
> automatically update from ukparse on an infrequent basis never
> work reliably for me.

Does this occur once you have done a complete checkout. At that point
the updates should be much smaller and shouldn't make the server fall
over so badly.

> Is there some timeout set on the server that might be causing
> these problems?  I'd be grateful for any help with this.

Hope the above is of some use. I should also mention you could create
a local copy of the repo by syncing the backup via rsync (slowly with
b/w under, say, 30kb/s) see:

http://lists.okfn.org/pipermail/kforge-user/2006-January/000029.html

(You'll just want the ukparse subitem by the way)

Rufus