[openbiblio-dev] New instance: eu11.okfn.org

Mon Nov 15 11:59:59 UTC 2010

To be fair, its not uncommon for libraries to measure system rebuild times
in days! Oxford for example were striving to get under the 24hr mark :)

As for dedupe, for the records that matched, it was unclear how to pick the
'correct' record without someone reading them. What we can do is assert that
they match, and work on ways of building the best display record from them.

Ben

On Nov 15, 2010 11:44 AM, "Rufus Pollock" <rufus.pollock at okfn.org> wrote:

(cc'ing openbiblio-dev)

On 14 November 2010 22:31, William Waites <ww at eris.okfn.org> wrote:
> * [2010-11-14 21:43:03 +0000] Rufus Pollock <rufus.pollock at okfn.org>
écrit:
>
> ] As per earlier discussion I have spun up a new m1.large amazon
> ] instance running ubuntu lucid. Machine identity:
>
> Thanks Rufus. It's chugging happily along importing
> records at a rate of about 140/sec (approx 7k
> triples/sec). I'll try to get all the data imported
> before telling the web interface to use it or
> exposing the sparql endpoint...

That's still very slow (i.e. 59h to do the whole lot!). Can one turn
off transactions or the like for bulk uploading to speed it up?

Also are you doing any de-duping (at least on entities)? (Since we're
creating them may be sensible to dedupe as part of upload ...)

Rufus

_______________________________________________
openbiblio-dev mailing list
openbiblio-dev at lists.okfn.org
http://lists.okfn.org/mailman/listinfo/openbiblio-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/openbiblio-dev/attachments/20101115/4e961aa1/attachment.html>