[open-bibliography] Updating distributed, federated, partially replicated bibliographic databases

[Mark MacGillivray]

HI Tom,

I have been working a bit with Rufus Pollock, Richard Jones, Jim
Pitman and Peter Murray-Rust on the BibServer code, we have a
prototype up at bibsoup.net. We are attempting to secure funding to
work on this for a year. The aim is to build a system where
individuals and small groups can easily upload their own collections,
however the same software can also handle larger collections. Ideally
I would like to populate it with large collections as well as enabling
small groups to do their thing, and for those large collections to be
easily updateable by their sources - either by direct management or by
access via API. The BibServer software is being built on
elasticsearch, and there is no reason why we and others could not run
multiple instances of these and share data between them.

We need to do a bit of work on agreeing the underlying format - we are
working with JSON, and specifying a BibJSON - the idea for now being
that it handles what you would usually find in bibtex, but should be
extensible. Further input into this would be appreciated! If anyone
else is interested in helping out with BibJSON, pleases let me know.
But I should point out, our core aim is simplicity and usability by
typical end users.


Mark




On Wed, Sep 14, 2011 at 11:08 AM, ianibbo at gmail.com <ianibbo at gmail.com> wrote:
> Have to say I have a massive amount of sympathy with this point of
> view. It does often worry me that projects like this have pushed back,
> rather than advanced, the cause of openly sharing standardised
> bibliographic info.
>
> No easy solutions here, but certainly very interested to contribute to
> anything that might emerge from this.
>
> Ian.
>
> On 13 September 2011 20:16, Tom Morris <tfmorris at gmail.com> wrote:
>> Like the XKCD cartoon about standards (http://xkcd.com/927/), it's
>> pretty clear that the pace of creation of new bibliographic databases
>> and services isn't going to abate any time soon as people create new
>> "one true unifying bibliographic database" instances.  Open Library,
>> Wikipedia, national libraries, Freebase, OCLC, etc will all have one
>> page per book, modulo duplicates, omissions, errors, and differences
>> in focus.
>>
>> As people grab one-time database dumps and republish them,
>> disconnected from their original source and without any provenance,
>> the problem will be compounded.
>>
>> How do we move from a static set of dumps to a dynamic set of update
>> streams that enable a vibrant ecosystem of cooperatively cataloged
>> entities?  A centralized OCLC-based monopoly is one solution, but not
>> a good one, in my opinion.
>>
>> Is anyone working on enabling such a set of federated catalogs?  Would
>> anyone like to?
>>
>> Tom
>>
>> p.s. There's an earlier thread which touches on this topic here:
>> http://lists.okfn.org/pipermail/open-bibliography/2010-May/000140.html
>>
>> _______________________________________________
>> open-bibliography mailing list
>> open-bibliography at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/open-bibliography
>>
>
>
>
> --
> Ian Ibbotson
> W: http://ianibbo.me
> E: ianibbo at gmail.com
> skype: ianibbo
> twitter: ianibbo
>
> _______________________________________________
> open-bibliography mailing list
> open-bibliography at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/open-bibliography
>