[annotator-dev] Upcoming changes to Annotator and the Annotator Store

[Rufus Pollock]

On 31 January 2012 23:35, Nick Stenning <nick at whiteink.com> wrote:
> Hi all,
>
> In order for this email to make any sense, I'm going to need to
> provide a little bit of historical/technical background. In short, the
> authentication system for AnnotateIt (http://annotateit.org) was in
> serious need of an overhaul to bring it in line with what Annotator
> was originally designed for. In particular, we want people to be able
> to use AnnotateIt as a service in which to store not just their own
> annotations, but also those of their users.
>
> The way this was originally intended to work goes as follows:
>
> 1) Alice builds a website with documents which need annotating, DocLand.
>
[...]

Really useful recap Nick -- and apologies if our original plan didn't
get perfectly implemented (it would be good to talk directly at some
point about that so I'm sure I understand exactly).

If you weren't aware, the original planning document [1] got written up here:

https://github.com/okfn/annotator/wiki/Authentication

We should definitely update / correct that as necessary.

[1]: http://lists.okfn.org/pipermail/okfn-help/2010-December/000977.html

> Now, due to time constraints, this isn't really how AnnotateIt works
> at the moment, and the code got a little confused about the
> distinction between users (Bob) and consumers (DocLand). This wasn't
> helped by the fact that AnnotateIt is one of its own consumers (we
> provide people with bookmarklets which allow them to annotate anything
> on the web).
>
> Over the past few days I've been working to resolve this confusion,
> and I'm slowly getting there. You can see the work at:

This is absolutely brilliant Nick. I've done a quick review and this
all looks *really* good.

> https://github.com/okfn/annotator-store/commits/feature/es+sql

ES stuff looks great (as does SQL). I've added a comment here about
some other places we've done wrapping of ES that could be useful to
look at: <https://github.com/okfn/annotator-store/commit/020e6cee2ef4535c8ff07412d84951464f6af29a#commitcomment-921773>

> https://github.com/okfn/annotator/commits/feature/fixauth

Aside: I was going to suggest an even more radical change (orthongonal
to ES/SQL change -- which is great!) which was to move to nodejs but
that can wait until later!

> The key features of the new system are as follows:

This is an excellent summary. Can I boot some tickets to record these
-- both for tracking and posterity?

> 1) Users of AnnotateIt will be able to use a bookmarklet to annotate
> around the web, as before. This bookmarklet will now no longer contain
> any code or data which, if stolen, would result in the compromise of
> your AnnotateIt account.
>
> 2) When you click the bookmarklet, you'll (finally) be able see the
> annotations of *any* AnnotateIt user who's annotated that page and
> elected for their annotations to be public.

Great (that was at least partly working in the past IIRC correctly a
demo I did last summer!)

> 3) Users of AnnotateIt will be able to register for multiple consumer
> key/secret pairs, just like you (used to) apply for Google Maps API
> keys.

Really nice.

> 4) Allowing registered AnnotateIt users to annotate your website is
> now a blindingly trivial two lines of code (once you've included
> Annotator's source files):
>
>    $('body').annotator()
>             .annotator('setupPlugins');

:-)

> 5) Annotations are now stored in ElasticSearch, meaning that the
> future looks bright for full-text search and analytics over the
> annotation database.

:-) ++

> I'll be keeping you posted when I get the code into a stable state,
> after which I hope we'll be able to migrate the data that's currently
> in AnnotateIt's CouchDB store, and get moving with a better, more
> flexible authentication system.
>
> In the meantime, I'd appreciate any comments or criticisms. Feel free
> to dive in on Github and comment on commits and issues, or reply here.

Really great stuff Nick. As I said I've already been taking a look but
I can dive in deeper. I'd also like to start work on a better
dashboard which uses the full power of the ES query system (facets,
stats etc ...)

Rufus