[okfn-labs] Fwd: [annotator-dev] Annotator architecture evolution

Mon Jun 24 17:26:43 UTC 2013

[apologies for x-post]

A major update on some future plans on http://okfnlabs.org/annotator/ for
those not on annotator-dev (but interested in text annotation,
openliterature, open humanities etc :-)

---------- Forwarded message ----------
From: Nick Stenning <nick at whiteink.com>
Date: 24 June 2013 17:04
Subject: [annotator-dev] Annotator architecture evolution
To: "annotator-dev at lists.okfn.org" <annotator-dev at lists.okfn.org>

Dear Annotators,

Over the past few months, I've been discussing the future of Annotator's
APIs with those of you I've met in person at conferences and cafés, or
online in #annotator and elsewhere. It's time to bring those discussions to
the wider community on this mailing list for review.

My primary concern is making integrators' lives easier.

Specifically, that means looking at how people currently use Annotator, and
how with appropriate changes to the internal architecture they would be
able to work faster and more productively.

This is a long email, but if you are working with Annotator on a daily
basis I'd really appreciate feedback, in particular on the first section,
which attempts to lay out the current architectural flaws in Annotator.

---- ---- ---- ---- ---- ----

# What's wrong right now?

## UI coupling

Implementing your own UI is difficult. The current highlighter, editor and
viewer are quite tightly bound to core Annotator and this has made life
difficult for Hypothesis, AustESE, and others.

## Storage coupling

The Store plugin is rather special, but this is implicit. It doesn't
implement a different plugin API to any other plugin, but rather hooks into
the existing annotationCreated/annotationUpdated events raised by core
Annotator.

On one level, this could be seen as a good thing, but my feeling is that it
substantially raises the complexity of Annotator as a whole. In particular,
storage plugins are the only plugins on which most of the rest of Annotator
depends, and in which success or failure of function calls may be entirely
outside the control of the client side code. The process by which the
results of these calls are communicated to the rest of Annotator and
plugins is currently mostly magic.

We need to make the privileged nature of storage plugins explicit, and
implement a clear mechanism for constructing and loading annotations
asynchronously, so that other plugins don't have to be exposed to the
complexity of the storage implementation.

## Inflexible matching

Annotator only supports one kind of selector by default, the
XPointer/TextPositionSelector method that it's used since day one.

We need to provide native support for matcher plugins, which receive
annotations at load time, add arbitrary data to those annotations (in the
XPointer case, this might be an array of DOM textNodes) which can be used
by view plugins (e.g. Highlighter, Viewer, etc.).

This data wouldn't be serialized and as such we need a more general pattern
for attaching local data to annotations. Something better than

https://github.com/okfn/annotator/blob/8ff0778/src/plugin/store.coffee#L462-L473

anyway.

## Interplugin communication more generally

There are a couple of places where one plugin relies on the actions of
another plugin. In some places this isn't done in a general or flexible
way, e.g.

https://github.com/okfn/annotator/blob/8ff0778/src/plugin/store.coffee#L105-L108

In other places, this integration is direct between plugins and almost
invisible:

https://github.com/okfn/annotator/blob/8ff0778/src/plugin/auth.coffee#L223-227

https://github.com/okfn/annotator/blob/8ff0778/src/plugin/store.coffee#L370

Wherever possible, this kind of integration should be mediated through a
common piece of infrastructure, namely Annotator core.

There's scope here to clean up some of the existing event naming, too.

## Removal and idempotence

Removing and reinstantiating an Annotator instance is currently quite
tiresome. We need a generic way of blatting existing Annotator instances
and replacing them with new ones.

---- ---- ---- ---- ---- ----

And now, proposals for improvement:

---- ---- ---- ---- ---- ----

## Plugin types

As I see it, there are essentially three types of plugin:

1. Generic plugins

A generic plugin can subscribe to Annotator events, such as annotation
creation, update, or deletion. They might bind their own events to the DOM
to present UI to create, update, or delete annotations. Also included might
be plugins that inform the user of error conditions, manage auth tokens,
etc.

2. Store plugins

Store plugins implement a common interface to CRUD and query operations on
backend data stores. Each operation might succeed or fail due to model
constraints, network outage, etc. A minimal interface might be:

  create :: obj -> Promise(ann)
  update :: ann -> Promise(ann)
  delete :: ann -> Promise(None)
  query :: id -> Promise(([ann], queryMeta))

Store plugins are managed by a new concept in Annotator-land, the registry.
The registry is responsible for abstracting other plugins and users from
the details of any particular Store plugin, for error handling, and for
integration with Matcher plugins (see below).

Critically, the registry can also receive events from the Store plugin,
which it will then distribute to observers (generic plugins). This gives us
a relatively sane mechanism for implementing Store plugins which stream
data from the backend store (i.e. realtime collaboration).

Only one store plugin will be active at any one time.

3. Matcher plugins

These are plugins that take annotations from their serialized form, and use
the current execution context (including, say, the current state of the
DOM) to add additional data to the annotations which can be used by generic
plugins to identify and draw annotations on the page.

## Asynchrony

Things that need to be done asynchronously return Promises. 'Nuff said.

---- ---- ---- ---- ---- ----

I reckon that making clear the different purposes of the different plugins
and extracting the stuff that is currently in Annotator core into
appropriate plugins will solve most of the issues people have encountered
integrating with Annotator.

This is, however, a big chunk of work, and I don't think it's going to be
possible to do so without breaking some existing integrations. While
obviously we should try and minimise breakage, I think it's possible that
the result of some of this work will become Annotator v2.0.0 (for an
explanation of why, see http://semver.org/).

Thanks for reading this far!

-N

_______________________________________________
annotator-dev mailing list
annotator-dev at lists.okfn.org
http://lists.okfn.org/mailman/listinfo/annotator-dev
Unsubscribe: http://lists.okfn.org/mailman/options/annotator-dev

-- 
*

Rufus Pollock

Founder and Co-Director | skype: rufuspollock |
@rufuspollock<https://twitter.com/rufuspollock>

The Open Knowledge Foundation <http://okfn.org/>

Empowering through Open Knowledge
http://okfn.org/ | @okfn <http://twitter.com/OKFN> | OKF on
Facebook<https://www.facebook.com/OKFNetwork>|
Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter>

*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/okfn-labs/attachments/20130624/4aed184b/attachment-0001.html>