[annotator-dev] JSON-LD as data interchange format [WAS Re: Branching v2.0.x]

Thu Oct 31 10:57:48 UTC 2013

Hi Nick,

On Thu, Oct 31, 2013 at 5:40 AM, Nick Stenning <nick at whiteink.com> wrote:
> I think we should be wary of exposing the full complexity of OA to users
> (or integrators, or plugin developers) unless we are clear on exactly
> what benefits that brings.

One thing to consider is that integrators may already be aware of
OpenAnnotation and JSON-LD ; or at least more aware of them than the
annotator's ad-hoc JSON serialization. So, hypothetically, OA+JSON-LD
could make it easier for people to integrate, not harder.

I say hypothetically because both are new enough that I'm not sure it
is the case :-D  But maybe, if JSON-LD and OpenAnnotation continue on
the trajectories they are on, this will not be the case soonish? It
seemed like a useful line-in-the-sand for a v2.0 release, to be able
to say OpenAnnotation inside, mostly (but not entirely) for marketing
reasons. But I guess that doesn't stop people from saying it anyway,
even if it's not entirely accurate.

> The mechanism used by Anna Gerber is the one I'd like us to consider
> pursuing, where Annotator continues to have its own internal model of
> the annotation, without any conceptual or technological "linked data"
> overhead. To speak concretely, that means that a plugin still deals with
> plain ol' JavaScript objects. No contexts, no "@"-symbols.
>
> It's then up to a specific storage engine to render that object into an
> interchange format of it's own choosing. Conveniently, JavaScript
> objects are more-or-less already an interchange format, so the default
> store can simply pass the object (or rather a JSON-stringified version
> thereof) back to the reference annotator-store.
>
> I am very keen, however, to see a parallel effort (which should be made
> vastly simpler by the changes now in progress for Annotator 2.0.x) to
> develop a store plugin which transforms the Annotator internal
> representation into JSON-LD and uses that as an interchange format.
> Perhaps someone will even integrate this with an existing triple store?

But the work would not end there. They would also need to create a
plugin for the annotator that understood OA+JSON-LD, and could convert
it to the internal objects used by the annotator-store, right? At
least this would be required if you wanted integration with the rest
of the plugin ecosystem.

Alas, it is easy to push Linked Data off mentally to a time when a
SPARQL aware triple store is being used. For me, this effectively
pushes it pretty far off into the future, at least for production
systems. And yet, I think there could be some real hypermedia benefits
to using OpenAnnotation and JSON-LD together.

I think of JSON-LD as a hypermedia format (HATEOAS), that allows
clients and servers to be decoupled a bit more. Clients can follow
their nose in the data rather than cooking in assumptions about the
API. This might allow annotation clients and servers to evolve
independently of each other, without the need to create a versioned
namespace for the API, and making everyone upgrade, and possibly
fragmenting the ecosystem. Yes, my lips are red from drinking from the
pitcher of REST Kool-Aid. Of course it would be possible to get some
of the benefits of HATEOAS without buying into OpenAnnotation and
JSON-LD. Randall's ticket about using URI-Templates is a good example
of an incremental improvement in this area.

To return to the topic at hand, I was mainly wondering if the
annotator-store could continue serializing JSON objects as it does now
with ElasticSearch, without having to parse triples or anything like
that. But perhaps there would need to be some work around validating
and normalizing the object, since it could be possible to say the same
thing multiple ways, if multiple contexts are used. There would
probably need to be a marshall/unmarshall method of some kind on the
Annotation class, which would also add a layer of complexity that
isn't there now.

Sorry to distract, I haven't been part of the 2.0 discussions, so I
took the opportunity to throw this out there.

//Ed