[annotator-dev] Annotator and Meteor

Randall Leeds tilgovi at hypothes.is
Thu Sep 11 18:47:05 UTC 2014 

On Thu, Sep 11, 2014 at 11:01 AM, Bryan <btbonval at gmail.com> wrote:

> Randall,
>
> I'm not sure I understand how to compare the serialized data as you
> suggest. The serialized data stores the location information as an
> xpath-range, which is a highly specialized metric.
>
> We have tried very hard to use the xpath and the offset of the serialized
> data, without access to xpath-range, and we have not been able to to crack
> the ordering egg without reimplementing xpath-range to compare the
> internally stored position information.
>

It's not really reimplementing anything in xpath-range, because xpath-range
doesn't compare positions.

Here's an algorithm:

 - Split on /
 - Split each component "tagName[position]" if it has a position
 - Sort these arrays
   A normal comparison sort works fine:
     > [['div', 2], ['p']] < [['div', 2], ['p', 3]]
     true
     > [['div', 1], ['p', 1]] < [['div', 2], ['p', 1]]
     true

This doesn't give you a complete result because XPath gives you the
relative order of siblings having the same tag, but not absolute ordering
of each child in its sibling set.

Also, if you are concerned about screen position, CSS can totally change
that in ways that don't match the structure.

But it should be good enough for some uses.

On Thu, Sep 11, 2014 at 1:57 PM, Randall Leeds <tilgovi at hypothes.is> wrote:

> I just checked and see that xpath-range is not exported, it's only used
> internally in the LegacyRanges plugin.
>
> But that module doesn't have any API for comparing the ranges. If the
> purpose is to compare the annotation positions before they are anchored,
> you should be able to do what you want without access to xpath-ranges. You
> will want to compare the serialized data. Just do so.
>
> On Thu, Sep 11, 2014 at 10:17 AM, Bryan <btbonval at gmail.com> wrote:
>
>> The real problem we're experiencing, as Ari pointed out later in the
>> thread, is being able to independently sort annotations in the order they
>> are presented in the contents of their page.
>>
>> Our assumption was that this must be handled through xpath range as we
>> could not find any better way to translate the annotation properties into
>> this ordering. One recommendation was made which we should look into using
>> compare document position. Sorry I'm typing on a phone, so put those last
>> three words together.
>>
>> I think we had tried something like that, but it only worked for
>> annotations in situ, not annotations as base objects not yet rendered. We
>> can take another look.
>>
>> The other option is simply to expose xpath range as noted in a few ways
>> to compare relative positions of annotations outside the context of a
>> rendered page.
>> On Sep 11, 2014 1:10 PM, "Randall Leeds" <tilgovi at hypothes.is> wrote:
>>
>>> We already build as a UMD in the output. One step ahead of you all :-D.
>>>
>>> I'm not sure what the problem is in this thread anymore.
>>>
>>> Is it that xpath-range isn't exported? We used to have Annotator.Range I
>>> think, before we split it out into a separate package. We can definitely
>>> assign it as a property of the Annotator global.
>>> On Sep 11, 2014 6:05 AM, "Benjamin Young" <bigbluehat at hypothes.is>
>>> wrote:
>>>
>>>> Looks a lot like UMD. :)
>>>>
>>>> Maybe give this post a look:
>>>>
>>>> http://dontkry.com/posts/code/browserify-and-the-universal-module-definition.html
>>>>
>>>> Apparently there are a good number of tools for browserify that can
>>>> work with other module types.
>>>>
>>>> This may just serve as surrounding educational material and not the
>>>> solution, but it looks promising all the same. :)
>>>> On Sep 11, 2014 4:34 AM, "Mitar" <mmitar at gmail.com> wrote:
>>>>
>>>>> Hi!
>>>>>
>>>>> One thing I noticed some project have is a GitHub hook which
>>>>> automatically compiles master branch into dist branch where you can
>>>>> then make a submodule and have files. I thing this would be very
>>>>> useful to have. Example:
>>>>>
>>>>> https://github.com/guardian/scribe/tree/dist
>>>>>
>>>>> And Annotator could also use Browserify, but not require it, wrapping
>>>>> the result in something like:
>>>>>
>>>>>  // AMD / RequireJS
>>>>>     if (typeof define !== 'undefined' && define.amd) {
>>>>>         define([], function () {
>>>>>             return Annotator;
>>>>>         });
>>>>>     }
>>>>>     // Node.js
>>>>>     else if (typeof module !== 'undefined' && module.exports) {
>>>>>         module.exports = Annotator;
>>>>>     }
>>>>>     // included directly via <script> tag
>>>>>     else {
>>>>>         root.Annotator = Annotator;
>>>>>     }
>>>>>
>>>>>
>>>>> Mitar
>>>>>
>>>>> On Wed, Sep 10, 2014 at 11:34 AM, Randall Leeds <tilgovi at hypothes.is>
>>>>> wrote:
>>>>> > You could grab the all the highlights and sort them using
>>>>> > document.compareDocumentPosition.
>>>>> >
>>>>> > Annotator stores the annotations using jQuery.fn.data on the
>>>>> highlight
>>>>> > nodes. If you need help with that let me know.
>>>>> >
>>>>> > On Sep 9, 2014 11:24 AM, "Ari Entlich" <atrigent at gmail.com> wrote:
>>>>> >>
>>>>> >> Actually, I just had another idea. The actual reason we want access
>>>>> to
>>>>> >> xpath-range is so that we can consume the ranges that are stored in
>>>>> >> annotation objects (SerializedRanges, if I understand correctly)
>>>>> and compare
>>>>> >> two of them to determine their relative position in the annotated
>>>>> document.
>>>>> >> We would like to display a list of annotations in the order that
>>>>> they appear
>>>>> >> in the document. Perhaps it would make sense to simply add such a
>>>>> comparison
>>>>> >> function to annotator? Or maybe there's a better way of
>>>>> accomplishing the
>>>>> >> desired sorting that I'm unaware of?
>>>>> >>
>>>>> >> Thanks.
>>>>> >>
>>>>> >> Ari
>>>>> >>
>>>>> >> On Tue, Sep 9, 2014 at 1:24 PM, Ari Entlich <atrigent at gmail.com>
>>>>> wrote:
>>>>> >>>
>>>>> >>> On Sat, Sep 6, 2014 at 9:58 PM, Randall Leeds <tilgovi at hypothes.is
>>>>> >
>>>>> >>> wrote:
>>>>> >>>>
>>>>> >>>> I think whatever you do is a stopgap for now.
>>>>> >>>>
>>>>> >>>> If it helps we can push (I think we already have?) a dev tag to
>>>>> npm.
>>>>> >>>>
>>>>> >>>> Build a separate repo for the meteor package that depends on
>>>>> annotator
>>>>> >>>> and you should hopefully get the built files if we're doing our
>>>>> part to
>>>>> >>>> upload the build.
>>>>> >>>
>>>>> >>> On Sun, Sep 7, 2014 at 12:34 AM, Randall Leeds <
>>>>> tilgovi at hypothes.is>
>>>>> >>> wrote:
>>>>> >>>>
>>>>> >>>> npm dependency for the meteor build. Then npm has the built
>>>>> sources.
>>>>> >>>> Meteor package doesn't know or care about browserify.
>>>>> >>>>
>>>>> >>>> If any downstream consumer has to care that we use browserify
>>>>> we're
>>>>> >>>> doing it wrong.
>>>>> >>>>
>>>>> >>>> The output is JavaScript.
>>>>> >>>
>>>>> >>> Ah ok, I do see that the annotator stuff uploaded to npm is just
>>>>> the
>>>>> >>> generated js files. That's probably good enough for most use
>>>>> cases, but
>>>>> >>> we've been thinking of customizing the browserify build to give us
>>>>> access to
>>>>> >>> the bundle's internal copy of xpath-range. To do this, I believe
>>>>> we would
>>>>> >>> need to care about the use of browserify. We could potentially
>>>>> create meteor
>>>>> >>> packages for both annotator and xpath-range and use both of them,
>>>>> but it
>>>>> >>> looks to me like the xpath-range in npm is NOT fully preprocessed
>>>>> - the
>>>>> >>> coffeescript step has been run, but the browserify step has not.
>>>>> In general
>>>>> >>> though, I do think it makes sense for us to just use the copy
>>>>> inside the
>>>>> >>> annotator bundle, since it's already there.
>>>>> >>>
>>>>> >>> Ari
>>>>> >>>
>>>>> >>>>
>>>>> >>>> On Sep 6, 2014 11:47 AM, "Ari Entlich" <atrigent at gmail.com>
>>>>> wrote:
>>>>> >>>>>
>>>>> >>>>> Hey all. Sorry for not jumping in sooner.
>>>>> >>>>>
>>>>> >>>>> Mitar, I did discover your annotator package in peerlibrary while
>>>>> >>>>> looking through your work. However, Andrew and Bryan are correct
>>>>> that you're
>>>>> >>>>> using an older version than we are and that your approach
>>>>> unfortunately will
>>>>> >>>>> not work for us. I don't believe it is possible to just "ignore
>>>>> browserify",
>>>>> >>>>> because the most recent annotator code is making extensive use
>>>>> of the
>>>>> >>>>> CommonJS-like module system that browserify provides. As far as
>>>>> I know,
>>>>> >>>>> meteor would not serve as an adequate replacement for browserify
>>>>> in this
>>>>> >>>>> respect.
>>>>> >>>>>
>>>>> >>>>> Essentially what needs to happen is that we need to have meteor
>>>>> call
>>>>> >>>>> browserify during its build process. I've found
>>>>> >>>>> Plugin.registerSourceHandler, which allows you to make build
>>>>> plugins that
>>>>> >>>>> say "I can handle files with filenames matching some pattern".
>>>>> However, this
>>>>> >>>>> seems to be aimed primarily at registering generic handlers for
>>>>> file types -
>>>>> >>>>> coffescript, less, sass, etc files. What we need is a way to
>>>>> specify how to
>>>>> >>>>> build a specific package and to add the build artifacts from
>>>>> that build
>>>>> >>>>> process to the meteor app. I'm currently investigating whether
>>>>> it would be
>>>>> >>>>> possible to do it in the package definition section of the
>>>>> package.js file
>>>>> >>>>> (http://docs.meteor.com/#pack_onUse).
>>>>> >>>>>
>>>>> >>>>> If the above winds up working, we can easily tell browserify to
>>>>> expose
>>>>> >>>>> the annotator bundle's copy of xpath-range. I'd rather not make
>>>>> changes to
>>>>> >>>>> annotator itself in order to get this working.
>>>>> >>>>>
>>>>> >>>>> A possible alternative to using git submodules is to simply
>>>>> declare
>>>>> >>>>> annotator as an npm dependency of the meteor package. If that
>>>>> winds up
>>>>> >>>>> working, I think it would be the preferable approach.
>>>>> >>>>>
>>>>> >>>>> Ari
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> On Fri, Sep 5, 2014 at 9:13 PM, Mitar <mmitar at gmail.com> wrote:
>>>>> >>>>>>
>>>>> >>>>>> Hi!
>>>>> >>>>>>
>>>>> >>>>>> You can package xpath-range as additional package as well, if
>>>>> you
>>>>> >>>>>> want.
>>>>> >>>>>>
>>>>> >>>>>> If you want to add xpath-range, you can always just add it
>>>>> inside the
>>>>> >>>>>> Meteor package for Annotator as a field of top-level object. So
>>>>> you
>>>>> >>>>>> create one more file which you add at the end of all files from
>>>>> >>>>>> Annotator and do something like:
>>>>> >>>>>>
>>>>> >>>>>> Annotator['xpath-range'] = xpath-range;
>>>>> >>>>>>
>>>>> >>>>>> Please start a repository and then we can look into the real
>>>>> code you
>>>>> >>>>>> have. Now it is a bit too abstract which issues you and how to
>>>>> advise
>>>>> >>>>>> you.
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> Mitar
>>>>> >>>>>>
>>>>> >>>>>> On Fri, Sep 5, 2014 at 12:24 PM, Bryan <btbonval at gmail.com>
>>>>> wrote:
>>>>> >>>>>> > Thanks Mitar, a git submodule might be a good approach for us.
>>>>> >>>>>> >
>>>>> >>>>>> > One of our issues is with respect to the encoding of
>>>>> Annotator xpath
>>>>> >>>>>> > offsets. We would rather not duplicate this code, but we do
>>>>> need
>>>>> >>>>>> > access to
>>>>> >>>>>> > xpath-range. As Annotator works now, it appears to include
>>>>> >>>>>> > xpath-range via
>>>>> >>>>>> > browserify, but browserify does not expose xpath-range to the
>>>>> >>>>>> > greater Meteor
>>>>> >>>>>> > application.
>>>>> >>>>>> > -Bryan
>>>>> >>>>>> >
>>>>> >>>>>> >
>>>>> >>>>>> > On Fri, Sep 5, 2014 at 3:13 PM, Mitar <mmitar at gmail.com>
>>>>> wrote:
>>>>> >>>>>> >>
>>>>> >>>>>> >> Hi!
>>>>> >>>>>> >>
>>>>> >>>>>> >> Yes, we are still using the old version, but I am not sure it
>>>>> >>>>>> >> matters.
>>>>> >>>>>> >> You can package it in the same way, I believe. So you add
>>>>> Annotator
>>>>> >>>>>> >> as
>>>>> >>>>>> >> a git submodule, you ignore browserify, and just add all
>>>>> >>>>>> >> CoffeeScript
>>>>> >>>>>> >> files to the package. Meteor will then convert them into
>>>>> JavaScript
>>>>> >>>>>> >> files and package them into one namespace automatically.
>>>>> Sometimes
>>>>> >>>>>> >> you
>>>>> >>>>>> >> have just to add one more file which exposes globally the
>>>>> symbols
>>>>> >>>>>> >> you
>>>>> >>>>>> >> want to have globally.
>>>>> >>>>>> >>
>>>>> >>>>>> >> I don't have time to do it myself now, but you can start
>>>>> with a
>>>>> >>>>>> >> GitHub
>>>>> >>>>>> >> repository for it now and let's see where you get stuck.
>>>>> >>>>>> >>
>>>>> >>>>>> >>
>>>>> >>>>>> >> Mitar
>>>>> >>>>>> >>
>>>>> >>>>>> >> On Fri, Sep 5, 2014 at 5:16 AM, Andrew - FinalsClub
>>>>> >>>>>> >> <andrew at finalsclub.org> wrote:
>>>>> >>>>>> >> > Thanks Mitar,
>>>>> >>>>>> >> >
>>>>> >>>>>> >> > I notice you are using an older version of Annotator
>>>>> (before
>>>>> >>>>>> >> > browserify
>>>>> >>>>>> >> > was added). Would the same thing be possible with
>>>>> Annotator 2.0?
>>>>> >>>>>> >> > We are
>>>>> >>>>>> >> > currently a bit stumped. Ari can share more details on our
>>>>> >>>>>> >> > efforts if you're
>>>>> >>>>>> >> > curious.
>>>>> >>>>>> >> >
>>>>> >>>>>> >> > Eventually, it would be useful to make a Meteor package so
>>>>> others
>>>>> >>>>>> >> > can
>>>>> >>>>>> >> > easily use Annotator in their meteor projects. If you have
>>>>> any
>>>>> >>>>>> >> > suggestions,
>>>>> >>>>>> >> > we would love guidance / help.
>>>>> >>>>>> >> >
>>>>> >>>>>> >> > Warmly,
>>>>> >>>>>> >> > Andrew
>>>>> >>>>>> >> >
>>>>> >>>>>> >> >
>>>>> >>>>>> >> >
>>>>> >>>>>> >> >
>>>>> >>>>>> >> >> On Sep 5, 2014, at 3:39 AM, Mitar <mmitar at gmail.com>
>>>>> wrote:
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >> Hi!
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >> You can see here how we are bundling Annotator for Meteor:
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >>
>>>>> https://github.com/peerlibrary/peerlibrary/tree/development/packages/annotator
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >> And here how we are using it:
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >>
>>>>> https://github.com/peerlibrary/peerlibrary/tree/development/client/lib/annotator
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >> Mitar
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >> On Wed, Sep 3, 2014 at 7:04 AM, Andrew - FinalsClub
>>>>> >>>>>> >> >> <andrew at finalsclub.org> wrote:
>>>>> >>>>>> >> >>> Hey All,
>>>>> >>>>>> >> >>>
>>>>> >>>>>> >> >>> Especially @Mitar. At FinalsClub, we have been working
>>>>> on a new
>>>>> >>>>>> >> >>> social
>>>>> >>>>>> >> >>> reading site built using Meteor (a newfangled JavaScript
>>>>> >>>>>> >> >>> framework). Meteor
>>>>> >>>>>> >> >>> is nice, because it offers isomorphic JavaScript - that
>>>>> is, the
>>>>> >>>>>> >> >>> same code to
>>>>> >>>>>> >> >>> run on the client and server. It also provides the magic
>>>>> of
>>>>> >>>>>> >> >>> reactive data
>>>>> >>>>>> >> >>> binding between clients and server.
>>>>> >>>>>> >> >>>
>>>>> >>>>>> >> >>> We have gotten a basic working version of Annotator with
>>>>> a
>>>>> >>>>>> >> >>> meteor-specific store plugin. (I can share the code if
>>>>> anyone
>>>>> >>>>>> >> >>> is curious.).
>>>>> >>>>>> >> >>> However, there are some aspects of Annotator that we are
>>>>> >>>>>> >> >>> grappling with.
>>>>> >>>>>> >> >>>
>>>>> >>>>>> >> >>> For instance, we were wondering how the XPath-Range
>>>>> plugin is
>>>>> >>>>>> >> >>> currently being used. Also, we want to include the
>>>>> coffeescript
>>>>> >>>>>> >> >>> (not
>>>>> >>>>>> >> >>> compiled JS) of Annotator in our project, but aren't
>>>>> sure how
>>>>> >>>>>> >> >>> to accomplish
>>>>> >>>>>> >> >>> that using Meteor's custom package system.
>>>>> >>>>>> >> >>>
>>>>> >>>>>> >> >>> @Mitar, I've seen your handle on a few meteor github
>>>>> issues and
>>>>> >>>>>> >> >>> wonder
>>>>> >>>>>> >> >>> if could help us. I've cc'd Bryan and Ari from our team
>>>>> if they
>>>>> >>>>>> >> >>> have
>>>>> >>>>>> >> >>> anything to add.
>>>>> >>>>>> >> >>>
>>>>> >>>>>> >> >>> Also, if anyone else is interested in helping, just say
>>>>> so.
>>>>> >>>>>> >> >>> We'd be
>>>>> >>>>>> >> >>> glad to put our work on the OpenAnnotation organization
>>>>> if
>>>>> >>>>>> >> >>> you'd like as
>>>>> >>>>>> >> >>> well.
>>>>> >>>>>> >> >>>
>>>>> >>>>>> >> >>> Best,
>>>>> >>>>>> >> >>> Andrew
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >>
>>>>> >>>>>> >> >> --
>>>>> >>>>>> >> >> http://mitar.tnode.com/
>>>>> >>>>>> >> >> https://twitter.com/mitar_m
>>>>> >>>>>> >>
>>>>> >>>>>> >>
>>>>> >>>>>> >>
>>>>> >>>>>> >> --
>>>>> >>>>>> >> http://mitar.tnode.com/
>>>>> >>>>>> >> https://twitter.com/mitar_m
>>>>> >>>>>> >
>>>>> >>>>>> >
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> --
>>>>> >>>>>> http://mitar.tnode.com/
>>>>> >>>>>> https://twitter.com/mitar_m
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> _______________________________________________
>>>>> >>>>> annotator-dev mailing list
>>>>> >>>>> annotator-dev at lists.okfn.org
>>>>> >>>>> https://lists.okfn.org/mailman/listinfo/annotator-dev
>>>>> >>>>> Unsubscribe:
>>>>> https://lists.okfn.org/mailman/options/annotator-dev
>>>>> >>>>>
>>>>> >>>
>>>>> >>
>>>>> >
>>>>> > _______________________________________________
>>>>> > annotator-dev mailing list
>>>>> > annotator-dev at lists.okfn.org
>>>>> > https://lists.okfn.org/mailman/listinfo/annotator-dev
>>>>> > Unsubscribe: https://lists.okfn.org/mailman/options/annotator-dev
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> http://mitar.tnode.com/
>>>>> https://twitter.com/mitar_m
>>>>> _______________________________________________
>>>>> annotator-dev mailing list
>>>>> annotator-dev at lists.okfn.org
>>>>> https://lists.okfn.org/mailman/listinfo/annotator-dev
>>>>> Unsubscribe: https://lists.okfn.org/mailman/options/annotator-dev
>>>>>
>>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/annotator-dev/attachments/20140911/ab6841af/attachment-0004.html>

More information about the annotator-dev mailing list