[data-protocols] SLEEP / The Cut-Out

Thu Jan 31 18:40:37 GMT 2013

On Thu, Jan 31, 2013 at 12:08 PM, Rufus Pollock <rufus.pollock at okfn.org>wrote:

> [...]
> > The result is:
> >
> > {objects: [
> >   [INDEX1,
> >    {type: "object_type",
> >     id: "unique id for objects of this type",
> >     data: {unstructured JSON data}
> >    }
> >   ], ...]}
> >
> > And to save updates:
> >
> > POST /bucket?since=INDEX&collection_id=ABCDEF
> > [{type: "object_type", id: "some id", data: {...}}, ...]
>
> I guess my question here (and one I had about SLEEP etc) is that a
> sync protocol *on its own* without information on e.g. generation of
> change objects, and the merge and the diff format is only so useful
> (and one is avoiding the hard part ;-) )
>
> For example, re SLEEP, a natural use case was using this with say 2
> sqlite databases. In that case the hard work would be generating the
> changes from the sqlite db
>
> Thus, I'm wondering if you have an example of using this protocol live
> to do syncing and, if so, how the sync side integrated with merging
> (and possibly diffing). (cf
> http://www.dataprotocols.org/en/latest/revisioning-data.html)

The only full example I have at this point is this modest example:
https://github.com/ianb/thecutout/tree/master/examples/files with the sync
integration code here:
https://github.com/ianb/thecutout/blob/master/examples/files/files.js#L546-587(IndexedDB
and other stuff takes up the bulk of the code).  The example
doesn't really do merging, all changes just overwrite.

The expectation is that all merging happens in the client, not on the
server, and that everyone converges on a single timeline.  So if client A
uploads object X, and then client B downloads it, and then client A edits
and uploads a revision, and client B edits its revision: client B will try
to upload, it's since stamp on the POST will be out of date, it will get
the updates it missed (the edit to X), and it should integrate that state
into its local state.  Then it can try to upload its edits again.

Integrating those edits into local state might mean throwing client A's
edits away away, wiping out the local copy, doing some content-based
intelligent merge, or putting the object in a conflicted state that
includes both edits and providing the user some interface to eventually
resolve that conflict.  But whatever it is, it's up to the client to figure
it out – the server doesn't understand anything about the objects
themselves.

> > It's not a peer-to-peer system, the server is considered the canonical
> > source of truth.  Clients must get all updates and resolve conflicts
> before
>
> Can you sync from multiple sources or is it one-way?

The primary use case is multiple clients using one server.  Having multiple
servers is not really a problem, though if there is overlap in the objects
being saved to the two servers then I can imagine there may be problems you
could get an edit cycle – e.g., client A saves object X to server 1, client
B retrieves that object, edits it and saves it to server 2, client A
fetches it from server 2, edits and saves it to server 1, and so one, never
settling down.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/data-protocols/attachments/20130131/3c8eac66/attachment.htm>