[ckan-dev] Tranformations

[Stefan Oderbolz]

Hi David,

this sounds very interesting, yet so familiar as we are currently thinking
about something like that. Our goal is to be able to compare datasets with
each other, to give users the possibility to combine different datasets
together (aggregation). The idea would be to have some kind of
meta-metadata (annotations) to describe datasets, and then allow users to
play around with all datasets that have such "annotations". Finally it
should be possible to create new datasets by combining already existing
ones by simply defining the needed transformations.

This is still in a very early phase and we are currently exploring the
possibilities. But if there are already plans to build such a system in
CKAN, it might make sense to work together.

Regards Stefan


On Wed, Nov 27, 2013 at 12:26 PM, David Raznick <david.raznick at okfn.org>wrote:

> Hello
>
> Agreed to all these points and that is the long term objective of this.
>
> Initially however, because we do not know exactly how the
> api/interfaces will work yet, and will only allow the datapusher for
> quicker development and faster iteration.
>
> However, the whole point is to keep this extensibility in mind while
> working on this so will will not find it difficult to add those
> interfaces later.
>
> Thanks
>
> David
>
>
>
> On 27 November 2013 11:15, Philippe Duchesne <pduchesne at gmail.com> wrote:
> > Hello David,
> >
> > this will be a great feature, glad to read that.
> >
> > One comment: it is mentioned that it will rely on an external service
> > (datapusher at first). Will the HTTP interface of such a service be
> > specified, so that third-party services can be easily developed/ported?
> And
> > even better, have the ability to register different services for
> different
> > transformation types?
> >
> > my 2c,
> >
> > --p.
> >
> >
> > On Wed, Nov 27, 2013 at 11:36 AM, David Raznick <david.raznick at okfn.org>
> > wrote:
> >>
> >> Hello All
> >>
> >> We are thinking of adding some basic data precessing (ETL)
> >> functionality to CKAN.   This will initially start off very small and
> >> bascially used for correcting incorrectly detected types when using
> >> the datapusher but could be used for many more things later on.
> >>
> >> I have written a basic wiki page for the initial thoughts on this
> >>
> >> https://github.com/okfn/ckan/wiki/CKAN-Transformations
> >>
> >>
> >> Any comments welcome
> >>
> >> David
> >> _______________________________________________
> >> ckan-dev mailing list
> >> ckan-dev at lists.okfn.org
> >> http://lists.okfn.org/mailman/listinfo/ckan-dev
> >> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> >
> >
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> >
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>



-- 
Liip AG  //  Feldstrasse 133 //  CH-8004 Zurich
Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20131202/98a2cf79/attachment-0002.html>