[ckan-dev] Removal of dataproxy backend from data explorer

Fri Jan 4 16:42:27 UTC 2013

I'm not a big fan of the current DataProxy, but I agree with Rufus
that its removal / replacement should be progressive. Eg if we disable
it easily 70% of CSVs in the datahub will stop showing up if we
upgrade it to 2.0 (after upgrading the datastore the migration process
didn't work for all files, maybe this has improved so I may be wrong).
Adding the messytables type detection may be a quick fix for the viz
issues.

The main problem is that from the user point of view DataProxy and
DataStore don't mean any difference in terms of previeweing, both
allow a csv to be displayed with Recline. If some of the resources
that were working with other versions stop doing so it would look
strange.

I also think that we should go towards making easier and more
transparent to push stuff to the datastore. Datastore/er is confusing,
perhaps enabling the Datastore should enable datastorer automatically
by default.

Adrià

On 4 January 2013 13:42, Dominik Moritz <dominik.moritz at okfn.org> wrote:
> If you need a pointer. Nigel and me are working on this at https://github.com/okfn/ckan-importer-service.
>
> On 4 Jan 2013, at 13:58, Rufus Pollock <rufus.pollock at okfn.org> wrote:
>
>> I think we should think hard before doing this. Setting up the
>> DataStore (and the DataStorer) is some degree of extra work. Removing
>> support for DataProxy then removes some very useful functionality that
>> often "just works" out of the box.
>>
>> Type information could be added *very* easily to DataProxy once
>> messytables supports streaming [1] and that works gets integrated with
>> data converters: https://github.com/okfn/data-converters
>>
>> rufus
>>
>> [1]: https://github.com/okfn/messytables/issues/24
>>
>> On 3 January 2013 09:22, John Glover <john.glover at okfn.org> wrote:
>>> Hello,
>>>
>>> There was a suggestion that we disable the dataproxy backend in the data
>>> explorer in CKAN 2.0, as the lack of type information makes visualisation
>>> difficult. In order to preview tabular data it would therefore first have to
>>> be added to the new CKAN datastore. Are there any objections to this?
>>>
>>> Thanks,
>>> John
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>
>>
>>
>>
>> --
>> Co-Founder, Open Knowledge Foundation
>> Promoting Open Knowledge in a Digital Age
>> http://www.okfn.org/ - http://blog.okfn.org/
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev