[ckan-dev] ckanext-importlib

Tue Jul 30 17:08:06 UTC 2013

Here's a bit more about ckanext-importlib below. I'll add this to the readme.

Dave

One-off imports of metadata into CKAN isn't very hard, and using
ckanclient directly is probably best for that. But when you are
continuously importing you have some challenges which this library
aims to help with:

* when you reimport a dataset you want to check if it already exists
in CKAN, using an ID stored in an extra field and possibly another
extra field naming the source

* you may import resources, which become grouped into datasets (e.g.
time series data) - ResourceSeriesLoader

* when you derive a unique name for a dataset from its title, you need
to avoid clashes.

ckanext-importlib was designed as a framework to be expanded, based on
the needs of the data.gov.uk ONS importer. But TBH it is not so
flexible. But even if you don't use it, you might want to steal stuff
from it.

On 29 July 2013 21:02, Ryan Maine <balrogmi at msn.com> wrote:
> Thanks a lot Adrian
>
> El 29/07/2013 18:20, "Adrià Mercader" <adria.mercader at okfn.org> escribió:
>
>> Hi Ryan,
>>
>> I don't know much about importlib myself, but looks like it is a
>> client for bulk importing datases through the API, similar to other
>> CKAN clients.
>> You can ask the commiters if you want more details about it.
>>
>> Adrià
>>
>> On 25 July 2013 17:21, Ryan Maine <balrogmi at msn.com> wrote:
>> > Hi!
>> >
>> > Maybe this is a silly question. What is ckanext-importlib used for?
>> > README file says that it allows for dataset importing using the API.
>> > But... why do I need this library to import datasets using the
>> > library? What's the advantage?
>> >
>> > Thanks!
>> >
>> > _______________________________________________
>> > ckan-dev mailing list
>> > ckan-dev at lists.okfn.org
>> > http://lists.okfn.org/mailman/listinfo/ckan-dev
>> > Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>