[ckan-dev] ckan-harvest NoneType problem

Thu Mar 15 09:28:29 UTC 2012

Hi Michael,

It looks like a bug on ckanext-harvest. I'll have a look this morning
and come back to you as soon as I get more details.

Sorry about that,

Adrià

On 15 March 2012 08:45, Michael Reichart <michael.reichart at gmail.com> wrote:
> Hi Adrià,
>
> I'm using ckan 1.6.1a with the latest ckanext-harvest from github.
> I found that the error is indicating, that ckan does not expect to be given
> an id when creating a new package. When I remove the id like this in
> ckanext-harvest base.py:
>
> del package_dict['id']
>
> importing works fine - but creates new packages every time the harvester is
> run even if nothing changed - because obviously the id's from remote and
> local don't match.
>
> Do I have to change the ckan package - creating - process to give a remote
> id? Or would you say it is better to change the lookup if the package
> already exists?
>
> Can I get the expected behaviour with other ckan / harvester versions?
> Unfortunately it looks like we would have to use ckan 1.6.1 due to other
> features we need.
>
> Thanks!
> michael
>
>
> 2012/3/14 Adrià Mercader <amercadero at gmail.com>
>>
>> Hi Michael,
>>
>> Sorry to hear you are having trouble with the harvesting extension.
>> Can you specify which versions of CKAN and ckanext-harvest are you
>> using?
>>
>> Thanks,
>>
>> Adrià
>>
>> On 14 March 2012 13:21, Michael Reichart <michael.reichart at gmail.com>
>> wrote:
>> > Hi,
>> >
>> > I managed to get rid of the NoneType-Error which seemed to result from
>> > some
>> > harvester-packages that where from previous calls.
>> >
>> > But now I have the problem, that the fetching of every single dataset
>> > results in an error:
>> >
>> > 2012-03-14 14:08:46,730 ERROR [ckanext.harvest.harvesters.base] Invalid
>> > package with GUID zuzuege: {'id': [u'Das Eingabefeld id wurde nicht
>> > erwartet.']}
>> >
>> > which basically means "didn't expect to get the input field 'id'"
>> >
>> > Any ideas why this could happen?? As I said, i try to harvest datasets
>> > from
>> > ckan.data.linz.gv.at to a relatively clean new ckan-installation ...
>> >
>> > thanks
>> > michael
>> >
>> >
>> >
>> > 2012/3/14 Michael Reichart <michael.reichart at gmail.com>
>> >>
>> >> Hi,
>> >>
>> >> I'm trying to use the ckan-harvest extension to harvest from a ckan
>> >> (tried
>> >> different ones - at the moment: ckan.data.linz.gv.at) to import in my
>> >> local
>> >> dev ckan.
>> >>
>> >> I wasn't able to import a single package since all failed with
>> >> different
>> >> errors
>> >> (mainly: NotFound('group not found') or ValidationError (because
>> >> of {'id':
>> >> [u'Das Eingabefeld id wurde nicht erwartet.']})
>> >>
>> >> Because of this I updated to the last version from github. Now I'm
>> >> permanently getting the error:
>> >>
>> >> AttributeError: 'NoneType' object has no attribute 'type'
>> >>
>> >> I mean, ok this is clearly a python problem - but am I the only one
>> >> experiencing this? Am I missing some configuration? Maybe another
>> >> python
>> >> version? I'm using Python 2.6
>> >> I also tried some of the harvesters vom ckanext-pdeu - but they are
>> >> failing with similar errors - the 'NoneType' appears on different
>> >> places in
>> >> code - even if I fix some, it reappears elsewhere.
>> >>
>> >> Is there any sort of deeper documentation to ckan-harvest?
>> >>
>> >> Thanks in advance!
>> >>
>> >> Michael
>> >>
>> >
>> >
>> > _______________________________________________
>> > ckan-dev mailing list
>> > ckan-dev at lists.okfn.org
>> > http://lists.okfn.org/mailman/listinfo/ckan-dev
>> >
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>