[ckan-dev] ckan-harvest NoneType problem

Thu Mar 15 08:45:49 UTC 2012

Hi Adrià,

I'm using ckan 1.6.1a with the latest ckanext-harvest from github.
I found that the error is indicating, that ckan does not expect to be given
an id when creating a new package. When I remove the id like this in
ckanext-harvest base.py:

del package_dict['id']

importing works fine - but creates new packages every time the harvester is
run even if nothing changed - because obviously the id's from remote and
local don't match.

Do I have to change the ckan package - creating - process to give a remote
id? Or would you say it is better to change the lookup if the package
already exists?

Can I get the expected behaviour with other ckan / harvester versions?
Unfortunately it looks like we would have to use ckan 1.6.1 due to other
features we need.

Thanks!
michael

2012/3/14 Adrià Mercader <amercadero at gmail.com>

> Hi Michael,
>
> Sorry to hear you are having trouble with the harvesting extension.
> Can you specify which versions of CKAN and ckanext-harvest are you
> using?
>
> Thanks,
>
> Adrià
>
> On 14 March 2012 13:21, Michael Reichart <michael.reichart at gmail.com>
> wrote:
> > Hi,
> >
> > I managed to get rid of the NoneType-Error which seemed to result from
> some
> > harvester-packages that where from previous calls.
> >
> > But now I have the problem, that the fetching of every single dataset
> > results in an error:
> >
> > 2012-03-14 14:08:46,730 ERROR [ckanext.harvest.harvesters.base] Invalid
> > package with GUID zuzuege: {'id': [u'Das Eingabefeld id wurde nicht
> > erwartet.']}
> >
> > which basically means "didn't expect to get the input field 'id'"
> >
> > Any ideas why this could happen?? As I said, i try to harvest datasets
> from
> > ckan.data.linz.gv.at to a relatively clean new ckan-installation ...
> >
> > thanks
> > michael
> >
> >
> >
> > 2012/3/14 Michael Reichart <michael.reichart at gmail.com>
> >>
> >> Hi,
> >>
> >> I'm trying to use the ckan-harvest extension to harvest from a ckan
> (tried
> >> different ones - at the moment: ckan.data.linz.gv.at) to import in my
> local
> >> dev ckan.
> >>
> >> I wasn't able to import a single package since all failed with different
> >> errors
> >> (mainly: NotFound('group not found') or ValidationError (because
> of {'id':
> >> [u'Das Eingabefeld id wurde nicht erwartet.']})
> >>
> >> Because of this I updated to the last version from github. Now I'm
> >> permanently getting the error:
> >>
> >> AttributeError: 'NoneType' object has no attribute 'type'
> >>
> >> I mean, ok this is clearly a python problem - but am I the only one
> >> experiencing this? Am I missing some configuration? Maybe another python
> >> version? I'm using Python 2.6
> >> I also tried some of the harvesters vom ckanext-pdeu - but they are
> >> failing with similar errors - the 'NoneType' appears on different
> places in
> >> code - even if I fix some, it reappears elsewhere.
> >>
> >> Is there any sort of deeper documentation to ckan-harvest?
> >>
> >> Thanks in advance!
> >>
> >> Michael
> >>
> >
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ckan-dev
> >
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20120315/769e69de/attachment-0001.html>