[ckan-dev] Problems with ckanext-harvest

Wed Jan 11 14:49:15 UTC 2012

Hi Adrià,

thanks for fixing this issue! Currently, I don't have any problems with 'Group not found', but now I'm facing another problem with 'default_groups'.

Unfortunately, all packages won't be added to my default group. I tried API v1 (with name) and v2 (with ID). I don't see any errors, but afterwards all packages won't be in my group.

This is the JSON I used in my configuration field:

{
 "api_version":"1",
 "default_tags":["new-tag-1","new-tag-2"],
 "default_groups":["edu"],
 "default_extras":{"new_extra":"Test","harvest_url":"{harvest_source_url}/package/{dataset_id}"}
} 

'default_tags' and 'default_extras' are working.

Best,

Philipp

-----Ursprüngliche Nachricht-----
Von: Adrià Mercader [mailto:amercadero at gmail.com] 
Gesendet: Dienstag, 10. Januar 2012 18:57
An: CKAN Development Discussions
Cc: Lämmel, Philipp
Betreff: Re: [ckan-dev] Problems with ckanext-harvest

Hi again Philipp,

I've pushed some changes to the ckanext-harvest repository that fix the Groups Not found issue.
Would you mind updating your repository and see if the issues come up again?

Thanks,

Adrià

2012/1/10 Adrià Mercader <amercadero at gmail.com>:
> Hi Philipp,
>
> Thanks for this report, I can reproduce it and will have a look at it now.
> I will get back to you when I find the problem.
>
> Regards,
> Adrià
>
> On 10 January 2012 13:54, "Lämmel, Philipp"
> <philipp.laemmel at fokus.fraunhofer.de> wrote:
>> Hi all,
>>
>>
>>
>> I'm having some problems with the harvester extension:
>>
>>
>>
>> I made a clean install of CKAN 1.5 on my Ubuntu 10.04 LTS machine and 
>> installed afterwards the harvester extension according to the readme 
>> on https://github.com/okfn/ckanext-harvest .
>>
>> I added a new source (CKAN v1.4.2a) on localhost/harvest and started 
>> the job (again according to the readme).
>>
>>
>>
>> But every dataset, which is already in a group won't be imported to 
>> my local CKAN with the following error message:
>>
>>
>>
>> 2012-01-10 14:39:27,719 INFO  [ckanext] Received harvest object id:
>> 40589254-0c81-495e-a1cb-fe2dfb5891da
>>
>> 2012-01-10 14:39:27,725 DEBUG 
>> [ckanext.harvest.harvesters.ckanharvester] In CKANHarvester 
>> fetch_stage
>>
>> 2012-01-10 14:39:27,878 DEBUG 
>> [ckanext.harvest.harvesters.ckanharvester] In CKANHarvester 
>> import_stage
>>
>> 2012-01-10 14:39:27,899 INFO  [ckanext.harvest.harvesters.base] 
>> Package with GUID f069919a-f4cc-4700-bf9c-6de2354d737d does not 
>> exist, let's create it
>>
>> 2012-01-10 14:39:27,916 ERROR [ckanext.harvest.harvesters.base] 
>> global name 'NotFound' is not defined
>>
>> Traceback (most recent call last):
>>
>>   File
>> "/usr/local/extensions/ckanext-harvest/ckanext/harvest/harvesters/bas
>> e.py", line 147, in _create_or_update_package
>>
>>     new_package = get_action('package_create_rest')(context, 
>> package_dict)
>>
>>   File "/usr/lib/pymodules/python2.6/ckan/logic/action/create.py", 
>> line 240, in package_create_rest
>>
>>     check_access('package_create_rest', context, data_dict)
>>
>>   File "/usr/lib/pymodules/python2.6/ckan/logic/__init__.py", line 
>> 129, in check_access
>>
>>     logic_authorization = new_authz.is_authorized(action, context,
>> data_dict)
>>
>>   File "/usr/lib/pymodules/python2.6/ckan/new_authz.py", line 16, in 
>> is_authorized
>>
>>     return auth_function(context, data_dict)
>>
>>   File "/usr/lib/pymodules/python2.6/ckan/logic/auth/create.py", line 
>> 120, in package_create_rest
>>
>>     return package_create(context, data_dict)
>>
>>   File "/usr/lib/pymodules/python2.6/ckan/logic/auth/create.py", line 
>> 16, in package_create
>>
>>     check2 = check_group_auth(context,data_dict)
>>
>>   File "/usr/lib/pymodules/python2.6/ckan/logic/auth/create.py", line 
>> 100, in check_group_auth
>>
>>     raise NotFound(_('Group was not found.'))
>>
>> NameError: global name 'NotFound' is not defined
>>
>>
>>
>>
>>
>> But the groups are present on my CKAN 1.5 (same name and title). 
>> Where is my mistake?
>>
>>
>>
>> Additionally, I tried harvesting thedatahub.org, to rule out problems 
>> with different CKAN versions, but I had the same problem. All 
>> packages, which aren't in a group, can be imported, but all other will be left out.
>>
>>
>>
>> Did I miss any settings?
>>
>>
>>
>>
>> Regards,
>>
>> Philipp
>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>