[ckan-dev] Harvester - Problem
Michael Reichart
michael.reichart at gmail.com
Sun Jan 20 21:57:49 UTC 2013
Hi,
maybe I'm understanding something totally wrong, but my problem seems to be
much more fundamental.
I've setup a clean CKAN 1.8.1b with ckanext-harvest (master branch from
19/01/2013) and organizations plugin enabled and ckan.auth.profile =
publisher. What I'm trying to do is to insert data with a harvester and add
all the datasets to a certain organization. But this doesn't work, even if
I use the standard ckan_harvester. If I add some logging, the package_dict
includes "groups" and "organizations" before creating the package, but they
seem to get lost there. Creating the package returns true.
I thought when using the organizations - extension the groups are used as
organizations. But when I try to add the datasets to the group, they are
not correctly added altough there are no errors in the harvesters view. How
do I have to set the correct organization? Are there any samples of this
configuration?
When I try to set ckan.harvest.auth.profile to "publisher" I have the
problem, that my admin user cannot add a publisher to the harvester (at
least the harvester-form tells me this) altough the admin user is member of
an organization. So I figured this may be because he's sysadmin, but with
any other user, no matter if editor or admin of the group I cannot open the
harvester administration and therefore also cannot add a publisher to the
harvester. Can anybody tell me how this is meant to use? Are there any
other roles I have to set? If yes, which and where do I find this
information? Does this solve my problem?
It's okay for me to have a different harvester or even different
harvester-users for each organization if this works ...
Thanks for your help, I really appreciate it!
Michael
2013/1/18 Michael Reichart <michael.reichart at gmail.com>
> Hi,
>
> I've tried to change my harvesting and made a new extension only for
> harvesting and didn't enable our other extension (with our custom schema)
> in the paster ini. But with this setup groups are also not correctly
> applied. Altough our custom scheme is not applied (our own
> form_to_db_scheme is not called).
>
> Now I've tried to use the publisher setting in ckan.harvest.auth.profile,
> but this gives me the error: "Please choose an organization".
> But in the harvester web-interface I get the message "Cannot add any
> publishers" - what do I make wrong? Does anybody have an example for a
> harvester with organizations and publisher-profile?
>
> Thanks for the help!
> Michael
>
>
> 2013/1/17 Michael Reichart <michael.reichart at gmail.com>
>
>> Hi,
>>
>> seems that my first email didn't come through the mailing-list:
>>
>> Hi,
>> datasets created and updated by the webinterface do not loose their
>> groups. I haven't tested the direct api-calls yet.
>> I can't share our full extension, but i've attached the full code of the
>> IDatasetForm
>> We're using a custom CKAN-Harvester (which only does some mapping to our
>> custom fields) based on ckanext-harvest from beginning of July 2012 could
>> be Version 1.6.12??
>>
>> From my understanding, it shouldn't be necessary to call our custom
>> schema to import from another CKAN as the data-objects are already in the
>> right form.
>>
>> Thanks!
>> Michael
>>
>>
>> class DgvatForm(SingletonPlugin):
>> implements(IRoutes)
>> implements(IConfigurer)
>> implements(IGenshiStreamFilter)
>> implements(IDatasetForm)
>>
>> log.fatal("Enter: %s" % __name__)
>> def package_form(self):
>> return 'package/datagvat_organization_form.html'
>>
>>
>> def before_map(self, map):
>> map.connect('/error/{action}',
>> controller='ckanext.dgvat_por.controllers.data_gv_at:DgvatErrorController')
>> map.connect('/error/{action}/{id}',
>> controller='ckanext.dgvat_por.controllers.data_gv_at:DgvatErrorController')
>>
>>
>> map.connect('/dataset/new',
>> controller='ckanext.dgvat_por.controllers.data_gv_at:DgvatPackageController',
>> action='new')
>> map.connect('/dataset/edit/{id}',
>> controller='ckanext.dgvat_por.controllers.data_gv_at:DgvatPackageController',
>> action='edit')
>> map.connect('/dataset/{id}.{format}',
>> controller='ckanext.dgvat_por.controllers.data_gv_at:DgvatPackageController',
>> action='read')
>> map.connect('/dataset/{id}',
>> controller='ckanext.dgvat_por.controllers.data_gv_at:DgvatPackageController',
>> action='read')
>> map.connect('/dataset/{id}/resource/{resource_id}',
>> controller='ckanext.dgvat_por.controllers.data_gv_at:DgvatPackageController',
>> action='resource_read')
>> map.connect('/dataset/{id}/resource/{resource_id}/embed',
>> controller='ckanext.dgvat_por.controllers.data_gv_at:DgvatPackageController',
>> action='resource_embedded_dataviewer')
>> map.connect('/dataset/editresources/{id}',
>> controller='ckanext.dgvat_por.controllers.data_gv_at:DgvatPackageController',
>> action='editresources')
>> map.connect('/user/edit/{id:.*}',
>> controller='ckanext.dgvat_por.controllers.dgvat_user:DgvatUserController',
>> action='edit')
>> #m.connect('/user/reset/{id:.*}', action='perform_reset')
>> map.connect('/user/register',
>> controller='ckanext.dgvat_por.controllers.dgvat_user:DgvatUserController',
>> action='register')
>> map.connect('/user/login',
>> controller='ckanext.dgvat_por.controllers.dgvat_user:DgvatUserController',
>> action='login')
>> map.connect('/user/_logout', '/user/logout',
>> controller='ckanext.dgvat_por.controllers.dgvat_user:DgvatUserController',
>> action='logout')
>> map.connect('/user/logged_in',
>> controller='ckanext.dgvat_por.controllers.dgvat_user:DgvatUserController',
>> action='logged_in')
>> map.connect('/user/logged_out',
>> controller='ckanext.dgvat_por.controllers.dgvat_user:DgvatUserController',
>> action='logged_out')
>> map.connect('/user/logged_out_redirect',
>> controller='ckanext.dgvat_por.controllers.dgvat_user:DgvatUserController',
>> action='logged_out_page')
>> #map.connect('/user/reset', action='request_reset')
>> #m.connect('/user/me', action='me')
>> map.connect('/user/set_lang/{lang}',
>> controller='ckanext.dgvat_por.controllers.dgvat_user:DgvatUserController',
>> action='set_lang')
>> #m.connect('/user/{id:.*}', action='read')
>> map.connect('/user',
>> controller='ckanext.dgvat_por.controllers.dgvat_user:DgvatUserController',
>> action='index')
>>
>> map.connect('home', '/',
>> controller='ckanext.dgvat_por.controllers.dgvat_cockpit:DgvatCockpitController',
>> action='search')
>> map.connect('about', '/about',
>> controller='ckanext.dgvat_por.controllers.dgvat_cockpit:DgvatCockpitController',
>> action='about')
>> map.connect('terms', '/terms',
>> controller='ckanext.dgvat_por.controllers.dgvat_cockpit:DgvatCockpitController',
>> action='terms')
>> map.connect('help', '/help',
>> controller='ckanext.dgvat_por.controllers.dgvat_cockpit:DgvatCockpitController',
>> action='help')
>> map.connect('register_api', '/register_api',
>> controller='ckanext.dgvat_por.controllers.dgvat_cockpit:DgvatCockpitController',
>> action='register')
>>
>> #log.fatal("==================================> %s" % map)
>> return map
>>
>>
>> def after_map(self, map):
>> #log.fatal("==================================> %s" % map)
>> return map
>>
>>
>> def update_config(self, config):
>> log.debug("update_config")
>> config['package_form'] = 'data_gv_at'
>> configure_template_directory(config, 'templates')
>> configure_public_directory(config, 'public')
>>
>>
>> def filter(self, stream):
>>
>> from pylons import request, tmpl_context as c
>> routes = request.environ.get('pylons.routes_dict')
>> log.debug(routes)
>> if routes and \
>> routes.get('controller') == 'package' and \
>> routes.get('action') == 'read' and c.pkg.id:
>>
>> # Add dataset id to the UI
>> stream = stream_filters.package_id_filter(stream, c.pkg)
>> return stream
>>
>>
>> def setup_template_variables(self, context, data_dict):
>> log.debug("setup_template_variables")
>>
>> def package_types(self):
>> return ['dataset']
>>
>> def is_fallback(self):
>> return True
>>
>> def form_to_db_schema(self):
>> log.fatal("Enter form to db!!!")
>> def add_to_extras(self, key, data, errors, context):
>> # get current number of extras
>> extras = data.get(('extras',), [])
>> if not extras:
>> data[('extras',)] = extras
>> #extras.append({'key': key[-1], 'value': data[key]})
>> #log.fatal("add to extras: %s, %s" % (key, data[key]))
>> #log.fatal("extra_number: %s" % extra_number)
>> #data[('extras', extra_number, 'key')] = key[0]
>> cats =""
>> allCat=[]
>> for cur in data[key]:
>> if cur != '[' and cur!=']':
>> cats += cur
>> for cat_desc, id in c.categorization:
>> if id in cats:
>> allCat.append(id)
>> log.fatal("added categorizations:id %s" % allCat)
>> log.fatal("cats, key: %s,%s" % (cats, data[key]))
>> log.fatal("is list: %s" % isinstance(data[key], list))
>> #data[('extras', extra_number, 'value')] = allCat
>> extras.append({'key': key[-1], 'value': allCat})
>> def read_template(self):
>> return 'package/read.html'
>>
>>
>> def check_data_dict(self, schema=None):
>> return
>>
>> def new_template(self):
>> return 'package/new.html'
>>
>> def comments_template(self):
>> return 'package/comments.html'
>>
>> def search_template(self):
>> return 'package/search.html'
>>
>> def read_template(self):
>> return 'package/read.html'
>>
>> def history_template(self):
>> return 'package/history.html'
>>
>> def package_form(self):
>> return 'package/datagvat_organization_form.html'
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130120/5f11fe8f/attachment-0001.html>
More information about the ckan-dev
mailing list