[ckan-dev] Issues with adding a custom input field

Adrià Mercader adria.mercader at okfn.org
Fri Jan 11 09:58:41 UTC 2013 

Hi Fabian,

I managed to get some time to look at your extension, please see
attached diff. This is an area where development is being made so your
feedback is much appreciated.

There were a couple of issues:

* form_to_db_schema and db_to_form_schema were using the same schema
(and an outdated one)
* The current implementation of IDatasetForm means that if your plugin
is extending DefaultDatasetForm you need to also include
form_to_db_schema_options, which gets called first (If you don't add
it, the default will be used and your custom schema will be ignored.
(Maybe we need to review this logic as it is a bit confusing)

This will correctly store and retrieve your custom field from the
database, but I'm afraid you'll find a new issue. As your field ends
with "_date", there is a known bug with the Solr indexing that raises
an error when saving the dataset. There is a fix under way, but on the
meantime you'll need to either rename your field or remove the *_date
dynamic field from the Solr schema.

https://github.com/okfn/ckan/issues/267


Hope this helps,

Adrià





On 8 January 2013 14:09, Kirstein, Fabian
<fabian.kirstein at fokus.fraunhofer.de> wrote:
> Hi Adrià,
>
> I've corrected the typo, but it makes no difference.
> I attached the traceback.
>
> Thanks again.
>
> URL: http://ubuntu:5000/dataset/amazon-s3
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/weberror/evalexception.py', line 431 in respond
>   app_iter = self.application(environ, detect_start_response)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
>   resp = self.call_func(req, *args, **self.kwargs)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
>   return self.func(req, *args, **kwargs)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/fanstatic/publisher.py', line 234 in __call__
>   return request.get_response(self.app)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
>   application, catch_exc_info=False)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
>   app_iter = application(self.environ, start_response)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
>   resp = self.call_func(req, *args, **self.kwargs)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
>   return self.func(req, *args, **kwargs)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/fanstatic/injector.py', line 54 in __call__
>   response = request.get_response(self.app)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
>   application, catch_exc_info=False)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
>   app_iter = application(self.environ, start_response)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/beaker/middleware.py', line 73 in __call__
>   return self.app(environ, start_response)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/beaker/middleware.py', line 155 in __call__
>   return self.wrap_app(environ, session_start_response)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/routes/middleware.py', line 131 in __call__
>   response = self.app(environ, start_response)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/pylons/wsgiapp.py', line 125 in __call__
>   response = self.dispatch(controller, environ, start_response)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/pylons/wsgiapp.py', line 324 in dispatch
>   return controller(environ, start_response)
> File '/home/fki/pyenv/src/ckan/ckan/lib/base.py', line 291 in __call__
>   res = WSGIController.__call__(self, environ, start_response)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/pylons/controllers/core.py', line 221 in __call__
>   response = self._dispatch_call()
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/pylons/controllers/core.py', line 172 in _dispatch_call
>   response = self._inspect_call(func)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/pylons/controllers/core.py', line 107 in _inspect_call
>   result = self._perform_call(func, args)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/pylons/controllers/core.py', line 60 in _perform_call
>   return func(**args)
> File '/home/fki/pyenv/src/ckan/ckan/controllers/package.py', line 327 in read
>   return render(template, loader_class=loader)
> File '/home/fki/pyenv/src/ckan/ckan/lib/base.py', line 194 in render
>   loader_class=loader_class)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/pylons/templating.py', line 249 in cached_template
>   return render_func()
> File '/home/fki/pyenv/src/ckan/ckan/lib/base.py', line 132 in render_template
>   return render_jinja2(template_name, globs)
> File '/home/fki/pyenv/src/ckan/ckan/lib/base.py', line 85 in render_jinja2
>   return template.render(**extra_vars)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/jinja2/environment.py', line 894 in render
>   return self.environment.handle_exception(exc_info, True)
> File '/home/fki/pyenv/src/ckanext-addfield/ckanext/field/templates/package/read.html', line 3 in top-level template code
>   {% set pkg = c.pkg_dict %}
> File '/home/fki/pyenv/src/ckan/ckan/templates/page.html', line 1 in top-level template code
>   {% extends "base.html" %}
> File '/home/fki/pyenv/src/ckan/ckan/templates/base.html', line 104 in top-level template code
>   {%- block page %}{% endblock -%}
> File '/home/fki/pyenv/src/ckan/ckan/templates/page.html', line 15 in block "page"
>   {%- block content %}
> File '/home/fki/pyenv/src/ckan/ckan/templates/page.html', line 18 in block "content"
>   {% block main_content %}
> File '/home/fki/pyenv/src/ckan/ckan/templates/page.html', line 30 in block "main_content"
>   {% block toolbar %}
> File '/home/fki/pyenv/src/ckan/ckan/templates/page.html', line 40 in block "toolbar"
>   {% block actions %}
> File '/home/fki/pyenv/src/ckan/ckan/templates/page.html', line 42 in block "actions"
>   {% block actions_content %}{% endblock %}
> File '/home/fki/pyenv/src/ckanext-addfield/ckanext/field/templates/package/read.html', line 16 in block "actions_content"
>   {% if h.check_access('package_update', {'id':pkg.id }) %}
> File '/home/fki/pyenv/src/ckan/ckan/lib/helpers.py', line 527 in check_access
>   check_access_logic(action, context, data_dict)
> File '/home/fki/pyenv/src/ckan/ckan/logic/__init__.py', line 207 in check_access
>   logic_authorization = is_authorized(action, context, data_dict)
> File '/home/fki/pyenv/src/ckan/ckan/new_authz.py', line 17 in is_authorized
>   return auth_function(context, data_dict)
> File '/home/fki/pyenv/src/ckan/ckan/logic/auth/update.py', line 15 in package_update
>   package = get_package_object(context, data_dict)
> File '/home/fki/pyenv/src/ckan/ckan/logic/auth/__init__.py', line 24 in get_package_object
>   return _get_object(context, data_dict, 'package', 'Package')
> File '/home/fki/pyenv/src/ckan/ckan/logic/auth/__init__.py', line 13 in _get_object
>   obj = getattr(model, class_name).get(id)
> File '/home/fki/pyenv/src/ckan/ckan/model/package.py', line 76 in get
>   pkg = query.first()
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py', line 2156 in first
>   ret = list(self[0:1])
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py', line 2023 in __getitem__
>   return list(res)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py', line 2227 in __iter__
>   return self._execute_and_instances(context)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py', line 2242 in _execute_and_instances
>   result = conn.execute(querycontext.statement, self._params)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py', line 1449 in execute
>   params)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py', line 1584 in _execute_clauseelement
>   compiled_sql, distilled_params
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py', line 1698 in _execute_context
>   context)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py', line 1691 in _execute_context
>   context)
> File '/home/fki/pyenv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py', line 331 in do_execute
>   cursor.execute(statement, parameters)
> ProgrammingError: (ProgrammingError) can't adapt type 'Undefined' 'SELECT package.id AS package_id, package.name AS package_name, package.title AS package_title, package.version AS package_version, package.url AS package_url, package.author AS package_author, package.author_email AS package_author_email, package.maintainer AS package_maintainer, package.maintainer_email AS package_maintainer_email, package.notes AS package_notes, package.license_id AS package_license_id, package.type AS package_type, package.state AS package_state, package.revision_id AS package_revision_id \nFROM package \nWHERE package.id = %(id_1)s ORDER BY package.name \n LIMIT %(param_1)s' {'id_1': Undefined, 'param_1': 1}
>
>
> -----Ursprüngliche Nachricht-----
> Von: ckan-dev-bounces at lists.okfn.org [mailto:ckan-dev-bounces at lists.okfn.org] Im Auftrag von Adrià Mercader
> Gesendet: Dienstag, 8. Januar 2013 10:43
> An: CKAN Development Discussions
> Betreff: Re: [ckan-dev] Issues with adding a custom input field
>
> Hi Fabian,
>
> You use "release_data" on form_to_db_schema and "release_date" on db_to_form_schema. Let's get this out of the way to make sure it's not affecting things.
>
> What ProgrammingError are you getting? Can you paste the full traceback?
>
>
> Adrià
>
> On 8 January 2013 08:07, Kirstein, Fabian <fabian.kirstein at fokus.fraunhofer.de> wrote:
>> Hi Adrià,
>>
>> thank you very much for having a look into my problem.
>> The solution you proposed is the same I've tried, but it just leads me
>> to an error message. I implemented just the basic idea and pushed it
>> to GitHub: https://github.com/FabiApfelkern/ckanext-addfield
>> If I want to view or edit a dataset I just get a " ProgrammingError".
>>
>> It would be absolutely great if you can have another look into it.
>> Fabian
>>
>>
>>
>> -----Ursprüngliche Nachricht-----
>> Von: ckan-dev-bounces at lists.okfn.org
>> [mailto:ckan-dev-bounces at lists.okfn.org] Im Auftrag von Adrià Mercader
>> Gesendet: Montag, 7. Januar 2013 17:17
>> An: CKAN Development Discussions
>> Betreff: Re: [ckan-dev] Issues with adding a custom input field
>>
>> Hi Fabian,
>>
>> When storing extra fields against the default CKAN schema, these get internally stored as extras, which can hold arbitrary key-value pairs.
>> You need to add the following converters to your custom schema on your form_to_db_schema and db_to_form_schema methods:
>>
>>
>> from ckan.logic.converters import convert_to_extras,
>> convert_from_extras
>>
>> class YourPlugin(IDatasetForm):
>>
>> def form_to_db_schema(self):
>>
>>  // get the default package schema
>>  //...
>>
>>    schema['release_date'] = [convert_to_extras] //You can add more
>> validators if needed
>>
>>    return schema
>>
>> def db_to_form_schema(self):
>>
>>  // get the default package schema
>>  //...
>>    schema['release_date'] = [convert_from_extras]
>>
>>    return schema
>>
>> After doing this, on additional_info.html, pkg_dict should have your extra fields (pkg_dict.release_date).
>>
>> Here is a real example of a custom schema that stores new fields (look for those with convert_to_extras):
>>
>> https://github.com/okfn/ckanext-harvest/blob/2.0-dataset-sources/ckane
>> xt/harvest/logic/schema.py
>>
>>
>> Hope this helps,
>>
>> Adrià
>>
>> On 7 January 2013 10:57, Kirstein, Fabian <fabian.kirstein at fokus.fraunhofer.de> wrote:
>>> Hello,
>>>
>>>
>>>
>>> do you need any additional information to answer my question? I can’t
>>> solve it by my own.
>>>
>>> I tried to save the inputted value with schema.update and
>>> convert_to_extras methods, but I
>>>
>>> am not sure how to apply it.
>>>
>>>
>>>
>>> Thanks!
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Von: ckan-dev-bounces at lists.okfn.org
>>> [mailto:ckan-dev-bounces at lists.okfn.org] Im Auftrag von Kirstein,
>>> Fabian
>>> Gesendet: Donnerstag, 3. Januar 2013 12:25
>>> An: ckan-dev at lists.okfn.org
>>> Betreff: [ckan-dev] Issues with adding a custom input field
>>>
>>>
>>>
>>> Hello everyone,
>>>
>>>
>>>
>>> this thread follows my „Customizing Forms in 2.0a“  thread. I have
>>> still issues adding some custom input fields, store the input and display it.
>>>
>>> I try to explain what to do. I’ve started to write my own custom
>>> extension implementing IDatasetForm . I’ve overwritten the
>>> package_metadata_fields.html and add
>>>
>>> an additional field, like this:
>>>
>>> {{ form.input('release_date', label=_('Release Date'),
>>> id='field-release-date', placeholder=_('2011-06-13'),
>>> value=data.release_date, error=errors.release_date,
>>> classes=['control-medium']) }}
>>>
>>> In the form_to_db_schema() method, I would like to store the value of
>>> the field as an extra field. I am not pretty sure how to access the
>>> inputted value.
>>>
>>> Then I want to override additional_info.html to display the stored value.
>>> But I’m not really sure how to pass data to such a snippet. The
>>> c-object which
>>>
>>> I can manipulate setup_template_variables() is not available in the snippet.
>>>
>>>
>>>
>>> I would really appreciate any help. I can’t be so hard to add a
>>> single input field, I guess I just have some misunderstandings.
>>>
>>> Thanks a lot.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: addfield.diff
Type: application/octet-stream
Size: 2049 bytes
Desc: not available
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130111/6891faf4/attachment-0003.obj>

More information about the ckan-dev mailing list