[ckan-dev] Continuous error: ckan.logic.ValidationError: None - {'offset': [u'Invalid integer']} - SOLVED (?)
lucia.espona at wsl.ch
lucia.espona at wsl.ch
Mon Nov 28 12:02:50 UTC 2016
Dear Stefan
Ok, thanks, I just wanted to know if there may be a unwanted side-effect by filtering the offset values.
The error mails were mainly caused by google bot trying to get wrong URL's, that generates tons of them over time.
In case someone wants to do this filtering, it took me while to set the filter for organizations because the URL route is defined in two places:
https://github.com/espona/ckan/blob/master/ckan/lib/plugins.py#L193
in addition to the place where all the others are defined:
https://github.com/espona/ckan/blob/master/ckan/config/routing.py#L319
Best,
Lucia
_________________________________________________________
Dr. Lucia Espona Pernas
Swiss Federal Institute for Forest, Snow and Landscape Research WSL
Hauptgebäaude Labortrakt (HL C21)
Zürcherstrasse 111
8903 Birmensdorf
Switzerland
+41 44 739 28 71 phone direct
+41 44 739 21 11 reception
www.wsl.ch
-----"ckan-dev" <ckan-dev-bounces at lists.okfn.org> wrote: -----
To: CKAN Development Discussions <ckan-dev at lists.okfn.org>
From: Stefan Oderbolz
Sent by: "ckan-dev"
Date: 28.11.2016 12:00
Subject: Re: [ckan-dev] Continuous error: ckan.logic.ValidationError: None - {'offset': [u'Invalid integer']} - SOLVED (?)
Hi Lucia,
I guess just laziness ;)
Usually these mails are no big deal, it's more a warning than an actually error. I mean nothing really bad happens, and it's usually just an idicator of a bad practice, just like you discovered now. The more these are fixed, the better.
- Stefan
On Fri, Nov 25, 2016 at 1:32 PM, <lucia.espona at wsl.ch> wrote:
Dear Stefan
I addition to fixing the URL in the config I also modified the ckan/config/routing.py so that the offset has to be a digit.
This way I do not get a server error every time a wrong url is requested for any reason.
A regexp was added to all the url's where an offset is required. For example:
m.connect('/dataset/activity/{id}/{offset:\d*}', action='activity')
Is there any reason why this filtering was not being done?
Best
Lucia
_________________________________________________________
Dr. Lucia Espona Pernas
Swiss Federal Institute for Forest, Snow and Landscape Research WSL
Hauptgebäaude Labortrakt (HL C21)
Zürcherstrasse 111
8903 Birmensdorf
Switzerland
+41 44 739 28 71 phone direct
+41 44 739 21 11 reception
www.wsl.ch
-----"ckan-dev" <ckan-dev-bounces at lists.okfn.org> wrote: -----
To: CKAN Development Discussions <ckan-dev at lists.okfn.org>
From: lucia.espona at wsl.ch
Sent by: "ckan-dev"
Date: 25.11.2016 12:05
Subject: Re: [ckan-dev] Continuous error: ckan.logic.ValidationError: None - {'offset': [u'Invalid integer']} - SOLVED
Dear Stefan
Thanks a lot for looking so much into that, it is really nice of you.
I am surprised about what happens because I haven't done any modification to the activity stream related templates.
In addition, in the development instance I never get such an error.
To achieve what you propose, I changed in the configuration the value of the favicon to contain the full URL:
#ckan.favicon = envidat-logo-icon.ico
ckan.favicon = http://www.envidat.ch/envidat-logo-icon.ico
Now the links are properly created:
<link rel="shortcut icon" href="http://www.envidat.ch/envidat-logo-icon.ico">
I haven't got the error again so I assume that was triggering the ValidationErrors as you said.
Best,
Lucia
_________________________________________________________
Dr. Lucia Espona Pernas
Swiss Federal Institute for Forest, Snow and Landscape Research WSL
Hauptgebäaude Labortrakt (HL C21)
Zürcherstrasse 111
8903 Birmensdorf
Switzerland
+41 44 739 28 71 phone direct
+41 44 739 21 11 reception
www.wsl.ch
-----"ckan-dev" <ckan-dev-bounces at lists.okfn.org> wrote: -----
To: CKAN Development Discussions <ckan-dev at lists.okfn.org>
From: Stefan Oderbolz
Sent by: "ckan-dev"
Date: 24.11.2016 17:14
Subject: Re: [ckan-dev] Continuous error: ckan.logic.ValidationError: None - {'offset': [u'Invalid integer']}
Hi Lucia,
this is an interesting case:
if you look at the error mail you see the CKAN_CURRENT_URL:Â /organization/activity/cryos/envidat-logo-icon.ico
The envidat-logo-icon.ico indicates that it is probably the favicon, and a quick look on envidat.ch confirms that.
Now the error you get is cause by the activity stream of an organisation page, in your case it's this one:Â http://www.envidat.ch/organization/activity/cryos/0
The "0" at the end is the "offset" parameter, which is used if you want to load multiple pages of the activity stream, by default the first page is loaded.
You are loading the favicon like that on every page (including the activity stream):<link rel="shortcut icon" href="envidat-logo-icon.ico" />
This is a relative URL and causes every visitor of the activity stream to load the following URL:
http://www.envidat.ch/organization/activity/cryos/envidat-logo-icon.ico
CKAN then tries to load a page with the offset "envidat-logo-icon.ico", which of course is not a valid integer offset.
To fix this, you should change the favicon HTML to this:<link rel="shortcut icon" href="https://webnotes.wsl.ch/envidat-logo-icon.ico" />
Best Regards Stefan
On Thu, Nov 24, 2016 at 4:41 PM, <lucia.espona at wsl.ch> wrote:
Dear all
I since I did a production deploy of CKAN 2.5.2, I get a ValidationError continuously regarding "offset".
Here the error:
Module ckan.controllers.group:864 in activity
>>Â context, {'id': c.group_dict['id'], 'offset': offset})
Module ckan.logic:416 in wrapped
>>Â result = _action(context, data_dict, **kw)
Module ckan.logic.action.get:2831 in organization_activity_list_html
>>ÿ activity_stream = organization_activity_list(context, data_dict)
Module ckan.logic:501 in wrapper
>>ÿ raise ValidationError(errors)
ValidationError: None - {'offset': [u'Invalid integer']}
Sometimes I get different values for offset thet look complete non sense with the same error. The mail from the server with the details is pasted below.
Does anyone have an idea where it comes from?
Thanks a lot,
Lucia
Subject: WebApp Error: <class 'ckan.logic.ValidationError'>: None - {'offset': [u'Invalid integer']}
Module ckan.controllers.group:864 in activity
<<ÿ ÿÿÿÿÿÿÿ # template context for the group/read.html template to retrieve later.
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ c.group_activity_stream = self._action('group_activity_list_html')(
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ context, {'id': c.group_dict['id'], 'offset': offset})
ÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ return render(self._activity_template(group_type),
>>ÿ context, {'id': c.group_dict['id'], 'offset': offset})
Module ckan.logic:416 in wrapped
<<ÿ ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ # check_access(action_name, context, data_dict=None)
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ result = _action(context, data_dict, **kw)
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ try:
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ audit = context['__auth_audit'][-1]
>>ÿ result = _action(context, data_dict, **kw)
Module ckan.logic.action.get:2831 in organization_activity_list_html
<<ÿ ÿÿÿ '''
ÿÿÿÿÿÿ ÿÿÿ activity_stream = organization_activity_list(context, data_dict)
ÿÿÿÿÿÿ ÿÿÿ offset = int(data_dict.get('offset', 0))
ÿÿÿÿÿÿ ÿÿÿ extra_vars = {
>>ÿ activity_stream = organization_activity_list(context, data_dict)
Module ckan.logic:501 in wrapper
<<ÿ ÿÿÿÿÿÿÿÿÿÿÿ data_dict, errors = _validate(data_dict, schema, context)
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ if errors:
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ raise ValidationError(errors)
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ return action(context, data_dict)
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ return wrapper
>>ÿ raise ValidationError(errors) ValidationError: None - {'offset': [u'Invalid integer']}
CGI Variables AUTH_TYPE 'cookie' CKAN_CURRENT_URL '/organization/activity/cryos/envidat-logo-icon.ico' CKAN_LANG 'en' CKAN_LANG_IS_DEFAULT True CONTENT_TYPE '; charset=utf-8' CONTEXT_DOCUMENT_ROOT '/var/www/html' DOCUMENT_ROOT '/var/www/html' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING 'gzip, deflate' HTTP_ACCEPT_LANGUAGE 'en-GB,en;q=0.5' HTTP_CONNECTION 'close' HTTP_COOKIE 'auth_tkt="!userid_type:unicode"; auth_tkt="!userid_type:unicode"' HTTP_HOST 'www.envidat.ch' HTTP_USER_AGENT 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0' HTTP_X_FORWARDED_FOR '193.134.202.20' HTTP_X_FORWARDED_HOST 'www.envidat.ch' HTTP_X_FORWARDED_SERVER 'www.envidat.ch' PATH_INFO '/organization/activity/cryos/envidat-logo-icon.ico' PATH_TRANSLATED '/etc/ckan/default/apache.wsgi/organization/activity/cryos/envidat-logo-icon.ico' REMOTE_ADDR '127.0.0.1' REMOTE_PORT '50136' REMOTE_USER u'admin' REMOTE_USER_DATA 'userid_type:unicode' REMOTE_USER_TOKENS [''] REQUEST_METHOD 'GET' REQUEST_SCHEME 'http' REQUEST_URI '/organization/activity/cryos/envidat-logo-icon.ico' SCRIPT_FILENAME '/etc/ckan/default/apache.wsgi' SERVER_ADDR '127.0.0.1' SERVER_ADMIN '[no address given]' SERVER_NAME 'www.envidat.ch' SERVER_PORT '80' SERVER_PROTOCOL 'HTTP/1.0' SERVER_SIGNATURE '<address>Apache/2.4.18 (Ubuntu) Server at www.envidat.ch Port 80</address>\n' SERVER_SOFTWARE 'Apache/2.4.18 (Ubuntu)' WSGI Variables apache.version (2, 4, 18) application <fanstatic.publisher.Delegator object at 0x7f8aa3a68490> beaker.cache <beaker.cache.CacheManager object at 0x7f8aa3a68550> beaker.get_session <bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x7f8aa3a65a90>> beaker.session {'_accessed_time': 1479995919.195716, '_creation_time': 1479995919.195716} fanstatic.needed <fanstatic.core.NeededResources object at 0x7f8aa1a05c10> mod_wsgi.application_group 'envidat03.wsl.ch|' mod_wsgi.callable_object 'application' mod_wsgi.daemon_connects '1' mod_wsgi.daemon_restarts '0' mod_wsgi.daemon_start '1479995919070143' mod_wsgi.enable_sendfile '0' mod_wsgi.handler_script '' mod_wsgi.input_chunked '0' mod_wsgi.listener_host '' mod_wsgi.listener_port '8080' mod_wsgi.process_group 'ckan_default' mod_wsgi.queue_start '1479995919069985' mod_wsgi.request_handler 'wsgi-script' mod_wsgi.request_start '1479995919068370' mod_wsgi.script_reloading '1' mod_wsgi.script_start '1479995919070178' mod_wsgi.version (4, 3, 0) paste.cookies
paste.registry <paste.registry.Registry object at 0x7f8aa1bb5090> paste.throw_errors True pylons.action_method <bound method OrganizationController.activity of <ckan.controllers.organization.OrganizationController object at 0x7f8aa181cad0>> pylons.controller <ckan.controllers.organization.OrganizationController object at 0x7f8aa181cad0> pylons.environ_config {'session': 'beaker.session', 'cache': 'beaker.cache'} pylons.pylons <pylons.util.PylonsContext object at 0x7f8aa181c3d0> pylons.routes_dict {'action': u'activity', 'controller': u'organization', 'id': u'cryos', 'offset': u'envidat-logo-icon.ico'} repoze.who.api <repoze.who.api.API object at 0x7f8aa1bb5410> repoze.who.identity <repoze.who identity (hidden, dict-like) at 140233395059384> repoze.who.logger <logging.Logger object at 0x7f8aa3a68750> repoze.who.plugins
routes.route <routes.route.Route object at 0x7f8aa457cf10> routes.url <routes.util.URLGenerator object at 0x7f8aa1a05510> webob._parsed_query_vars (GET([]), '') webob.adhoc_attrs {'response': <Response at 0x7f8aa1a05f50 200 OK>, 'language': 'en-us'} wsgi process 'Multi process AND threads (?)' wsgi.file_wrapper <type 'mod_wsgi.FileWrapper'> wsgiorg.routing_args (<routes.util.URLGenerator object at 0x7f8aa1a05510>, {'action': u'activity', 'controller': u'organization', 'id': u'cryos', 'offset': u'envidat-logo-icon.ico'})
_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
https://lists.okfn.org/mailman/listinfo/ckan-dev
Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
--
Liip AG ÿ//ÿLimmatstrasse 183ÿ// ÿCH-8005 Zrich
Tel +41 43 500 39 80 // GnuPG 0x7B588C67 //ÿwww.liip.ch
_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
https://lists.okfn.org/mailman/listinfo/ckan-dev
Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
https://lists.okfn.org/mailman/listinfo/ckan-dev
Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
https://lists.okfn.org/mailman/listinfo/ckan-dev
Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
--
Liip AG ÿ//ÿLimmatstrasse 183ÿ// ÿCH-8005 Zrich
Tel +41 43 500 39 80 // GnuPG 0x7B588C67 //ÿwww.liip.ch
_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
https://lists.okfn.org/mailman/listinfo/ckan-dev
Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20161128/0cf07f2e/attachment-0003.html>
More information about the ckan-dev
mailing list