[ckan-dev] ckan-dev Digest, Vol 87, Issue 1
David Hernández Gutiérrez
davisoski at gmail.com
Wed Jan 3 14:18:41 UTC 2018
Hi, Matt.
Thanks for your comments!!
*Are you certain that log is coming at the time of the Internal Server
Error? I don't think the two are related.*
What I'm doing is:
- delete all content in /var/log/httpd
- restart server: systemctl restart http
- Load page
- Go to my dataset and add a new resource
- *The Internal Server Error appears*
- Refresh page
- The new resource is correctly added
In /var/log/httpd I get:
[Wed Jan 03 14:03:43.090223 2018] [:error] [pid 21500] 2018-01-03
14:03:43,089 INFO [ckan.lib.base] / render time 5.954 seconds
[Wed Jan 03 14:03:48.198780 2018] [:error] [pid 21501] 2018-01-03
14:03:48,198 INFO [ckan.lib.base] /api/i18n/es render time 0.012 seconds
[Wed Jan 03 14:03:52.058837 2018] [:error] [pid 21501] 2018-01-03
14:03:52,058 INFO [ckan.lib.base] /api/i18n/es render time 0.009 seconds
[Wed Jan 03 14:04:09.164814 2018] [:error] [pid 21500] 2018-01-03
14:04:09,164 WARNI [ckan.lib.maintain] Function _resource_preview() in
module ckan.controllers.package has been deprecated and will be removed in
a later release of ckan. Resource preview is deprecated. Please use the new
resource views
[Wed Jan 03 14:04:09.199736 2018] [:error] [pid 21500] 2018-01-03
14:04:09,199 WARNI [ckan.lib.maintain] Function _resource_preview() in
module ckan.controllers.package has been deprecated and will be removed in
a later release of ckan. Resource preview is deprecated. Please use the new
resource views
[Wed Jan 03 14:04:09.216164 2018] [:error] [pid 21500] 2018-01-03
14:04:09,215 WARNI [ckan.lib.maintain] Function _resource_preview() in
module ckan.controllers.package has been deprecated and will be removed in
a later release of ckan. Resource preview is deprecated. Please use the new
resource views
[Wed Jan 03 14:04:09.229743 2018] [:error] [pid 21500] 2018-01-03
14:04:09,229 WARNI [ckan.lib.maintain] Function _resource_preview() in
module ckan.controllers.package has been deprecated and will be removed in
a later release of ckan. Resource preview is deprecated. Please use the new
resource views
[Wed Jan 03 14:04:09.243469 2018] [:error] [pid 21500] 2018-01-03
14:04:09,243 WARNI [ckan.lib.maintain] Function _resource_preview() in
module ckan.controllers.package has been deprecated and will be removed in
a later release of ckan. Resource preview is deprecated. Please use the new
resource views
[Wed Jan 03 14:04:09.256915 2018] [:error] [pid 21500] 2018-01-03
14:04:09,256 WARNI [ckan.lib.maintain] Function _resource_preview() in
module ckan.controllers.package has been deprecated and will be removed in
a later release of ckan. Resource preview is deprecated. Please use the new
resource views
[Wed Jan 03 14:04:09.269974 2018] [:error] [pid 21500] 2018-01-03
14:04:09,269 WARNI [ckan.lib.maintain] Function _resource_preview() in
module ckan.controllers.package has been deprecated and will be removed in
a later release of ckan. Resource preview is deprecated. Please use the new
resource views
[Wed Jan 03 14:04:09.284840 2018] [:error] [pid 21500] 2018-01-03
14:04:09,284 WARNI [ckan.lib.maintain] Function _resource_preview() in
module ckan.controllers.package has been deprecated and will be removed in
a later release of ckan. Resource preview is deprecated. Please use the new
resource views
[Wed Jan 03 14:04:11.382815 2018] [:error] [pid 21501] 2018-01-03
14:04:11,382 INFO [ckan.lib.base] /api/i18n/es render time 0.008 seconds
[Wed Jan 03 14:04:14.721055 2018] [:error] [pid 21501] 2018-01-03
14:04:14,720 INFO [ckan.lib.base] /api/i18n/es render time 0.008 seconds
[Wed Jan 03 14:04:17.308348 2018] [:error] [pid 21501] 2018-01-03
14:04:17,308 INFO [ckan.lib.base] /api/i18n/es render time 0.008 seconds
[Wed Jan 03 14:04:19.468321 2018] [:error] [pid 21501] 2018-01-03
14:04:19,468 INFO [ckan.lib.base] /api/i18n/es render time 0.007 seconds
[Wed Jan 03 14:04:49.788517 2018] [:error] [pid 21500] 2018-01-03
14:04:49,788 INFO [ckan.lib.base]
/dataset/25dcfd31-bdb5-42ab-9c15-0bfd8ad2f478/resource/204f2d94-5c89-44cb-893c-8dea9abbe366/download/myresource.pdf
render time 0.032 seconds
[Wed Jan 03 14:04:49.851185 2018] [:error] [pid 21500] 2018-01-03
14:04:49,850 INFO [ckan.lib.base]
/dataset/25dcfd31-bdb5-42ab-9c15-0bfd8ad2f478/resource/204f2d94-5c89-44cb-893c-8dea9abbe366/download/myresource.pdf
render time 0.054 seconds
[Wed Jan 03 14:04:49.854777 2018] [:error] [pid 21500] [remote
172.19.153.13:52] mod_wsgi (pid=21500): Exception occurred processing WSGI
script '/etc/ckan/default/apache.wsgi'.
[Wed Jan 03 14:04:49.854903 2018] [:error] [pid 21500] [remote
172.19.153.13:52] TypeError: expected byte string object for header name,
value of type unicode found
[Wed Jan 03 14:04:57.747296 2018] [:error] [pid 21501] 2018-01-03
14:04:57,747 INFO [ckan.lib.base]
/api/2/util/resource/format_autocomplete render time 0.027 seconds
[Wed Jan 03 14:05:08.631256 2018] [:error] [pid 21500]
/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py:79:
SAWarning: Usage of the 'related attribute set' operation is not currently
supported within the execution stage of the flush process. Results may not
be consistent. Consider using alternative event listeners or
connection-level operations instead.
[Wed Jan 03 14:05:08.631296 2018] [:error] [pid 21500]
sess._flush_warning("related attribute set")
[Wed Jan 03 14:05:11.287437 2018] [:error] [pid 21500] 2018-01-03
14:05:11,287 INFO [ckan.lib.base] /dataset/new_resource/mydataset render
time 3.512 seconds
[Wed Jan 03 14:05:11.290799 2018] [:error] [pid 21500] [remote
172.19.153.13:52] mod_wsgi (pid=21500): Exception occurred processing WSGI
script '/etc/ckan/default/apache.wsgi'.
[Wed Jan 03 14:05:11.290876 2018] [:error] [pid 21500] [remote
172.19.153.13:52] TypeError: expected byte string object for header name,
value of type unicode found
About *datastore*, I'm *not *using this feature:
This is what I have in my config file (/etc/ckan/default/development.ini)
#ckan.datastore.write_url = postgresql://ckan_default:pass@localhost
/datastore_default
ckan.plugins = stats text_view image_view recline_view pdf_view
geojson_view datapreview dcat
ckan.views.default_views = image_view text_view recline_view pdf_view
geojson_view
One of your suggestions:
you could adjust the file types that datapusher tries to upload
automatically in the config file or temporarily disable the datapusher
plugin completely.
I'm not using *datapusher*
*## Datapusher settings*
*This is what I have in /etc/ckan/default/development.ini*
# Make sure you have set up the DataStore
#ckan.datapusher.formats = csv xls xlsx tsv application/csv
application/vnd.ms-excel
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
#ckan.datapusher.url = http://127.0.0.1:8800/
#ckan.datapusher.assume_task_stale_after = 3600
Thanks
2018-01-03 12:00 GMT+00:00 <ckan-dev-request at lists.okfn.org>:
> Send ckan-dev mailing list submissions to
> ckan-dev at lists.okfn.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> or, via email, send a message with subject or body 'help' to
> ckan-dev-request at lists.okfn.org
>
> You can reach the person managing the list at
> ckan-dev-owner at lists.okfn.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of ckan-dev digest..."
>
> Today's Topics:
>
> 1. Problems updating resources in CKAN 2.7.2 + Centos 7.4
> (David Hern?ndez Guti?rrez)
> 2. Re: Problems updating resources in CKAN 2.7.2 + Centos 7.4
> (Matthew Fullerton)
>
>
> ---------- Mensaje reenviado ----------
> From: "David Hernández Gutiérrez" <davisoski at gmail.com>
> To: ckan-dev at lists.okfn.org
> Cc:
> Bcc:
> Date: Wed, 3 Jan 2018 10:10:37 +0000
> Subject: [ckan-dev] Problems updating resources in CKAN 2.7.2 + Centos 7.4
>
> Hi.
>
> I'm experience something very annoying in my installation of ckan 2.7.2 in
> Centos 7.4 (httpd+tomcat 7).
>
> I have followed this info to install it:
>
> https://github.com/ckan/ckan/wiki/How-to-Install-CKAN
>
> https://github.com/ckan/ckan/wiki/How-to-install-CKAN-2.x-on-CentOS-7
>
> https://github.com/ckan/ckan/wiki/How-to-install-CKAN-2.5.2-on-CentOS-6.8
>
> https://github.com/ckan/ckan/wiki/How-to-Install-CKAN-2.0-on-CentOS-6.3
>
>
> When I upload file, the resource is correctly upload but I get an *Internal
> Server Error, *later I refresh page an the resource is correctly updated.
>
> [Wed Jan 03 10:03:59.113908 2018] [:error] [pid 18195] 2018-01-03
> 10:03:59,113 INFO [ckan.lib.base] /api/i18n/es render time 0.086 seconds
> [Wed Jan 03 10:03:59.123945 2018] [:error] [pid 18195] 2018-01-03
> 10:03:59,123 INFO [ckan.lib.base] /api/i18n/es render time 0.004 seconds
> [Wed Jan 03 10:04:11.793452 2018] [:error] [pid 18196] 2018-01-03
> 10:04:11,793 INFO [ckan.lib.base] /api/2/util/resource/format_autocomplete
> render time 0.028 seconds
> [Wed Jan 03 10:04:14.847291 2018] [:error] [pid 18195] 2018-01-03
> 10:04:14,846 INFO [ckan.lib.base] /api/2/util/resource/format_autocomplete
> render time 0.023 seconds
> [Wed Jan 03 10:04:15.202347 2018] [:error] [pid 18196] 2018-01-03
> 10:04:15,202 INFO [ckan.lib.base] /api/2/util/resource/format_autocomplete
> render time 0.020 seconds
> [Wed Jan 03 10:04:26.837181 2018] [:error] [pid 18195]
> /usr/lib/ckan/default/lib/python2.7/site-packages/
> sqlalchemy/orm/unitofwork.py:79: SAWarning: Usage of the 'related
> attribute set' operation is not currently supported within the execution
> stage of the flush process. Results may not be consistent. Consider using
> alternative event listeners or connection-level operations instead.
> [Wed Jan 03 10:04:26.837230 2018] [:error] [pid 18195]
> sess._flush_warning("related attribute set")
> [Wed Jan 03 10:04:29.450327 2018] [:error] [pid 18195] [remote
> 172.19.153.13:12] mod_wsgi (pid=18195): Exception occurred processing
> WSGI script '/etc/ckan/default/apache.wsgi'.
> *[Wed Jan 03 10:04:29.450386 2018] [:error] [pid 18195] [remote
> 172.19.153.13:12 <http://172.19.153.13:12>] TypeError: expected byte string
> object for header name, value of type unicode found*
>
>
> There is an issue in github with something similar:
>
> https://github.com/ckan/ckan/issues/3921
> Any idea to fix this??
>
> Thanks
>
>
> ---------- Mensaje reenviado ----------
> From: Matthew Fullerton <matt.fullerton at gmail.com>
> To: CKAN Development Discussions <ckan-dev at lists.okfn.org>
> Cc:
> Bcc:
> Date: Wed, 3 Jan 2018 12:50:13 +0100
> Subject: Re: [ckan-dev] Problems updating resources in CKAN 2.7.2 + Centos
> 7.4
> Are you certain that log is coming at the time of the Internal Server
> Error? I don't think the two are related.
>
> What might be happening is this:
> - You upload the resource, and after a successful upload, you are
> redirected to the resource view page
> - In the background, the resource is being uploaded to the datastore
> - The first view of the resource you get is before the datastore upload is
> completed
> - Something goes wrong somewhere because the resource is not in the
> datastore, possibly something with the resource view itself, or another bug
>
> To get more insight, you could adjust the file types that datapusher tries
> to upload automatically in the config file or temporarily disable the
> datapusher plugin completely. That will leave you at point 3 above and give
> you more time to debug.
>
> That's all just a guess though at what's going on from your description
> and a bit of experience...
>
> Best,
> Matt
>
> On 3 January 2018 at 11:10, David Hernández Gutiérrez <davisoski at gmail.com
> > wrote:
>
>> Hi.
>>
>> I'm experience something very annoying in my installation of ckan 2.7.2
>> in Centos 7.4 (httpd+tomcat 7).
>>
>> I have followed this info to install it:
>>
>> https://github.com/ckan/ckan/wiki/How-to-Install-CKAN
>>
>> https://github.com/ckan/ckan/wiki/How-to-install-CKAN-2.x-on-CentOS-7
>>
>> https://github.com/ckan/ckan/wiki/How-to-install-CKAN-2.5.2-on-CentOS-6.8
>>
>> https://github.com/ckan/ckan/wiki/How-to-Install-CKAN-2.0-on-CentOS-6.3
>>
>>
>> When I upload file, the resource is correctly upload but I get an *Internal
>> Server Error, *later I refresh page an the resource is correctly updated.
>>
>> [Wed Jan 03 10:03:59.113908 2018] [:error] [pid 18195] 2018-01-03
>> 10:03:59,113 INFO [ckan.lib.base] /api/i18n/es render time 0.086 seconds
>> [Wed Jan 03 10:03:59.123945 2018] [:error] [pid 18195] 2018-01-03
>> 10:03:59,123 INFO [ckan.lib.base] /api/i18n/es render time 0.004 seconds
>> [Wed Jan 03 10:04:11.793452 2018] [:error] [pid 18196] 2018-01-03
>> 10:04:11,793 INFO [ckan.lib.base] /api/2/util/resource/format_autocomplete
>> render time 0.028 seconds
>> [Wed Jan 03 10:04:14.847291 2018] [:error] [pid 18195] 2018-01-03
>> 10:04:14,846 INFO [ckan.lib.base] /api/2/util/resource/format_autocomplete
>> render time 0.023 seconds
>> [Wed Jan 03 10:04:15.202347 2018] [:error] [pid 18196] 2018-01-03
>> 10:04:15,202 INFO [ckan.lib.base] /api/2/util/resource/format_autocomplete
>> render time 0.020 seconds
>> [Wed Jan 03 10:04:26.837181 2018] [:error] [pid 18195]
>> /usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py:79:
>> SAWarning: Usage of the 'related attribute set' operation is not currently
>> supported within the execution stage of the flush process. Results may not
>> be consistent. Consider using alternative event listeners or
>> connection-level operations instead.
>> [Wed Jan 03 10:04:26.837230 2018] [:error] [pid 18195]
>> sess._flush_warning("related attribute set")
>> [Wed Jan 03 10:04:29.450327 2018] [:error] [pid 18195] [remote
>> 172.19.153.13:12] mod_wsgi (pid=18195): Exception occurred processing
>> WSGI script '/etc/ckan/default/apache.wsgi'.
>> *[Wed Jan 03 10:04:29.450386 2018] [:error] [pid 18195] [remote
>> 172.19.153.13:12 <http://172.19.153.13:12>] TypeError: expected byte string
>> object for header name, value of type unicode found*
>>
>>
>> There is an issue in github with something similar:
>>
>> https://github.com/ckan/ckan/issues/3921
>> Any idea to fix this??
>>
>> Thanks
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20180103/7d14df53/attachment-0002.html>
More information about the ckan-dev
mailing list