[ckan-dev] Updating Solr to support point search and confusion with Spatial Search Widget

Tue Jul 21 18:00:39 UTC 2015

Hi all,

I have installed CKAN on Ubuntu 14.04, and I've added ckanext-spatial
extension with the Spatial Search Widget and Dataset Extent Map.
I have added a few datasets with extra attribute with 'spatial' key and
both points and polygons.

Now, both are shown very nice in Dataset Extend Map, but when I try to
filter datasets in search, only those that have a polygon associated with
them are shown in results.

Form the installation instructions, it says that polygons and points are
supported in solr-spatial-field backend (which requires solr 4.x+), but I
am using solr backend (with solr 3.6.2 that came with running apt-get as
specified in installation instructions). The thing is that first I tried to
update solr to 4.0.0 but I failed, and only then I realised that search
with polygons is working, but not with points.

So I am confused. How come the search is working with polygons at all if I
don't have the right solr version and I haven't even put JTS, and how to
enable the search to take into account datasets with point.

Maybe another question is how to update solr - the repository does not have
other versions, and I couldn't do it manually because I could not find
related folders and files in solr and jetty?
Anyone has a clue how to proceed?

If it helps, I list here the output with an error in the console that I get
when adding a dataset with spatial attribute which is a point (I don't see
the same error when adding a polygon).

2015-07-21 19:49:29,922 DEBUG [ckanext.spatial.plugin] Received: u'{
"type": "Point", "coordinates": [-3.716667,40.383335] }'
2015-07-21 19:49:29,956 DEBUG [ckanext.spatial.lib] Extent for package
10f6c260-76d8-481e-8f75-a7bc3e9735d2 unchanged
2015-07-21 19:49:31,470 INFO  [ckan.lib.base]  /api/3/action/resource_show
render time 0.037 seconds
2015-07-21 19:49:31,942 DEBUG [ckanext.spatial.plugin] Received: u'{
"type": "Point", "coordinates": [-3.716667,40.383335] }'
2015-07-21 19:49:32,000 DEBUG [ckanext.spatial.lib] Extent for package
10f6c260-76d8-481e-8f75-a7bc3e9735d2 unchanged
2015-07-21 19:49:32,633 ERROR [ckanext.spatial.plugin] Solr backend only
supports bboxes, ignoring geometry { "type": "Point", "coordinates":
[-3.716667,40.383335] }
2015-07-21 19:49:33,393 INFO  [ckan.lib.base]
 /dataset/new_resource/bristol-energy-in-buildings render time 3.790 seconds
2015-07-21 19:49:33,403 INFO  [ckan.lib.base]  /api/3/action/resource_show
render time 0.231 seconds
/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py:459:
SAWarning: Unicode type received non-unicode bind param value.
  param.append(processors[key](compiled_params[key]))
Fetching from:
http://127.0.0.1:5000/dataset/10f6c260-76d8-481e-8f75-a7bc3e9735d2/resource/faaacf90-e43a-4c34-892e-5c770d41e863/download/Bristol.csv
2015-07-21 19:49:33,966 WARNI [ckan.lib.maintain] Function
_resource_preview() in module ckan.controllers.package has been deprecated
and will be removed in a later release of ckan. Resource preview is
deprecated. Please use the new resource views
2015-07-21 19:49:34,464 INFO  [ckan.lib.base]
 /dataset/10f6c260-76d8-481e-8f75-a7bc3e9735d2/resource/faaacf90-e43a-4c34-892e-5c770d41e863/download/Bristol.csv
render time 0.196 seconds
2015-07-21 19:49:34,620 WARNI [ckan.lib.maintain] Function get_action() in
module ckan.lib.helpers has been deprecated and will be removed in a later
release of ckan. helpers.get_action() is deprecated and will be removed in
a future version of CKAN. Instead, please use the extra_vars param to
render() in your controller to pass results from action functions to your
templates.
2015-07-21 19:49:35,513 WARNI [ckan.lib.maintain] Function get_action() in
module ckan.lib.helpers has been deprecated and will be removed in a later
release of ckan. helpers.get_action() is deprecated and will be removed in
a future version of CKAN. Instead, please use the extra_vars param to
render() in your controller to pass results from action functions to your
templates.
Deleting "faaacf90-e43a-4c34-892e-5c770d41e863" from datastore.
2015-07-21 19:49:36,837 INFO  [ckan.lib.base]
 /api/3/action/datastore_delete render time 0.063 seconds
Determined headers and types: [{'type': u'timestamp', 'id': u'date_time'},
{'type': u'numeric', 'id': u'consumption'}, {'type': u'text', 'id':
u'Intervention'}, {'type': u'text', 'id': u'Building'}, {'type': u'text',
'id': u'Energy'}, {'type': u'text', 'id': u'City'}]
Saving chunk 0
2015-07-21 19:49:37,919 INFO  [ckan.lib.base]  /api/3/action/resource_show
render time 0.425 seconds
Fetching from:
http://127.0.0.1:5000/dataset/10f6c260-76d8-481e-8f75-a7bc3e9735d2/resource/faaacf90-e43a-4c34-892e-5c770d41e863/download/Bristol.csv
2015-07-21 19:49:38,150 INFO  [ckan.lib.base]
 /dataset/bristol-energy-in-buildings render time 4.359 seconds
2015-07-21 19:49:39,227 INFO  [ckan.lib.base]
 /dataset/10f6c260-76d8-481e-8f75-a7bc3e9735d2/resource/faaacf90-e43a-4c34-892e-5c770d41e863/download/Bristol.csv
render time 0.412 seconds
Deleting "faaacf90-e43a-4c34-892e-5c770d41e863" from datastore.
2015-07-21 19:49:40,568 INFO  [ckan.lib.base]
 /api/3/action/datastore_create render time 2.245 seconds
Saving chunk 1
2015-07-21 19:49:41,267 INFO  [ckan.lib.base]
 /api/3/action/datastore_delete render time 0.313 seconds
Determined headers and types: [{'type': u'timestamp', 'id': u'date_time'},
{'type': u'numeric', 'id': u'consumption'}, {'type': u'text', 'id':
u'Intervention'}, {'type': u'text', 'id': u'Building'}, {'type': u'text',
'id': u'Energy'}, {'type': u'text', 'id': u'City'}]
2015-07-21 19:49:41,629 INFO  [ckan.lib.base]
 /api/3/action/datastore_create render time 0.394 seconds
Successfully pushed 274 entries to "faaacf90-e43a-4c34-892e-5c770d41e863".
Saving chunk 0
2015-07-21 19:49:42,479 INFO  [ckan.lib.base]
 /api/3/action/datapusher_hook render time 0.338 seconds
2015-07-21 19:49:42,946 INFO  [ckan.lib.base]
 /api/3/action/datastore_create render time 0.396 seconds
Saving chunk 1
2015-07-21 19:49:43,975 INFO  [ckan.lib.base]
 /api/3/action/datastore_create render time 0.692 seconds
Successfully pushed 274 entries to "faaacf90-e43a-4c34-892e-5c770d41e863".
2015-07-21 19:49:44,235 INFO  [ckan.lib.base]  /api/i18n/en render time
0.003 seconds
2015-07-21 19:49:44,652 INFO  [ckan.lib.base]
 /api/3/action/datapusher_hook render time 0.169 seconds

Cheers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20150721/8a4d5836/attachment-0002.html>