[ckan-dev] Spatial search limits

Tue Apr 17 20:05:02 UTC 2012

I've hit a limit with the spatial search provided in ckanext-spatial.
If you do a search over a geographical area that has too many results
then I get a SOLR exception.

Dataset search error: ('SOLR returned an error running
query: {\'sort\': \'score desc, name asc\', \'fq\': \'
+site_id:"dgu-shazam" +state:active\', \'facet.mincount\'
: 1, \'rows\': 11, \'facet.field\': [\'groups\', \'tags\',
\'res_format\', \'license\', \'resource-type\', \'UKL
P\', \'license_id-is-ogl\', \'publisher\'], \'wt\': \'json\',
\'facet.limit\': \'50\', \'facet\': \'true\', \'q\
': u\'(id:ff4cc143-c00a-46b7-81bb-5095372847b6 OR
id:2e279200-0ec7-4f15-a2a5-cdc68e277944 OR id:5a705e99-5425-43
37-abbc-195516e32ae4 OR id:0fe66473-8eca-4114-9642-20fb8cc11391 OR ...snip...

Jetty has a limit of the size of request that is sent to SOLR. It can
be increased to 64k http://drupal.org/node/443980 but by my
calculations that sets the limit at about 1500 results, which is still
too low for our site.

Have you experienced this and do you have any thoughts of how to overcome this?

(BTW the relevant ckanext-spatial tests are currently broken due to
the new API schema check process. I strongly believe that CKAN should
only check that every parameter is provided when submitting by form -
it is way too inconvenient in the API and logic layer. This is a form
specific thing.)

David