[ckan-dev] Solr problem when installing latest ckan

Fri Apr 19 10:36:34 UTC 2013

Mmm, looks like Solr is set up fine and returning the correct response
if queried directly, but for some reason, is failing when CKAN tries
to contact it.
I'm out of ideas I'm afraid.

Can you paste the full traceback somewhere like gist or pastebin?

Adrià

On 17 April 2013 15:17, Lasse Vestergaard <ibbernik at gmail.com> wrote:
> Hi Adriá.
>
> My solr_url=http://localhost:8983/solr/
>
> Should it be my public ip / domain name?
>
> The response from the url you send (I tested it through curl because my
> server i af remote Ubuntu server edition):
>
> <?xml version="1.0" encoding="UTF-8"?>
> <response>
> <lst name="responseHeader"><int name="status">0</int><int
> name="QTime">0</int><lst name="params"><str
> name="q">*:*</str></lst></lst><result name="response" numFound="2"
> start="0"><doc><str name="capacity">public</str><str
> name="data_dict">{"license_title": "Creative Commons Non-Commercial (Any)",
> "maintainer": null, "relationships_as_object": [], "private": false,
> "maintainer_email": null, "num_tags": 2, "id":
> "9ed13222-434f-42e0-8541-4afac6694426", "metadata_created":
> "2013-04-15T19:05:56.744452", "metadata_modified":
> "2013-04-15T19:05:56.744452", "author": null, "author_email": null, "state":
> "active", "version": null, "license_id": "cc-nc", "type": "dataset",
> "resources": [], "num_resources": 0, "tags": [{"vocabulary_id": null,
> "display_name": "Flexible \u30a1", "name": "Flexible \u30a1",
> "revision_timestamp": "2013-04-15T19:05:56.744452", "state": "active", "id":
> "bdcff1d0-d221-4db2-b343-6ae67ccda727"}, {"vocabulary_id": null,
> "display_name": "russian", "name": "russian", "revision_timestamp":
> "2013-04-15T19:05:56.744452", "state": "active", "id":
> "a56850e6-05d5-4ce3-bfe0-4a6622103425"}], "tracking_summary": {"total": 0,
> "recent": 0}, "groups": [{"capacity": "public", "description": "These are
> books that David likes.", "title": "Dave's books", "created":
> "2013-04-15T19:05:56.788501", "approval_status": "approved",
> "is_organization": false, "state": "active", "image_url": "", "revision_id":
> "22a0fb26-6aac-4b41-9d73-9d48042cb61f", "type": "group", "id":
> "316bf831-fcae-4906-bd17-95711af2e418", "name": "david"}], "organization":
> null, "relationships_as_subject": [], "revision_timestamp":
> "2013-04-15T19:05:56.744452", "name": "warandpeace", "isopen": false, "url":
> null, "notes": null, "owner_org": null, "extras": [], "license_url":
> "http://creativecommons.org/licenses/by-nc/2.0/", "title": "A Wonderful
> Story", "revision_id": "22a0fb26-6aac-4b41-9d73-9d48042cb61f"}</str><str
> name="dataset_type">dataset</str><str name="entity_type">package</str><arr
> name="groups"><str>david</str></arr><str
> name="id">9ed13222-434f-42e0-8541-4afac6694426</str><str
> name="index_id">05530c41f8bdcfe6957bebd60cfe587d</str><date
> name="indexed_ts">2013-04-15T19:05:57.12Z</date><str
> name="license_id">cc-nc</str><date
> name="metadata_created">2013-04-15T19:05:56.744Z</date><date
> name="metadata_modified">2013-04-15T19:05:56.744Z</date><str
> name="name">warandpeace</str><str
> name="revision_id">22a0fb26-6aac-4b41-9d73-9d48042cb61f</str><str
> name="site_id">54.246.105.104</str><str name="state">active</str><arr
> name="tags"><str>Flexible ァ</str><str>russian</str></arr><str name="title">A
> Wonderful Story</str></doc><doc><str name="capacity">public</str><str
> name="data_dict">{"license_title": "Other (Open)", "maintainer": null,
> "relationships_as_object": [], "private": false, "maintainer_email": null,
> "num_tags": 3, "id": "c5d91996-1e4f-4bb6-b6c1-b3ecb87d1fca",
> "metadata_created": "2013-04-15T19:05:56.744452", "metadata_modified":
> "2013-04-15T19:05:56.744452", "author": null, "author_email": null, "state":
> "active", "version": "0.7a", "license_id": "other-open", "type": "dataset",
> "resources": [{"resource_group_id": "efa90137-e2fb-4673-8044-a91482683b79",
> "cache_last_updated": null, "revision_timestamp":
> "2013-04-15T19:05:56.744452", "size_extra": "123", "webstore_last_updated":
> null, "id": "6ed505bc-ae4d-4226-981b-48525f080455", "size": null, "state":
> "active", "last_modified": null, "hash": "abc123", "description": "Full
> text. Needs escaping: \" Umlaut: \u00fc", "format": "plain text",
> "tracking_summary": {"total": 0, "recent": 0}, "mimetype_inner": null,
> "mimetype": null, "cache_url": null, "name": null, "created":
> "2013-04-15T19:05:56.827598", "url":
> "http://www.annakarenina.com/download/x=1&y=2", "webstore_url": null,
> "position": 0, "revision_id": "22a0fb26-6aac-4b41-9d73-9d48042cb61f",
> "resource_type": null}, {"resource_group_id":
> "efa90137-e2fb-4673-8044-a91482683b79", "cache_last_updated": null,
> "revision_timestamp": "2013-04-15T19:05:56.744452", "size_extra": "345",
> "webstore_last_updated": null, "id": "760347fd-c4b8-4cc4-8e96-a01d2c783a93",
> "size": null, "state": "active", "last_modified": null, "hash": "def456",
> "description": "Index of the novel", "format": "JSON", "tracking_summary":
> {"total": 0, "recent": 0}, "mimetype_inner": null, "mimetype": null,
> "cache_url": null, "name": null, "created": "2013-04-15T19:05:56.827619",
> "url": "http://www.annakarenina.com/index.json", "webstore_url": null,
> "position": 1, "revision_id": "22a0fb26-6aac-4b41-9d73-9d48042cb61f",
> "resource_type": null}], "num_resources": 2, "tags": [{"vocabulary_id":
> null, "display_name": "Flexible \u30a1", "name": "Flexible \u30a1",
> "revision_timestamp": "2013-04-15T19:05:56.744452", "state": "active", "id":
> "bdcff1d0-d221-4db2-b343-6ae67ccda727"}, {"vocabulary_id": null,
> "display_name": "russian", "name": "russian", "revision_timestamp":
> "2013-04-15T19:05:56.744452", "state": "active", "id":
> "a56850e6-05d5-4ce3-bfe0-4a6622103425"}, {"vocabulary_id": null,
> "display_name": "tolstoy", "name": "tolstoy", "revision_timestamp":
> "2013-04-15T19:05:56.744452", "state": "active", "id":
> "015f988b-f916-4e8c-baed-73548d59c6db"}], "tracking_summary": {"total": 0,
> "recent": 0}, "groups": [{"capacity": "public", "description": "These are
> books that David likes.", "title": "Dave's books", "created":
> "2013-04-15T19:05:56.788501", "approval_status": "approved",
> "is_organization": false, "state": "active", "image_url": "", "revision_id":
> "22a0fb26-6aac-4b41-9d73-9d48042cb61f", "type": "group", "id":
> "316bf831-fcae-4906-bd17-95711af2e418", "name": "david"}, {"capacity":
> "public", "description": "Roger likes these books.", "title": "Roger's
> books", "created": "2013-04-15T19:05:56.790544", "approval_status":
> "approved", "is_organization": false, "state": "active", "image_url": "",
> "revision_id": "22a0fb26-6aac-4b41-9d73-9d48042cb61f", "type": "group",
> "id": "7a7d3ded-567f-49d5-b2a4-ff1af6be931c", "name": "roger"}],
> "organization": null, "relationships_as_subject": [], "revision_timestamp":
> "2013-04-15T19:05:56.744452", "name": "annakarenina", "isopen": true, "url":
> "http://www.annakarenina.com", "notes": "Some test notes\n\n### A 3rd level
> heading\n\n**Some bolded text.**\n\n*Some italicized text.*\n\nForeign
> characters:\nu with umlaut \u00fc\n66-style quote \u201c\nforeign word:
> th\u00fcmb\n\nNeeds escaping:\nleft arrow
> <\n\n<http://ckan.net/>\n\n", "owner_org": null, "extras":
> [{"state": "active", "value": "romantic novel", "revision_timestamp":
> "2013-04-15T19:05:56.744452", "package_id":
> "c5d91996-1e4f-4bb6-b6c1-b3ecb87d1fca", "key": "genre", "revision_id":
> "22a0fb26-6aac-4b41-9d73-9d48042cb61f", "id":
> "4fc0b636-a1ee-4c76-b5a2-b74be8c638bd"}, {"state": "active", "value":
> "book", "revision_timestamp": "2013-04-15T19:05:56.744452", "package_id":
> "c5d91996-1e4f-4bb6-b6c1-b3ecb87d1fca", "key": "original media",
> "revision_id": "22a0fb26-6aac-4b41-9d73-9d48042cb61f", "id":
> "82575b20-3302-4716-81ff-f34108772aae"}], "title": "A Novel By Tolstoy",
> "revision_id": "22a0fb26-6aac-4b41-9d73-9d48042cb61f"}</str><str
> name="dataset_type">dataset</str><str name="entity_type">package</str><str
> name="extras_genre">romantic novel</str><str
> name="extras_originalmedia">book</str><arr
> name="groups"><str>david</str><str>roger</str></arr><str
> name="id">c5d91996-1e4f-4bb6-b6c1-b3ecb87d1fca</str><str
> name="index_id">3875b7964b4193e979ebe967f8d93062</str><date
> name="indexed_ts">2013-04-15T19:05:57.419Z</date><str
> name="license_id">other-open</str><date
> name="metadata_created">2013-04-15T19:05:56.744Z</date><date
> name="metadata_modified">2013-04-15T19:05:56.744Z</date><str
> name="name">annakarenina</str><str name="notes">Some test notes
>
> ### A 3rd level heading
>
> **Some bolded text.**
>
> *Some italicized text.*
>
> Foreign characters:
> u with umlaut ü
> 66-style quote “
> foreign word: thümb
>
> Needs escaping:
> left arrow <
>
> <http://ckan.net/>
>
> </str><arr name="res_description"><str>Full text. Needs escaping: " Umlaut:
> ü</str><str>Index of the novel</str></arr><arr name="res_format"><str>plain
> text</str><str>JSON</str></arr><arr
> name="res_url"><str>http://www.annakarenina.com/download/x=1&y=2</str><str>http://www.annakarenina.com/index.json</str></arr><str
> name="revision_id">22a0fb26-6aac-4b41-9d73-9d48042cb61f</str><str
> name="site_id">54.246.105.104</str><str name="state">active</str><arr
> name="tags"><str>Flexible
> ァ</str><str>russian</str><str>tolstoy</str></arr><str name="title">A Novel
> By Tolstoy</str><str name="url">http://www.annakarenina.com</str><str
> name="version">0.7a</str></doc></result>
> </response>
>
> Regards
>
> Lasse
>
>
>
> 2013/4/17 Adrià Mercader <adria.mercader at okfn.org>
>>
>> Hi Lasse,
>>
>> Looks like Solr is returning an HTML error instead of the expected
>> response.
>>
>> What is the value for your solr_url setting in your ini file?
>> What do you see if you load this URL on your browser?
>>
>> {solr_url}/select/?q=*:*&wt=json
>>
>>
>> Adrià
>>
>>
>> On 17 April 2013 09:38, Lasse Vestergaard <ibbernik at gmail.com> wrote:
>> > Hi again.
>> >
>> > It seems like I have managed to fix some of the problem. I found out
>> > that I
>> > needed a forward slash (/) at the end of the url given in solr_url in
>> > development.ini. So, right now the initial error has gone, but I'm
>> > getting
>> > some other ones intead :
>> >
>> > SolrException: HTTP code=Unknown XML response from server: <html ...,
>> > reason=None
>> > 2013-04-17 08:28:22,393 WARNI [ckan.lib.search] Problems were found
>> > while
>> > connecting to the SOLR server
>> > 2013-04-17 08:28:22,953 INFO  [ckan.lib.base]  /api/i18n/en render time
>> > 0.032 seconds
>> > 2013-04-17 08:30:13,674 ERROR [ckan.lib.search.query] Expecting value:
>> > line
>> > 3 column 1 (char 2)
>> > Traceback (most recent call last):
>> > File "/home/ubuntu/pyenv/src/ckan/ckan/lib/search/query.py", line 369,
>> > in
>> > run
>> > data = json.loads(solr_response)
>> > File
>> > "/home/ubuntu/pyenv/lib/python2.7/site-packages/simplejson/__init__.py",
>> > line 461, in loads
>> > return _default_decoder.decode(s)
>> > File
>> > "/home/ubuntu/pyenv/lib/python2.7/site-packages/simplejson/decoder.py",
>> > line 374, in decode
>> > obj, end = self.raw_decode(s)
>> > File
>> > "/home/ubuntu/pyenv/lib/python2.7/site-packages/simplejson/decoder.py",
>> > line 393, in raw_decode
>> > return self.scan_once(s, idx=_w(s, idx).end())
>> >
>> > and this:
>> >
>> > ckan.lib.search.query] Expecting value: line 3 column 1 (char 2)
>> > Traceback (most recent call last):
>> > File "/home/ubuntu/pyenv/src/ckan/ckan/lib/search/query.py", line 369,
>> > in
>> > run
>> > data = json.loads(solr_response)
>> > File
>> > "/home/ubuntu/pyenv/lib/python2.7/site-packages/simplejson/__init__.py",
>> > line 461, in loads
>> > return _default_decoder.decode(s)
>> > File
>> > "/home/ubuntu/pyenv/lib/python2.7/site-packages/simplejson/decoder.py",
>> > line 374, in decode
>> > obj, end = self.raw_decode(s)
>> > File
>> > "/home/ubuntu/pyenv/lib/python2.7/site-packages/simplejson/decoder.py",
>> > line 393, in raw_decode
>> > return self.scan_once(s, idx=_w(s, idx).end())
>> > JSONDecodeError: Expecting value: line 3 column 1 (char 2)
>> > 2013-04-17 08:30:58,151 ERROR [ckan.controllers.package] Dataset search
>> > error: (JSONDecodeError('Expecting value: line 3 column 1 (char 2)',),)
>> > 2013-04-17 08:30:58,415 INFO  [ckan.lib.base]  /dataset render time
>> > 0.296
>> > seconds
>> > 2013-04-17 08:30:58,604 INFO  [ckan.lib.base]  /api/i18n/en render time
>> > 0.024 seconds
>> >
>> > Any ideas?
>> >
>> > /Lasse
>> >
>> >
>> >
>> > 2013/4/16 Lasse Vestergaard <ibbernik at gmail.com>
>> >>
>> >> Hi all.
>> >>
>> >> Have installed ckan on an Ubuntu server (12.04), from this link:
>> >>
>> >> http://docs.ckan.org/en/latest/install-from-source.html (2.1 i guess)
>> >>
>> >> Everything installed fine. And I can connect to the front page of ckan.
>> >> There are two problems though.
>> >>
>> >> When I'm logged in as a sysadmin (in a web browser) I can't access
>> >> ckan-admin/authz. I just get page not found. Any suggestions?
>> >>
>> >> An other problem is that it seems like I have a problem with solr. When
>> >> I
>> >> uncomment (enable) solr_url in development.ini, the front page just
>> >> says:
>> >>
>> >> Server Error
>> >> An internal server error occurred
>> >>
>> >> When I look in apache error.log I get the following message:
>> >>
>> >> [Tue Apr 16 13:26:38 2013] [error] 2013-04-16 13:26:38,319 ERROR
>> >> [ckan.lib.search.common] ''
>> >> [Tue Apr 16 13:26:38 2013] [error] Traceback (most recent call last):
>> >> [Tue Apr 16 13:26:38 2013] [error]   File
>> >> "/home/ubuntu/pyenv/src/ckan/ckan/lib/search/common.py", line 51, in
>> >> is_available
>> >> [Tue Apr 16 13:26:38 2013] [error]     conn.query("*:*", rows=1)
>> >> [Tue Apr 16 13:26:38 2013] [error]   File
>> >> "/home/ubuntu/pyenv/lib/python2.7/site-packages/solr/core.py", line
>> >> 703, in
>> >> query
>> >> [Tue Apr 16 13:26:38 2013] [error]     return self.select(*args,
>> >> **params)
>> >> [Tue Apr 16 13:26:38 2013] [error]   File
>> >> "/home/ubuntu/pyenv/lib/python2.7/site-packages/solr/core.py", line
>> >> 798, in
>> >> __call__
>> >> [Tue Apr 16 13:26:38 2013] [error]     xml = self.raw(**params)
>> >> [Tue Apr 16 13:26:38 2013] [error]   File
>> >> "/home/ubuntu/pyenv/lib/python2.7/site-packages/solr/core.py", line
>> >> 823, in
>> >> raw
>> >> [Tue Apr 16 13:26:38 2013] [error]     rsp = conn._post(self.selector,
>> >> request, conn.form_headers)
>> >> [Tue Apr 16 13:26:38 2013] [error]   File
>> >> "/home/ubuntu/pyenv/lib/python2.7/site-packages/solr/core.py", line
>> >> 639, in
>> >> _post
>> >> [Tue Apr 16 13:26:38 2013] [error]     return
>> >> check_response_status(self.conn.getresponse())
>> >> [Tue Apr 16 13:26:38 2013] [error]   File
>> >> "/usr/lib/python2.7/httplib.py",
>> >> line 1030, in getresponse
>> >> [Tue Apr 16 13:26:38 2013] [error]     response.begin()
>> >> [Tue Apr 16 13:26:38 2013] [error]   File
>> >> "/usr/lib/python2.7/httplib.py",
>> >> line 407, in begin
>> >> [Tue Apr 16 13:26:38 2013] [error]     version, status, reason =
>> >> self._read_status()
>> >> [Tue Apr 16 13:26:38 2013] [error]   File
>> >> "/usr/lib/python2.7/httplib.py",
>> >> line 371, in _read_status
>> >> [Tue Apr 16 13:26:38 2013] [error]     raise BadStatusLine(line)
>> >> [Tue Apr 16 13:26:38 2013] [error] BadStatusLine: ''
>> >> [Tue Apr 16 13:26:38 2013] [error] 2013-04-16 13:26:38,320 WARNI
>> >> [ckan.lib.search] Problems were found while connecting to the SOLR
>> >> server
>> >> [Tue Apr 16 13:26:38 2013] [error] 2013-04-16 13:26:38,891 INFO
>> >> [ckan.lib.base]  /api/i18n/en render time 0.001 seconds
>> >>
>> >> I can use curl to connect to http://localhost:8983/solr/ and I get a
>> >> welcome message, so in that regard it seems like solr is setup up
>> >> correctly.
>> >> I'm not sure if the problem really relates to WSGI.
>> >>
>> >> Do you have any ideas?
>> >>
>> >> Best regards
>> >>
>> >> Lasse Vestergaard
>> >
>> >
>> >
>> > _______________________________________________
>> > ckan-dev mailing list
>> > ckan-dev at lists.okfn.org
>> > http://lists.okfn.org/mailman/listinfo/ckan-dev
>> > Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>> >
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>