[ckan-dev] Error running solr in a container

Pedro G Silva pedro.gracio at deimos.com.pt
Tue May 31 16:05:34 UTC 2016 

I replied to derek (mistakly) private in other email:
Hi Derek
The JTS.jar is to enable the spatial support in solr. If you don't need it,
skip it. Or maybe try to mkdir -p and cp the jar before installing solr,
that way you don't have to deal with restarting the service
About the 404 error,  /solr/select/  should be  /solr/ckan/select/
check solr_url in dev.ini

Derek it might not be documented in ckan documentation since ckan still
oficially uses solr 3 or 4 (?) and you are using solr 5.

On Tue, May 31, 2016 at 3:59 PM, Derek Hohls <dhohls at csir.co.za> wrote:

> Thanks Adrià
>
> Yes it does - Pedro sent me the same clue "off list" and it is has worked
> (up to the point of the next error).
>
>  >>> Adrià Mercader <adria.mercader at okfn.org> 31/05/16 11:28 AM >>>
> Hi Derek,
>
> If I understood correctly, you created a Solr core named "ckan". In that
> case your sorl_url setting in the CKAN ini file should be:
>
> solr_url = http://{solr_host}/solr/ckan
>
> Replace {solr_host} with the appropriate value.
>
> Does this help?
>
> Adrià
>
>
>
> On 30 May 2016 at 10:39, Derek Hohls <dhohls at csir.co.za> wrote:
>
>> Hi again
>>
>> After some further work, it seems as though the ckan "core" is now
>> correctly setup for solr; the admin page at http://my.ip:8983/solr/#/ckan
>> displays without any obvious errors (although, to be honest, I do not know
>> enough to tell if there is something missing that should be there).
>>
>> I have tried to implement all Pedro's changes for my solr 5 instance
>> (adding the new .jar file is a bit tricky, as one cannot restart the solr
>> service from inside the container...)
>>
>> I am still hitting the same error though: the call being made to
>> /solr/select/ by CKAN causes a 404 error; which can be duplicated in the
>> browser:
>> HTTP ERROR 404
>>
>> Problem accessing /solr/select/. Reason:
>>
>>     Not Found
>>
>> ------------------------------
>> *Powered by Jetty://*
>>
>> There is nothing obvious in the solr.log file - running a tail while
>> doing a page refresh does not trigger any new log entry,
>>
>> I am not quite sure how to proceed from here?  Any other ideas or
>> suggestions would be welcome!
>>
>> Thanks
>> Derek
>>
>>
>> >>> "Derek Hohls" <dhohls at csir.co.za> 27/05/16 5:16 PM >>>
>> Just for the record, the log is under:
>>
>> /opt/solr/server/logs/solr.log
>>
>> I am working on automating the install by extracting the necessary files
>> from a " run once" instance of a solr container, then overwriting the
>> default schema.xml with the one supplied by CKAN, then remounting all these
>> files back into the final instance of solr  (as per instructions in
>> https://github.com/docker-solr/docker-solr/blob/master/5.3/scripts/docker-entrypoint.sh)
>> to auto-create the ckan core files.
>>
>> The solr container is working; but I need to get the CKAN container to
>> communicate with it.
>>
>> Thanks for the help so far.
>>
>> Derek
>>
>>
>> >>> Pedro G Silva <pedro.gracio at deimos.com.pt> 27/05/16 11:45 AM >>>
>> logs are in /var/solr/logs
>> if you want to automate the install you can just curl or wget that
>> webpage to create the core
>>
>> On Thu, May 26, 2016 at 2:09 PM, Derek Hohls <dhohls at csir.co.za> wrote:
>>
>>> Hi Pedro
>>>
>>> Thanks for all the help here! - I did suspect it might be a missing core
>>> issue.  I am still on a learning curve with these technologies and am not
>>> sure what is / is not a requirement or pre-requisite for each.
>>>
>>> I obviously missed the part in the CKAN install guide where I had to
>>> manually create the core; but I will look that up as I need to find a way
>>> to fully automate it at install time (i.e. without access to a browser as
>>> you suggest).
>>>
>>> Have you any quick pointers about where the solr log is?  One of my
>>> frustrations about Java-based installs is that developers seem to get very
>>> creative about where everything goes (even between installs of exactly the
>>> same package).  It appears that the Docker install has placed
>>> everything under /opt/solr but I see your commands reference /var/solr ...
>>>
>>> I will check, in the meantime, whether these other files you referenced,
>>> are correct on my side.
>>>
>>> Thanks
>>> Derek
>>>
>>>
>>> >>> Pedro G Silva <pedro.gracio at deimos.com.pt> 25/05/16 4:55 PM >>>
>>> Im using solr 5. Derek try 2 things:
>>> -see if you have a proxy set somewhere
>>> -check the solr logs for more detailed info
>>> -check that in that solr version that you have you created the core, on
>>> version 5 the link to solr is solr_url:8983/solr/core_name , and you have
>>> to manually create this core, since solr 5 doesn't automatically create
>>> cores. also you should check that all needed files were copied. I'll leave
>>> a snippet of my commands. Hope it helps
>>> 1
>>> 2
>>> 3
>>> 4
>>> 5
>>> 6
>>> 7
>>> 8
>>> 9
>>> 10
>>> 11
>>> wget https://archive.apache.org/dist/lucene/solr/5.1.0/solr-5.1.0.tgz
>>> tar xzf solr-5.1.0.tgz solr-5.1.0/bin/install_solr_service.sh
>>> --strip-components=2
>>> sudo bash ./install_solr_service.sh solr-5.1.0.tgz
>>> sudo -u solr mkdir -p /var/solr/data/configsets/ckan/conf
>>> sudo ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml
>>> /var/solr/data/configsets/ckan/conf/schema.xml
>>> sudo -u solr cp
>>> /opt/solr/server/solr/configsets/basic_configs/conf/solrconfig.xml
>>> /var/solr/data/configsets/ckan/conf/.
>>> sudo -u solr cp
>>> /opt/solr/server/solr/configsets/basic_configs/conf/synonyms.txt
>>> /var/solr/data/configsets/ckan/conf/.
>>> sudo -u solr cp
>>> /opt/solr/server/solr/configsets/basic_configs/conf/protwords.txt
>>> /var/solr/data/configsets/ckan/conf/.
>>> wget http://search.maven.org/remotecontent?filepath=com
>>> /vividsolutions/jts/1.13/jts-1.13.jar -O jts-1.13.jar
>>> sudo -u solr cp jts-1.13.jar
>>> /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/.
>>> sudo service solr restart
>>>
>>> Go to the following link to create the SOLR Core
>>>
>>>
>>> http://localhost:8983/solr/admin/cores?action=CREATE&name=ckan&configSet=ckan
>>>
>>> On Wed, May 25, 2016 at 3:18 PM, David Read <
>>> david.read at hackneyworkshop.com> wrote:
>>>
>>>> Derek,
>>>>
>>>> Where did you get the idea ckan only supports solr v3.6? Because
>>>> wherever it is, it needs fixing.
>>>>
>>>> I think most people run SOLR v4 and several are on v5.
>>>> e.g. https://github.com/ckan/ckan/issues/2318
>>>>
>>>> Dave
>>>>
>>>> On 25 May 2016 at 10:55, Derek Hohls <dhohls at csir.co.za> wrote:
>>>> > Hi
>>>> >
>>>> > Hope the list can help with this problem...
>>>> >
>>>> > I have setup and run a full CKAN install without problems in a VM.
>>>> >
>>>> > Now I am trying to run solr in a separate container.  I have done
>>>> this,
>>>> > using the Docker image from
>>>> https://hub.docker.com/r/idguk/docker-solr/ (the
>>>> > official solr Docker could not be used as it does not support the
>>>> very old
>>>> > version - 3.6 -  which is used by CKAN). The Docker is linked, via
>>>> the "-v"
>>>> > option, to the `schema.xml` file sourced from
>>>> > `python-ckan_2.5-trusty_amd64.deb`.
>>>> >
>>>> > However, calling:
>>>> >
>>>> >     ckan db init
>>>> >
>>>> > triggers off an error; specifically this:
>>>> >
>>>> > Traceback (most recent call last):
>>>> >   File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/common.py",
>>>> line 51,
>>>> > in is_available
>>>> >     conn.query("*:*", rows=1)
>>>> >   File
>>>> >
>>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>>> > 703, in query
>>>> >     return self.select(*args, **params)
>>>> >   File
>>>> >
>>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>>> > 798, in __call__
>>>> >     xml = self.raw(**params)
>>>> >   File
>>>> >
>>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>>> > 823, in raw
>>>> >     rsp = conn._post(self.selector, request, conn.form_headers)
>>>> >   File
>>>> >
>>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>>> > 639, in _post
>>>> >     return check_response_status(self.conn.getresponse())
>>>> >   File
>>>> >
>>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>>> > 1097, in check_response_status
>>>> >     raise ex
>>>> >
>>>> > On checking the `solr/core.py` file, it seems as if its the call to
>>>> > `solr/select/` that triggers the issue.
>>>> >
>>>> > Access via a browser to this URL (the container version of solr)
>>>> shows a
>>>> > Java error:
>>>> >
>>>> > java.lang.NullPointerException
>>>> >     at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:92)
>>>> >     at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>>>> >     at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>>> >     at
>>>> > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>>>> >     at
>>>> >
>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:306)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>>>> >     at
>>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>> >     at org.mortbay.jetty.Server.handle(Server.java:326)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>>> >     at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>>> >     at
>>>> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>>> >     at
>>>> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>> >     at
>>>> >
>>>> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
>>>> >     at
>>>> >
>>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>> >
>>>> > But in the VM-version of solr, the same call generates an XML
>>>> document:
>>>> >
>>>> > <?xml version="1.0" encoding="UTF-8"?>
>>>> > <response>
>>>> >     <lst name="responseHeader">
>>>> >     <int name="status">0</int>
>>>> >     <int name="QTime">1</int>
>>>> >     <lst name="params"/></lst>
>>>> >     <result name="response" numFound="0" start="0"/>
>>>> > </response>
>>>> >
>>>> > I'd appreciate if anyone could highlight what I have missed in the
>>>> > configuration that could cause this error.
>>>> >
>>>> > Thanks
>>>> > Derek
>>>> >
>>>> >
>>>> > --
>>>> > This message is subject to the CSIR's copyright terms and conditions,
>>>> e-mail
>>>> > legal notice, and implemented Open Document Format (ODF) standard.
>>>> > The full disclaimer details can be found at
>>>> > http://www.csir.co.za/disclaimer.html.
>>>> >
>>>> >
>>>> > This message has been scanned for viruses and dangerous content by
>>>> > MailScanner,
>>>> > and is believed to be clean.
>>>> >
>>>> >
>>>> > Please consider the environment before printing this email.
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > ckan-dev mailing list
>>>> > ckan-dev at lists.okfn.org
>>>> > https://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>> >
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>>
>>>
>>>
>>>
>>> --
>>>
>>>
>>> [image: DEIMOS Engenharia, SA] <http://www.deimos.pt>
>>> *Pedro Gracio Silva*
>>> Aerospace Systems / Production Center
>>>
>>> *Av. D. Joao II, Lt 1.17.01, 10 ? 1998-023 Lisboa, Portugal*
>>> Ph: +351 21 893 3020 ? Fax: +351 21 896 9099
>>>
>>>
>>>
>>> --
>>> This message is subject to the CSIR's copyright terms and conditions,
>>> e-mail legal notice, and implemented Open Document Format (ODF) standard.
>>> The full disclaimer details can be found at
>>> http://www.csir.co.za/disclaimer.html.
>>>
>>>
>>> This message has been scanned for viruses and dangerous content by
>>> *MailScanner* <http://www.mailscanner.info/>,
>>> and is believed to be clean.
>>>
>>>
>>> Please consider the environment before printing this email.
>>>
>>> --
>>> This message is subject to the CSIR's copyright terms and conditions,
>>> e-mail legal notice, and implemented Open Document Format (ODF) standard.
>>> The full disclaimer details can be found at
>>> http://www.csir.co.za/disclaimer.html.
>>>
>>>
>>> This message has been scanned for viruses and dangerous content by
>>> *MailScanner* <http://www.mailscanner.info/>,
>>> and is believed to be clean.
>>>
>>>
>>> Please consider the environment before printing this email.
>>>
>>>
>>
>>
>> --
>>
>>
>> [image: DEIMOS Engenharia, SA] <http://www.deimos.pt>
>> *Pedro Gracio Silva*
>> Aerospace Systems / Production Center
>>
>> *Av. D. Joao II, Lt 1.17.01, 10 ? 1998-023 Lisboa, Portugal*
>> Ph: +351 21 893 3020 ? Fax: +351 21 896 9099
>>
>>
>>
>> --
>> This message is subject to the CSIR's copyright terms and conditions,
>> e-mail legal notice, and implemented Open Document Format (ODF) standard.
>> The full disclaimer details can be found at
>> http://www.csir.co.za/disclaimer.html.
>>
>>
>> This message has been scanned for viruses and dangerous content by
>> *MailScanner* <http://www.mailscanner.info/>,
>> and is believed to be clean.
>>
>>
>> Please consider the environment before printing this email.
>>
>> --
>> This message is subject to the CSIR's copyright terms and conditions,
>> e-mail legal notice, and implemented Open Document Format (ODF) standard.
>> The full disclaimer details can be found at
>> http://www.csir.co.za/disclaimer.html.
>>
>>
>> This message has been scanned for viruses and dangerous content by
>> *MailScanner* <http://www.mailscanner.info/>,
>> and is believed to be clean.
>>
>>
>> Please consider the environment before printing this email.
>>
>> --
>> This message is subject to the CSIR's copyright terms and conditions,
>> e-mail legal notice, and implemented Open Document Format (ODF) standard.
>> The full disclaimer details can be found at
>> http://www.csir.co.za/disclaimer.html.
>>
>>
>> This message has been scanned for viruses and dangerous content by
>> *MailScanner* <http://www.mailscanner.info/>,
>> and is believed to be clean.
>>
>>
>> Please consider the environment before printing this email.
>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>
> --
> This message is subject to the CSIR's copyright terms and conditions,
> e-mail legal notice, and implemented Open Document Format (ODF) standard.
> The full disclaimer details can be found at
> http://www.csir.co.za/disclaimer.html.
>
>
> This message has been scanned for viruses and dangerous content by
> *MailScanner* <http://www.mailscanner.info/>,
> and is believed to be clean.
>
>
> Please consider the environment before printing this email.
>
> --
> This message is subject to the CSIR's copyright terms and conditions,
> e-mail legal notice, and implemented Open Document Format (ODF) standard.
> The full disclaimer details can be found at
> http://www.csir.co.za/disclaimer.html.
>
>
> This message has been scanned for viruses and dangerous content by
> *MailScanner* <http://www.mailscanner.info/>,
> and is believed to be clean.
>
>
> Please consider the environment before printing this email.
>
>


-- 


[image: DEIMOS Engenharia, SA] <http://www.deimos.pt>
*Pedro Gracio Silva*
Aerospace Systems / Production Center

*Av. D. Joao II, Lt 1.17.01, 10 • 1998-023 Lisboa, Portugal*
Ph: +351 21 893 3020 • Fax: +351 21 896 9099
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20160531/f5eaffb3/attachment-0003.html>

More information about the ckan-dev mailing list