[ckan-dev] Error running solr in a container
Derek Hohls
dhohls at csir.co.za
Mon May 30 09:39:38 UTC 2016
Hi again
After some further work, it seems as though the ckan "core" is now correctly setup for solr; the admin page at http://my.ip:8983/solr/#/ckan
displays without any obvious errors (although, to be honest, I do not
know enough to tell if there is something missing that should be there).
I have tried to implement all Pedro's changes for my solr 5 instance (adding the new .jar file is a bit tricky, as one cannot restart the solr service from inside the container...)
I am still hitting the same error though: the call being made to /solr/select/ by CKAN causes a 404 error; which can be duplicated in the browser:
HTTP ERROR 404
Problem accessing /solr/select/. Reason:
Not FoundPowered by Jetty://
There is nothing obvious in the solr.log file - running a tail while doing a page refresh does not trigger any new log entry,
I am not quite sure how to proceed from here? Any other ideas or suggestions would be welcome!
Thanks
Derek
>>> "Derek Hohls" <dhohls at csir.co.za> 27/05/16 5:16 PM >>>
Just for the record, the log is under:
/opt/solr/server/logs/solr.log
I am working on automating the install by extracting the necessary files from a " run once" instance of a solr container, then overwriting the default schema.xml with the one supplied by CKAN, then remounting all these files back into the final instance of solr (as per instructions in https://github.com/docker-solr/docker-solr/blob/master/5.3/scripts/docker-entrypoint.sh) to auto-create the ckan core files.
The solr container is working; but I need to get the CKAN container to communicate with it.
Thanks for the help so far.
Derek
>>> Pedro G Silva <pedro.gracio at deimos.com.pt> 27/05/16 11:45 AM >>>
logs are in /var/solr/logsif you want to automate the install you can just curl or wget that webpage to create the core
On Thu, May 26, 2016 at 2:09 PM, Derek Hohls <dhohls at csir.co.za> wrote:
Hi Pedro
Thanks for all the help here! - I did suspect it might be a missing core issue. I am still on a learning curve with these technologies and am not sure what is / is not a requirement or pre-requisite for each.
I obviously missed the part in the CKAN install guide where I had to manually create the core; but I will look that up as I need to find a way to fully automate it at install time (i.e. without access to a browser as you suggest).
Have you any quick pointers about where the solr log is? One of my frustrations about Java-based installs is that developers seem to get very creative about where everything goes (even between installs of exactly the same package). It appears that the Docker install has placed everything under /opt/solr but I see your commands reference /var/solr ...
I will check, in the meantime, whether these other files you referenced, are correct on my side.
Thanks
Derek
>>> Pedro G Silva <pedro.gracio at deimos.com.pt> 25/05/16 4:55 PM >>>
Im using solr 5. Derek try 2 things:-see if you have a proxy set somewhere
-check the solr logs for more detailed info
-check that in that solr version that you have you created the core, on version 5 the link to solr is solr_url:8983/solr/core_name , and you have to manually create this core, since solr 5 doesn't automatically create cores. also you should check that all needed files were copied. I'll leave a snippet of my commands. Hope it helps
1
2
3
4
5
6
7
8
9
10
11
wget https://archive.apache.org/dist/lucene/solr/5.1.0/solr-5.1.0.tgz
tar xzf solr-5.1.0.tgz solr-5.1.0/bin/install_solr_service.sh --strip-components=2
sudo bash ./install_solr_service.sh solr-5.1.0.tgz
sudo -u solr mkdir -p /var/solr/data/configsets/ckan/conf
sudo ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml /var/solr/data/configsets/ckan/conf/schema.xml
sudo -u solr cp /opt/solr/server/solr/configsets/basic_configs/conf/solrconfig.xml /var/solr/data/configsets/ckan/conf/.
sudo -u solr cp /opt/solr/server/solr/configsets/basic_configs/conf/synonyms.txt /var/solr/data/configsets/ckan/conf/.
sudo -u solr cp /opt/solr/server/solr/configsets/basic_configs/conf/protwords.txt /var/solr/data/configsets/ckan/conf/.
wget http://search.maven.org/remotecontent?filepath=com/vividsolutions/jts/1.13/jts-1.13.jar -O jts-1.13.jar
sudo -u solr cp jts-1.13.jar /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/.
sudo service solr restart
Go to the following link to create the SOLR Core
http://localhost:8983/solr/admin/cores?action=CREATE&name=ckan&configSet=ckan
On Wed, May 25, 2016 at 3:18 PM, David Read <david.read at hackneyworkshop.com> wrote:
Derek,
Where did you get the idea ckan only supports solr v3.6? Because
wherever it is, it needs fixing.
I think most people run SOLR v4 and several are on v5.
e.g. https://github.com/ckan/ckan/issues/2318
Dave
On 25 May 2016 at 10:55, Derek Hohls <dhohls at csir.co.za> wrote:
> Hi
>
> Hope the list can help with this problem...
>
> I have setup and run a full CKAN install without problems in a VM.
>
> Now I am trying to run solr in a separate container. I have done this,
> using the Docker image from https://hub.docker.com/r/idguk/docker-solr/ (the
> official solr Docker could not be used as it does not support the very old
> version - 3.6 - which is used by CKAN). The Docker is linked, via the "-v"
> option, to the `schema.xml` file sourced from
> `python-ckan_2.5-trusty_amd64.deb`.
>
> However, calling:
>
> ckan db init
>
> triggers off an error; specifically this:
>
> Traceback (most recent call last):
> File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/common.py", line 51,
> in is_available
> conn.query("*:*", rows=1)
> File
> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
> 703, in query
> return self.select(*args, **params)
> File
> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
> 798, in __call__
> xml = self.raw(**params)
> File
> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
> 823, in raw
> rsp = conn._post(self.selector, request, conn.form_headers)
> File
> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
> 639, in _post
> return check_response_status(self.conn.getresponse())
> File
> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
> 1097, in check_response_status
> raise ex
>
> On checking the `solr/core.py` file, it seems as if its the call to
> `solr/select/` that triggers the issue.
>
> Access via a browser to this URL (the container version of solr) shows a
> Java error:
>
> java.lang.NullPointerException
> at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:92)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:306)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>
> But in the VM-version of solr, the same call generates an XML document:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">1</int>
> <lst name="params"/></lst>
> <result name="response" numFound="0" start="0"/>
> </response>
>
> I'd appreciate if anyone could highlight what I have missed in the
> configuration that could cause this error.
>
> Thanks
> Derek
>
>
> --
> This message is subject to the CSIR's copyright terms and conditions, e-mail
> legal notice, and implemented Open Document Format (ODF) standard.
> The full disclaimer details can be found at
> http://www.csir.co.za/disclaimer.html.
>
>
> This message has been scanned for viruses and dangerous content by
> MailScanner,
> and is believed to be clean.
>
>
> Please consider the environment before printing this email.
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
https://lists.okfn.org/mailman/listinfo/ckan-dev
Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
--
Pedro Gracio Silva
Aerospace Systems / Production Center
Av. D. Joao II, Lt 1.17.01, 10 ? 1998-023 Lisboa, Portugal
Ph: +351 21 893 3020 ? Fax: +351 21 896 9099
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.
Please consider the environment before printing this email.
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.
Please consider the environment before printing this email.
--
Pedro Gracio Silva
Aerospace Systems / Production Center
Av. D. Joao II, Lt 1.17.01, 10 ? 1998-023 Lisboa, Portugal
Ph: +351 21 893 3020 ? Fax: +351 21 896 9099
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.
Please consider the environment before printing this email.
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.
Please consider the environment before printing this email.
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.
Please consider the environment before printing this email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20160530/f58c8f18/attachment-0003.html>
More information about the ckan-dev
mailing list