[ckan-dev] Error running solr in a container
Adrià Mercader
adria.mercader at okfn.org
Tue May 31 09:20:26 UTC 2016
Hi Derek,
If I understood correctly, you created a Solr core named "ckan". In that
case your sorl_url setting in the CKAN ini file should be:
solr_url = http://{solr_host}/solr/ckan
Replace {solr_host} with the appropriate value.
Does this help?
Adrià
On 30 May 2016 at 10:39, Derek Hohls <dhohls at csir.co.za> wrote:
> Hi again
>
> After some further work, it seems as though the ckan "core" is now
> correctly setup for solr; the admin page at http://my.ip:8983/solr/#/ckan
> displays without any obvious errors (although, to be honest, I do not know
> enough to tell if there is something missing that should be there).
>
> I have tried to implement all Pedro's changes for my solr 5 instance
> (adding the new .jar file is a bit tricky, as one cannot restart the solr
> service from inside the container...)
>
> I am still hitting the same error though: the call being made to
> /solr/select/ by CKAN causes a 404 error; which can be duplicated in the
> browser:
> HTTP ERROR 404
>
> Problem accessing /solr/select/. Reason:
>
> Not Found
>
> ------------------------------
> *Powered by Jetty://*
>
> There is nothing obvious in the solr.log file - running a tail while
> doing a page refresh does not trigger any new log entry,
>
> I am not quite sure how to proceed from here? Any other ideas or
> suggestions would be welcome!
>
> Thanks
> Derek
>
>
> >>> "Derek Hohls" <dhohls at csir.co.za> 27/05/16 5:16 PM >>>
> Just for the record, the log is under:
>
> /opt/solr/server/logs/solr.log
>
> I am working on automating the install by extracting the necessary files
> from a " run once" instance of a solr container, then overwriting the
> default schema.xml with the one supplied by CKAN, then remounting all these
> files back into the final instance of solr (as per instructions in
> https://github.com/docker-solr/docker-solr/blob/master/5.3/scripts/docker-entrypoint.sh)
> to auto-create the ckan core files.
>
> The solr container is working; but I need to get the CKAN container to
> communicate with it.
>
> Thanks for the help so far.
>
> Derek
>
>
> >>> Pedro G Silva <pedro.gracio at deimos.com.pt> 27/05/16 11:45 AM >>>
> logs are in /var/solr/logs
> if you want to automate the install you can just curl or wget that webpage
> to create the core
>
> On Thu, May 26, 2016 at 2:09 PM, Derek Hohls <dhohls at csir.co.za> wrote:
>
>> Hi Pedro
>>
>> Thanks for all the help here! - I did suspect it might be a missing core
>> issue. I am still on a learning curve with these technologies and am not
>> sure what is / is not a requirement or pre-requisite for each.
>>
>> I obviously missed the part in the CKAN install guide where I had to
>> manually create the core; but I will look that up as I need to find a way
>> to fully automate it at install time (i.e. without access to a browser as
>> you suggest).
>>
>> Have you any quick pointers about where the solr log is? One of my
>> frustrations about Java-based installs is that developers seem to get very
>> creative about where everything goes (even between installs of exactly the
>> same package). It appears that the Docker install has placed everything
>> under /opt/solr but I see your commands reference /var/solr ...
>>
>> I will check, in the meantime, whether these other files you referenced,
>> are correct on my side.
>>
>> Thanks
>> Derek
>>
>>
>> >>> Pedro G Silva <pedro.gracio at deimos.com.pt> 25/05/16 4:55 PM >>>
>> Im using solr 5. Derek try 2 things:
>> -see if you have a proxy set somewhere
>> -check the solr logs for more detailed info
>> -check that in that solr version that you have you created the core, on
>> version 5 the link to solr is solr_url:8983/solr/core_name , and you have
>> to manually create this core, since solr 5 doesn't automatically create
>> cores. also you should check that all needed files were copied. I'll leave
>> a snippet of my commands. Hope it helps
>> 1
>> 2
>> 3
>> 4
>> 5
>> 6
>> 7
>> 8
>> 9
>> 10
>> 11
>> wget https://archive.apache.org/dist/lucene/solr/5.1.0/solr-5.1.0.tgz
>> tar xzf solr-5.1.0.tgz solr-5.1.0/bin/install_solr_service.sh
>> --strip-components=2
>> sudo bash ./install_solr_service.sh solr-5.1.0.tgz
>> sudo -u solr mkdir -p /var/solr/data/configsets/ckan/conf
>> sudo ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml
>> /var/solr/data/configsets/ckan/conf/schema.xml
>> sudo -u solr cp
>> /opt/solr/server/solr/configsets/basic_configs/conf/solrconfig.xml
>> /var/solr/data/configsets/ckan/conf/.
>> sudo -u solr cp
>> /opt/solr/server/solr/configsets/basic_configs/conf/synonyms.txt
>> /var/solr/data/configsets/ckan/conf/.
>> sudo -u solr cp
>> /opt/solr/server/solr/configsets/basic_configs/conf/protwords.txt
>> /var/solr/data/configsets/ckan/conf/.
>> wget http://search.maven.org/remotecontent?filepath=com
>> /vividsolutions/jts/1.13/jts-1.13.jar -O jts-1.13.jar
>> sudo -u solr cp jts-1.13.jar
>> /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/.
>> sudo service solr restart
>>
>> Go to the following link to create the SOLR Core
>>
>>
>> http://localhost:8983/solr/admin/cores?action=CREATE&name=ckan&configSet=ckan
>>
>> On Wed, May 25, 2016 at 3:18 PM, David Read <
>> david.read at hackneyworkshop.com> wrote:
>>
>>> Derek,
>>>
>>> Where did you get the idea ckan only supports solr v3.6? Because
>>> wherever it is, it needs fixing.
>>>
>>> I think most people run SOLR v4 and several are on v5.
>>> e.g. https://github.com/ckan/ckan/issues/2318
>>>
>>> Dave
>>>
>>> On 25 May 2016 at 10:55, Derek Hohls <dhohls at csir.co.za> wrote:
>>> > Hi
>>> >
>>> > Hope the list can help with this problem...
>>> >
>>> > I have setup and run a full CKAN install without problems in a VM.
>>> >
>>> > Now I am trying to run solr in a separate container. I have done this,
>>> > using the Docker image from
>>> https://hub.docker.com/r/idguk/docker-solr/ (the
>>> > official solr Docker could not be used as it does not support the very
>>> old
>>> > version - 3.6 - which is used by CKAN). The Docker is linked, via the
>>> "-v"
>>> > option, to the `schema.xml` file sourced from
>>> > `python-ckan_2.5-trusty_amd64.deb`.
>>> >
>>> > However, calling:
>>> >
>>> > ckan db init
>>> >
>>> > triggers off an error; specifically this:
>>> >
>>> > Traceback (most recent call last):
>>> > File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/common.py",
>>> line 51,
>>> > in is_available
>>> > conn.query("*:*", rows=1)
>>> > File
>>> >
>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>> > 703, in query
>>> > return self.select(*args, **params)
>>> > File
>>> >
>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>> > 798, in __call__
>>> > xml = self.raw(**params)
>>> > File
>>> >
>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>> > 823, in raw
>>> > rsp = conn._post(self.selector, request, conn.form_headers)
>>> > File
>>> >
>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>> > 639, in _post
>>> > return check_response_status(self.conn.getresponse())
>>> > File
>>> >
>>> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/solr/core.py", line
>>> > 1097, in check_response_status
>>> > raise ex
>>> >
>>> > On checking the `solr/core.py` file, it seems as if its the call to
>>> > `solr/select/` that triggers the issue.
>>> >
>>> > Access via a browser to this URL (the container version of solr) shows
>>> a
>>> > Java error:
>>> >
>>> > java.lang.NullPointerException
>>> > at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:92)
>>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>> > at
>>> > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>> > at
>>> >
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>>> > at
>>> >
>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:306)
>>> > at
>>> >
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>> > at
>>> >
>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>> > at
>>> >
>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>> > at
>>> >
>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>> > at
>>> >
>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>>> > at
>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>> > at
>>> >
>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>> > at
>>> >
>>> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>>> > at
>>> >
>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>> > at org.mortbay.jetty.Server.handle(Server.java:326)
>>> > at
>>> > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>> > at
>>> >
>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>> > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>> > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>> > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>> > at
>>> >
>>> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
>>> > at
>>> >
>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>> >
>>> > But in the VM-version of solr, the same call generates an XML document:
>>> >
>>> > <?xml version="1.0" encoding="UTF-8"?>
>>> > <response>
>>> > <lst name="responseHeader">
>>> > <int name="status">0</int>
>>> > <int name="QTime">1</int>
>>> > <lst name="params"/></lst>
>>> > <result name="response" numFound="0" start="0"/>
>>> > </response>
>>> >
>>> > I'd appreciate if anyone could highlight what I have missed in the
>>> > configuration that could cause this error.
>>> >
>>> > Thanks
>>> > Derek
>>> >
>>> >
>>> > --
>>> > This message is subject to the CSIR's copyright terms and conditions,
>>> e-mail
>>> > legal notice, and implemented Open Document Format (ODF) standard.
>>> > The full disclaimer details can be found at
>>> > http://www.csir.co.za/disclaimer.html.
>>> >
>>> >
>>> > This message has been scanned for viruses and dangerous content by
>>> > MailScanner,
>>> > and is believed to be clean.
>>> >
>>> >
>>> > Please consider the environment before printing this email.
>>> >
>>> >
>>> > _______________________________________________
>>> > ckan-dev mailing list
>>> > ckan-dev at lists.okfn.org
>>> > https://lists.okfn.org/mailman/listinfo/ckan-dev
>>> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>> >
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>
>>
>>
>>
>> --
>>
>>
>> [image: DEIMOS Engenharia, SA] <http://www.deimos.pt>
>> *Pedro Gracio Silva*
>> Aerospace Systems / Production Center
>>
>> *Av. D. Joao II, Lt 1.17.01, 10 ? 1998-023 Lisboa, Portugal*
>> Ph: +351 21 893 3020 ? Fax: +351 21 896 9099
>>
>>
>>
>> --
>> This message is subject to the CSIR's copyright terms and conditions,
>> e-mail legal notice, and implemented Open Document Format (ODF) standard.
>> The full disclaimer details can be found at
>> http://www.csir.co.za/disclaimer.html.
>>
>>
>> This message has been scanned for viruses and dangerous content by
>> *MailScanner* <http://www.mailscanner.info/>,
>> and is believed to be clean.
>>
>>
>> Please consider the environment before printing this email.
>>
>> --
>> This message is subject to the CSIR's copyright terms and conditions,
>> e-mail legal notice, and implemented Open Document Format (ODF) standard.
>> The full disclaimer details can be found at
>> http://www.csir.co.za/disclaimer.html.
>>
>>
>> This message has been scanned for viruses and dangerous content by
>> *MailScanner* <http://www.mailscanner.info/>,
>> and is believed to be clean.
>>
>>
>> Please consider the environment before printing this email.
>>
>>
>
>
> --
>
>
> [image: DEIMOS Engenharia, SA] <http://www.deimos.pt>
> *Pedro Gracio Silva*
> Aerospace Systems / Production Center
>
> *Av. D. Joao II, Lt 1.17.01, 10 ? 1998-023 Lisboa, Portugal*
> Ph: +351 21 893 3020 ? Fax: +351 21 896 9099
>
>
>
> --
> This message is subject to the CSIR's copyright terms and conditions,
> e-mail legal notice, and implemented Open Document Format (ODF) standard.
> The full disclaimer details can be found at
> http://www.csir.co.za/disclaimer.html.
>
>
> This message has been scanned for viruses and dangerous content by
> *MailScanner* <http://www.mailscanner.info/>,
> and is believed to be clean.
>
>
> Please consider the environment before printing this email.
>
> --
> This message is subject to the CSIR's copyright terms and conditions,
> e-mail legal notice, and implemented Open Document Format (ODF) standard.
> The full disclaimer details can be found at
> http://www.csir.co.za/disclaimer.html.
>
>
> This message has been scanned for viruses and dangerous content by
> *MailScanner* <http://www.mailscanner.info/>,
> and is believed to be clean.
>
>
> Please consider the environment before printing this email.
>
> --
> This message is subject to the CSIR's copyright terms and conditions,
> e-mail legal notice, and implemented Open Document Format (ODF) standard.
> The full disclaimer details can be found at
> http://www.csir.co.za/disclaimer.html.
>
>
> This message has been scanned for viruses and dangerous content by
> *MailScanner* <http://www.mailscanner.info/>,
> and is believed to be clean.
>
>
> Please consider the environment before printing this email.
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20160531/1bacd8cf/attachment-0003.html>
More information about the ckan-dev
mailing list