[ckan-dev] problems configuring SOLR 4.5.0 with CKAN 2.0
Elena Camossi
elena.camossi at ext.jrc.ec.europa.eu
Thu Oct 31 16:02:16 UTC 2013
Hi everyone,
I'm trying to install CKAN 2.0 on Linux RedHAT using last stable version of
Solr, 4.5.0, with Tomcat 7.
After having solved some issue with solr installation (it has changed quite
a bit with respect to the steps in CKAN documentation, that were for an
older version of Solr apparently) I have my Solr instance running, but it's
still not working correctly with CKAN.
I'm at the final configuration steps, but when I create the database tables
with command
paster db init -c /etc/ckan/production.ini
I get the following error
2013-10-31 15:17:07,344 ERROR [ckan.lib.search.common] HTTP code=404,
reason=Not Found
Traceback (most recent call last):
File "/usr/local/ckan/pyenv/src/ckan/ckan/lib/search/common.py", line 51,
in is_available
conn.query("*:*", rows=1)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 703, in query
return self.select(*args, **params)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 798, in __call__
xml = self.raw(**params)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 823, in raw
rsp = conn._post(self.selector, request, conn.form_headers)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 639, in _post
return check_response_status(self.conn.getresponse())
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 1097, in check_response_status
raise ex
SolrException: HTTP code=404, reason=Not Found
2013-10-31 15:17:07,345 WARNI [ckan.lib.search] Problems were found while
connecting to the SOLR server
2013-10-31 15:17:07,516 INFO [ckan.model] CKAN database version remains as:
68
2013-10-31 15:17:07,564 INFO [ckan.model] Database initialised
Initialising DB: SUCCESS
when I add the user admin
(with command paster --plugin=ckan sysadmin add admin -c
/etc/ckan/production.ini, because I found out that
paster --plugin=ckan user add admin -c /etc/ckan/production.ini doesn't
work)
I got a similar error
paster --plugin=ckan sysadmin add admin -c /etc/ckan/production.ini
2013-10-31 15:24:31,262 ERROR [ckan.lib.search.common] HTTP code=404,
reason=Not Found
Traceback (most recent call last):
File "/usr/local/ckan/pyenv/src/ckan/ckan/lib/search/common.py", line 51,
in is_available
conn.query("*:*", rows=1)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 703, in query
return self.select(*args, **params)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 798, in __call__
xml = self.raw(**params)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 823, in raw
rsp = conn._post(self.selector, request, conn.form_headers)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 639, in _post
return check_response_status(self.conn.getresponse())
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 1097, in check_response_status
raise ex
SolrException: HTTP code=404, reason=Not Found
2013-10-31 15:24:31,263 WARNI [ckan.lib.search] Problems were found while
connecting to the SOLR server
Then user admin is added
User "admin" not found
Create new user: admin? [y/n]y
Password:
Confirm password:
Creating admin user
Added admin as sysadmin
and when I check the user list with command paster --plugin=ckan user list
-c /etc/ckan/production.ini
i see the user has been added but I have still an error
Traceback (most recent call last):
File "/usr/local/ckan/pyenv/src/ckan/ckan/lib/search/common.py", line 51,
in is_available
conn.query("*:*", rows=1)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 703, in query
return self.select(*args, **params)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 798, in __call__
xml = self.raw(**params)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 823, in raw
rsp = conn._post(self.selector, request, conn.form_headers)
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 639, in _post
return check_response_status(self.conn.getresponse())
File "/usr/local/ckan/pyenv/lib/python2.6/site-packages/solr/core.py",
line 1097, in check_response_status
raise ex
SolrException: HTTP code=404, reason=Not Found
2013-10-31 15:39:13,128 WARNI [ckan.lib.search] Problems were found while
connecting to the SOLR server
Users:
count = 4
name=
name=admin
name=logged_in
name=visitor
I went through the next steps of CKAN configuration, but of course it
doesn't work, and the problem is most likely related to Solr configuration.
I post below the steps I followed for configuring Solr, hoping that someone
can give me some hint on when i went wrong.
regards,
-Elena
-------------------------------------- Installing and configuring SOLR
----------------------------------------------------------------------------
-----
1) Download and extract Apache SOLR (if you get an error, check if the
name of the .tgz file is solr-xxx.tgz or apache-solr-xxx.tgz)
curl http://archive.apache.org/dist/lucene/solr/4.5.0/solr-4.5.0.tgz | tar
xzf -
2) Create directories to hold SOLR cores (just one, in our case, called
ckan):
mkdir -p /usr/share/solr/ckan /var/lib/solr/data/ckan /etc/solr/ckan
. /usr/share/solr/ckan is solr core home, where directory conf is
. /var/lib/solr/data/ckan is data directory
. /etc/solr/ckan is the actual folder for core configuration files
3) Cd to the directory where solr-4.5.0.tgz was downloaded and unpacked
and Copy the Apache SOLR war to the desired location (you can eventually
change the name of the file to solr.war)
cp solr-4.5.0/dist/solr-4.5.0.war /usr/share/solr
cp -r solr-4.5.0/example/solr/collection1/conf /etc/solr/ckan
4) Create directory in solr home for libs and copy all .jar there
mkdir /usr/share/solr/lib
cp solr-4.5.0/dist/* /usr/share/solr/lib
cp solr-4.5.0/contrib/analysis-extras/lib/* /usr/share/solr/lib
cp solr-4.5.0/contrib/clustering/lib/* /usr/share/solr/lib
cp solr-4.5.0/contrib/dataimporthandler/lib/* /usr/share/solr/lib
cp solr-4.5.0/contrib/extraction/lib/* /usr/share/solr/lib
cp solr-4.5.0/contrib/langid/lib/* /usr/share/solr/lib
cp solr-4.5.0/contrib/uima/lib/* /usr/share/solr/lib
cp solr-4.5.0/contrib/velocity/lib/* /usr/share/solr/lib
To avoid copying all jar, consider to edit file
/etc/solr/ckan/conf/solrconfig.xml to remove all unnecessary sections (what
is not required by CKAN?)
5) Remove or comment in /etc/solr/ckan/conf/solrconfig.xml < dataDir>
section, which is no longer necessary
6) Create a symbolic link between the configurations in /etc and /usr.
ln -s /etc/solr/ckan/conf /usr/share/solr/ckan/conf
7) Remove the provided schema from the configured core and link the
schema files in the CKAN source.
rm -f /etc/solr/ckan/conf/schema.xml
ln -s /usr/local/ckan/pyenv/src/ckan/ckan/config/solr/schema-2.0.xml
/etc/solr/ckan/conf/schema.xml
8) For SOLR multicore installation, for each core create in
/opt/tomcat7/conf/Catalina/localhost/ a new xml file specifying the context
(the solr home) for each core, each with the following content
<Context docBase="/usr/share/solr/solr-4.5.0.war" crossContext="true">
<Environment name="solr/home" type="java.lang.String" value="[complete
path to solr core home]" override="true" />
</Context>
For example, create file /opt/tomcat7/conf/Catalina/localhost/solr.xml:
<Context docBase="/usr/share/solr/solr-4.5.0.war" crossContext="true">
<Environment name="solr/home" type="java.lang.String"
value="/usr/share/solr" override="true" />
</Context>
8) Create a new file, called /usr/share/solr/solr.xml, specifying the
configuration for each core. For core ckan is:
<solr persistent="true" sharedLib="lib">
<cores adminPath="/admin/cores" defaultCoreName="ckan">
<core name ="ckan-schema-2.0" instanceDir="ckan"> <property
name="dataDir" value="/var/lib/solr/data/ckan" /></core>
</cores>
</solr>
Enabling Tomcat
1) Copy the solr jars library from /usr/share/solr/conf/lib/ into your
container's main lib directory (Tomcat). These jars will set up SLF4J and
log4j, Solr's new logging features.
cd <solr download directory>
cp -f solr-4.5.0/dist/* /opt/tomcat7/lib/
2) Set Permissions to enable tomcat user to access solr configuration
chown -R tomcat:tomcat /usr/share/solr /var/lib/solr
3) Set up tomcat to start at startup, or restart the service is it's
running
chkconfig tomcat on
service tomcat start
or
service tomcat restart
4) Check if Tomcat and Solr are running correctly:
go to http://localhost:8080/ and check if tomcat is running.
Then, go to http://localhost:8080/manager/html and connect as admin.
Among the applications you should see solr, and going to
http://localhost:80804/solr you should access Solr admin page.
The path depends on the name of the file in
/opt/tomcat7/conf/Catalina/localhost. It is [.]/solr if the name of the file
is /opt/tomcat7/conf/Catalina/localhost/solr.xml.
More information about the ckan-dev
mailing list