[ckan-dev] [CKAN][HARVESTER] Solr issue

Sun Oct 13 16:18:44 UTC 2013

Hi All,

I'm having a strange behavior when testing Supervisor during the ckanext-harvest installation (See https://github.com/okfn/ckanext-harvest#setting-up-the-harvesters-on-a-production-server).

When launching :

$ sudo supervisorctl start ckan_gather_consumer
ckan_gather_consumer: ERROR (abnormal termination)

The log shows :

Traceback (most recent call last):
  File "/opt/ckan/default/bin/paster", line 9, in <module>
    load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
  File "/opt/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 104, in run
    invoke(command, command_name, options, args[1:])
  File "/opt/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 143, in invoke
    exit_code = runner.run(args)
  File "/opt/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 238, in run
    result = self.command()
  File "/opt/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 88, in command
    self._load_config()
  File "/opt/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 147, in _load_config
    super(Harvester, self)._load_config()
  File "/opt/ckan/default/src/ckan/ckan/lib/cli.py", line 94, in _load_config
    load_environment(conf.global_conf, conf.local_conf)
  File "/opt/ckan/default/src/ckan/ckan/config/environment.py", line 232, in load_environment
    p.load_all(config)
  File "/opt/ckan/default/src/ckan/ckan/plugins/core.py", line 124, in load_all
    unload_all()
  File "/opt/ckan/default/src/ckan/ckan/plugins/core.py", line 182, in unload_all
    unload(*reversed(_PLUGINS))
  File "/opt/ckan/default/src/ckan/ckan/plugins/core.py", line 210, in unload
    plugins_update()
  File "/opt/ckan/default/src/ckan/ckan/plugins/core.py", line 116, in plugins_update
    environment.update_config()
  File "/opt/ckan/default/src/ckan/ckan/config/environment.py", line 270, in update_config
    search.check_solr_schema_version()
  File "/opt/ckan/default/src/ckan/ckan/lib/search/__init__.py", line 291, in check_solr_schema_version
    res = urllib2.urlopen(req)
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1177, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno -2] Name or service not known>

The solr url is set as follows :

solr_url = http://localhost:8080/solr/

(solr runs under tomcat6)

The CKAN admin url est working : http://localhost:8080/solr/admin/file/?file=schema.xml (I get the schema-2.0.xml)

When running by hand the following command :

$ /opt/ckan/default/bin/paster --plugin=ckanext-harvest harvester gather_consumer --config=/etc/ckan/default/production.ini

everything's working without error ...

Do you have any clue ?

didier