[ckan-dev] Problems installing harvesting exctension for ckan

Mon Jan 26 21:21:42 UTC 2015

(I'm a newbie arounde here, so please let me know if this is not the right
place to pose these questions, and point to the proper mail list.)

I have installed ckan 2.2.1 from package in a vritualbox machine with
ubuntu server 12.04, with filestore, datastore and datapusher installed and
working ok.

I'm following this guide (
https://github.com/ckan/ckanext-harvest/blob/master/README.rst) and I have
several questions:

1. Where should I install the plugin?

I guess is at '/usr/lib/ckan/default' so that it gest installed on '
/usr/lib/ckan/default/src/ckanext-harvest'

2. When I do the pip install part I have to run it with sudo (and I guess
that arises other troubles)

$ pip install -e git+
https://github.com/okfn/ckanext-harvest.git@stable#egg=ckanext-harvest
OSError: [Errno 13] Permission denied: '/usr/lib/ckan/default/build'

Same with the requirements

$ pip install -r pip-requirements.txt
OSError: [Errno 13] Permission denied: '/usr/lib/ckan/default/build'

3. The 'paster harvester gather_consumer' command does not work

After installing everything with sudo I create a source with

paster --plugin=ckanext-harvest harvester source "ckan3" "http://ckan3"
"ckan" "ckan3 harvesting" True "org1" "Manual"
"{\"default_tags\":[\"ckan3\",\"harvest\"],\"default_groups\":[\"ckan3\"],\"user\":\"ckanadmin\",\"read_only\":true}"
--config=/etc/ckan/default/production.ini

But when I try to start the gather_consumer service I get the following
error:

$ paster --plugin=ckanext-harvest harvester gather_consumer
--config=/etc/ckan/default/production.ini
  File
"/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/queue.py", line
5, in <module>
    import pika
ImportError: No module named pika

I guess the problem is I installed everything with sudo, so pika module
canno be found.

So I have to

$ sudo su
$ source /usr/lib/ckan/default/bin/activate
$ cd /usr/lib/ckan/default/src/ckanext-harvest
$ pip install -r pip-requirements.txt

4. Socket error: connection refused when trying to connect to redis

After that, when I run (as root) this command:

paster --plugin=ckanext-harvest harvester gather_consumer
--config=/etc/ckan/default/production.ini

pika connection using {'retry_delay': 2.0, 'frame_max': 10000,
'channel_max': 0, 'locale': 'en_US', 'socket_timeout': 0.25, 'ssl': False,
'host': 'localhost', 'ssl_options': {}, 'virtual_host': '/', 'heartbeat':
0, 'credentials': <pika.credentials.PlainCredentials object at 0x39cc550>,
'backpressure_detection': False, 'port': 5672, 'connection_attempts': 1}

2015-01-23 19:21:22,782 ERROR [pika.adapters.base_connection] socket error:
Connection refused
2015-01-23 19:21:22,782 WARNI [pika.adapters.base_connection] Could not
connect due to "Connection refused," retrying in 2 sec
2015-01-23 19:21:22,783 ERROR [pika.adapters.base_connection] Could not
connect: Connection refused

  File
"/usr/lib/ckan/default/local/lib/python2.7/site-packages/pika/adapters/base_connection.py",
line 120, in _adapter_connect
    self.params.retry_delay)
pika.exceptions.AMQPConnectionError: 2.0

---

Those are the errors I found trying to install the harvest extension. I'd
like to solve them and then, help improve the documentation. Any help will
be much appreciated, of course

Saludos

Sas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20150126/3b56a5b5/attachment-0002.html>