[ckan-dev] Problems installing harvesting extension for ckan

Tue Jan 27 06:38:53 UTC 2015

Newbie error here, I added the "ckan.harvest.mq.type = redis" configuration
at the end of the file, without realizing it was in a different section
(formatter_generic), I moved it to [app:main] section and now at least it
works, anyway the rest of the questions are still valid, in case anybody
can give me a hand.

Here's someone who seemed to have the same problem:
https://lists.okfn.org/pipermail/ckan-dev/2014-November/008325.html

saludos

sas

---------- Forwarded message ----------

> From: Sebastian Scarano <opensas at gmail.com>
> To: ckan-dev at lists.okfn.org
> Cc:
> Date: Mon, 26 Jan 2015 18:21:42 -0300
> Subject: [ckan-dev] Problems installing harvesting exctension for ckan
> (I'm a newbie arounde here, so please let me know if this is not the right
> place to pose these questions, and point to the proper mail list.)
>
> I have installed ckan 2.2.1 from package in a vritualbox machine with
> ubuntu server 12.04, with filestore, datastore and datapusher installed and
> working ok.
>
>
> I'm following this guide (
> https://github.com/ckan/ckanext-harvest/blob/master/README.rst) and I
> have several questions:
>
> 1. Where should I install the plugin?
>
> I guess is at '/usr/lib/ckan/default' so that it gest installed on '
> /usr/lib/ckan/default/src/ckanext-harvest'
>
> 2. When I do the pip install part I have to run it with sudo (and I guess
> that arises other troubles)
>
> $ pip install -e git+
> https://github.com/okfn/ckanext-harvest.git@stable#egg=ckanext-harvest
> OSError: [Errno 13] Permission denied: '/usr/lib/ckan/default/build'
>
> Same with the requirements
>
> $ pip install -r pip-requirements.txt
> OSError: [Errno 13] Permission denied: '/usr/lib/ckan/default/build'
>
> 3. The 'paster harvester gather_consumer' command does not work
>
> After installing everything with sudo I create a source with
>
> paster --plugin=ckanext-harvest harvester source "ckan3" "http://ckan3"
> "ckan" "ckan3 harvesting" True "org1" "Manual"
> "{\"default_tags\":[\"ckan3\",\"harvest\"],\"default_groups\":[\"ckan3\"],\"user\":\"ckanadmin\",\"read_only\":true}"
> --config=/etc/ckan/default/production.ini
>
> But when I try to start the gather_consumer service I get the following
> error:
>
> $ paster --plugin=ckanext-harvest harvester gather_consumer
> --config=/etc/ckan/default/production.ini
>   File
> "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/queue.py", line
> 5, in <module>
>     import pika
> ImportError: No module named pika
>
> I guess the problem is I installed everything with sudo, so pika module
> canno be found.
>
> So I have to
>
> $ sudo su
> $ source /usr/lib/ckan/default/bin/activate
> $ cd /usr/lib/ckan/default/src/ckanext-harvest
> $ pip install -r pip-requirements.txt
>
> 4. Socket error: connection refused when trying to connect to redis
>
> After that, when I run (as root) this command:
>
> paster --plugin=ckanext-harvest harvester gather_consumer
> --config=/etc/ckan/default/production.ini
>
> pika connection using {'retry_delay': 2.0, 'frame_max': 10000,
> 'channel_max': 0, 'locale': 'en_US', 'socket_timeout': 0.25, 'ssl': False,
> 'host': 'localhost', 'ssl_options': {}, 'virtual_host': '/', 'heartbeat':
> 0, 'credentials': <pika.credentials.PlainCredentials object at 0x39cc550>,
> 'backpressure_detection': False, 'port': 5672, 'connection_attempts': 1}
>
> 2015-01-23 19:21:22,782 ERROR [pika.adapters.base_connection] socket
> error: Connection refused
> 2015-01-23 19:21:22,782 WARNI [pika.adapters.base_connection] Could not
> connect due to "Connection refused," retrying in 2 sec
> 2015-01-23 19:21:22,783 ERROR [pika.adapters.base_connection] Could not
> connect: Connection refused
>
>   File
> "/usr/lib/ckan/default/local/lib/python2.7/site-packages/pika/adapters/base_connection.py",
> line 120, in _adapter_connect
>     self.params.retry_delay)
> pika.exceptions.AMQPConnectionError: 2.0
>
> ---
>
> Those are the errors I found trying to install the harvest extension. I'd
> like to solve them and then, help improve the documentation. Any help will
> be much appreciated, of course
>
> Saludos
>
> Sas
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20150127/daaa4a01/attachment-0002.html>