[ckan-dev] Configuring the datastore

• Previous message: [ckan-dev] Configuring the datastore
• Next message: [ckan-dev] Prelim SPARQL endpoint - request for test
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Okay, I'm completely stumped.

Following the directions in the README file, I install ckanext-datastorer from source (but I pass the –I switch to pip otherwise I run into other issues).

It mostly succeeds except it is unable install ckanext-archiver.

So I can pip install requires.txt (again, with the –I switch).

Again, that seems to have mostly succeeded.

Then I apt-get install supervisor and setup the config file as in the README.

No celery process is started.

Manually running the following command: paster --plugin=ckan celeryd --config=/etc/ckan/std/std.ini
Produces a traceback:

pkg_resources.VersionConflict: (SQLAlchemy 0.5.8 (/usr/lib/pymodules/python2.6), Requirement.parse('SQLAlchemy>=0.6.6'))

And trying the alternate command: sudo -u ckanstd /var/lib/ckan/std/pyenv/bin/paster --plugin=ckan celeryd --config=/etc/ckan/std/std.ini
Produces the following error:
sudo: /var/lib/ckan/std/pyenv/bin/paster: command not found

The log from pip follow below. I'm quite confused by the directions in the README file. They seem to conflict with the documentation about installing CKAN extensions. Further the Datastore API documentation does not make any mention about the need to explicitly install the ckanext-datastore package. It just says to edit the ckan config file and install elastic search.

Thanks,
Salman

shaq at u10:~/git/ckanext-datastorer$ sudo pip install -e ./ -I
Checking out file:///home/shaq/git/ckanext-datastorer
  Running setup.py egg_info for package from file:///home/shaq/git/ckanext-datastorer
Downloading/unpacking messytables>=0.2 (from ckanext-datastorer)
  Downloading messytables-0.2.0.tar.gz
  Running setup.py egg_info for package messytables
Downloading/unpacking celery>=2.4.2 (from ckanext-datastorer)
  Downloading celery-2.5.3.tar.gz (946Kb): 946Kb downloaded
  Running setup.py egg_info for package celery
    no previously-included directories found matching 'tests/*.pyc'
    no previously-included directories found matching 'docs/*.pyc'
    no previously-included directories found matching 'contrib/*.pyc'
    no previously-included directories found matching 'celery/*.pyc'
    no previously-included directories found matching 'examples/*.pyc'
    no previously-included directories found matching 'bin/*.pyc'
    no previously-included directories found matching 'docs/.build'
    no previously-included directories found matching 'docs/graffles'
    no previously-included directories found matching '.tox/*'
Downloading/unpacking kombu-sqlalchemy==1.1.0 (from ckanext-datastorer)
  Could not fetch URL http://pypi.python.org/simple/kombu-sqlalchemy/1.1.0: HTTP Error 404: Not Found (kombu-sqlalchemy/1.1.0)
  Will skip URL http://pypi.python.org/simple/kombu-sqlalchemy/1.1.0 when looking for download links for kombu-sqlalchemy==1.1.0 (from ckanext-datastorer)
  Downloading kombu-sqlalchemy-1.1.0.tar.gz
  Running setup.py egg_info for package kombu-sqlalchemy
    warning: no files found matching '*' under directory 'djkombu'
Downloading/unpacking ckanext-archiver>=0.0 (from ckanext-datastorer)
  Could not fetch URL http://pypi.python.org/simple/ckanext-archiver: HTTP Error 404: Not Found (ckanext-archiver does not have any releases)
  Will skip URL http://pypi.python.org/simple/ckanext-archiver when looking for download links for ckanext-archiver>=0.0 (from ckanext-datastorer)
  Could not find any downloads that satisfy the requirement ckanext-archiver>=0.0 (from ckanext-datastorer)
No distributions at all found for ckanext-archiver>=0.0 (from ckanext-datastorer)
Storing complete log in ./pip-log.txt

shaq at u10:~/git/ckanext-datastorer$ sudo pip install -r requires.txt -I
Checking out messytables from git+https://github.com/okfn/messytables.git#egg=messytables checkout from git+https://github.com/okfn/messytables.git#egg=messytables (from -r requires.txt (line 1))
  Updating clone ./src/messytables
  Running setup.py egg_info for package messytables
Checking out ckanext-archiver from git+https://github.com/okfn/ckanext-archiver.git#egg=ckanext-archiver checkout from git+https://github.com/okfn/ckanext-archiver.git#egg=ckanext-archiver (from -r requires.txt (line 2))
  Updating clone ./src/ckanext-archiver
  Running setup.py egg_info for package ckanext-archiver
Downloading/unpacking xlrd==0.7.1 (from messytables->-r requires.txt (line 1))
  Running setup.py egg_info for package xlrd
Downloading/unpacking openpyxl==1.5.7 (from messytables->-r requires.txt (line 1))
  Running setup.py egg_info for package openpyxl
    warning: no files found matching '*.xml' under directory 'src/openpyxl/tests/test_data'
    warning: no files found matching '*.rels' under directory 'src/openpyxl/tests/test_data'
    warning: no files found matching '*.xlsx' under directory 'src/openpyxl/tests/test_data'
    warning: no files found matching 'CREDITS'
Downloading/unpacking celery>=2.3.3 (from ckanext-archiver->-r requires.txt (line 2))
  Running setup.py egg_info for package celery
    no previously-included directories found matching 'tests/*.pyc'
    no previously-included directories found matching 'docs/*.pyc'
    no previously-included directories found matching 'contrib/*.pyc'
    no previously-included directories found matching 'celery/*.pyc'
    no previously-included directories found matching 'examples/*.pyc'
    no previously-included directories found matching 'bin/*.pyc'
    no previously-included directories found matching 'docs/.build'
    no previously-included directories found matching 'docs/graffles'
    no previously-included directories found matching '.tox/*'
Downloading/unpacking kombu-sqlalchemy>=1.1.0 (from ckanext-archiver->-r requires.txt (line 2))
  Running setup.py egg_info for package kombu-sqlalchemy
    warning: no files found matching '*' under directory 'djkombu'
Downloading/unpacking SQLAlchemy>=0.6.6 (from ckanext-archiver->-r requires.txt (line 2))
  Downloading SQLAlchemy-0.7.7.tar.gz (2.6Mb): 2.6Mb downloaded
  Running setup.py egg_info for package SQLAlchemy
    warning: no files found matching '*.jpg' under directory 'doc'
    no previously-included directories found matching 'doc/build/output'
Downloading/unpacking requests==0.6.4 (from ckanext-archiver->-r requires.txt (line 2))
  Could not fetch URL http://pypi.python.org/simple/requests/0.6.4: HTTP Error 404: Not Found (requests/0.6.4)
  Will skip URL http://pypi.python.org/simple/requests/0.6.4 when looking for download links for requests==0.6.4 (from ckanext-archiver->-r requires.txt (line 2))
  Downloading requests-0.6.4.tar.gz
  Running setup.py egg_info for package requests
Downloading/unpacking flask==0.8 (from ckanext-archiver->-r requires.txt (line 2))
  Could not fetch URL http://pypi.python.org/simple/flask/0.8: HTTP Error 404: Not Found (flask/0.8)
  Will skip URL http://pypi.python.org/simple/flask/0.8 when looking for download links for flask==0.8 (from ckanext-archiver->-r requires.txt (line 2))
  Downloading Flask-0.8.tar.gz (494Kb): 494Kb downloaded
  Running setup.py egg_info for package flask
    warning: no files found matching '*' under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'examples'
    warning: no previously-included files matching '*.pyo' found under directory 'examples'
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'docs/_themes/.git'
Downloading/unpacking importlib (from celery>=2.3.3->ckanext-archiver->-r requires.txt (line 2))
  Downloading importlib-1.0.2.zip
  Running setup.py egg_info for package importlib
Downloading/unpacking anyjson>=0.3.1 (from celery>=2.3.3->ckanext-archiver->-r requires.txt (line 2))
  Downloading anyjson-0.3.1.tar.gz
  Running setup.py egg_info for package anyjson
Downloading/unpacking kombu>=2.1.5,<3.0 (from celery>=2.3.3->ckanext-archiver->-r requires.txt (line 2))
  Downloading kombu-2.1.8.tar.gz (273Kb): 273Kb downloaded
  Running setup.py egg_info for package kombu
Downloading/unpacking python-dateutil>=1.5,<2.0 (from celery>=2.3.3->ckanext-archiver->-r requires.txt (line 2))
  Downloading python-dateutil-1.5.tar.gz (232Kb): 232Kb downloaded
  Running setup.py egg_info for package python-dateutil
Downloading/unpacking ordereddict (from celery>=2.3.3->ckanext-archiver->-r requires.txt (line 2))
  Downloading ordereddict-1.1.tar.gz
  Running setup.py egg_info for package ordereddict
Downloading/unpacking Werkzeug>=0.6.1 (from flask==0.8->ckanext-archiver->-r requires.txt (line 2))
  Could not fetch URL http://trac.pocoo.org/repos/werkzeug/trunk (from http://pypi.python.org/simple/Werkzeug/): HTTP Error 404: Not Found
  Will skip URL http://trac.pocoo.org/repos/werkzeug/trunk when looking for download links for Werkzeug>=0.6.1 (from flask==0.8->ckanext-archiver->-r requires.txt (line 2))
  Downloading Werkzeug-0.8.3.tar.gz (1.1Mb): 1.1Mb downloaded
  Running setup.py egg_info for package Werkzeug
    warning: no files found matching '*' under directory 'werkzeug/debug/templates'
    warning: no files found matching '*' under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'examples'
    warning: no previously-included files matching '*.pyo' found under directory 'examples'
    no previously-included directories found matching 'docs/_build'
Downloading/unpacking Jinja2>=2.4 (from flask==0.8->ckanext-archiver->-r requires.txt (line 2))
  Downloading Jinja2-2.6.tar.gz (389Kb): 389Kb downloaded
  Running setup.py egg_info for package Jinja2
    warning: no previously-included files matching '*' found under directory 'docs/_build'
    warning: no previously-included files matching '*.pyc' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
Downloading/unpacking amqplib>=1.0 (from kombu>=2.1.5,<3.0->celery>=2.3.3->ckanext-archiver->-r requires.txt (line 2))
  Downloading amqplib-1.0.2.tgz (58Kb): 58Kb downloaded
  Running setup.py egg_info for package amqplib
Installing collected packages: amqplib, anyjson, celery, ckanext-archiver, flask, importlib, Jinja2, kombu, kombu-sqlalchemy, messytables, openpyxl, ordereddict, python-dateutil, requests, SQLAlchemy, Werkzeug, xlrd
  Running setup.py install for SQLAlchemy
    building 'sqlalchemy.cprocessors' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c lib/sqlalchemy/cextension/processors.c -o build/temp.linux-i686-2.6/lib/sqlalchemy/cextension/processors.o
    unable to execute gcc: No such file or directory
    ***************************************************************************
    command 'gcc' failed with exit status 1
    WARNING: The C extension could not be compiled, speedups are not enabled.
    Failure information, if any, is above.
    Retrying the build without the C extension now.
    ***************************************************************************
    warning: no files found matching '*.jpg' under directory 'doc'
    no previously-included directories found matching 'doc/build/output'
    ***************************************************************************
    WARNING: The C extension could not be compiled, speedups are not enabled.
    Plain-Python build succeeded.
    ***************************************************************************
  Running setup.py install for ordereddict
  Running setup.py install for openpyxl
    warning: no files found matching '*.xml' under directory 'src/openpyxl/tests/test_data'
    warning: no files found matching '*.rels' under directory 'src/openpyxl/tests/test_data'
    warning: no files found matching '*.xlsx' under directory 'src/openpyxl/tests/test_data'
    warning: no files found matching 'CREDITS'
  Running setup.py install for kombu-sqlalchemy
    warning: no files found matching '*' under directory 'djkombu'
  Running setup.py install for amqplib
  Running setup.py develop for ckanext-archiver
    Creating /usr/local/lib/python2.6/dist-packages/ckanext-archiver.egg-link (link to .)
    Adding ckanext-archiver 0.1 to easy-install.pth file

    Installed /home/shaq/git/ckanext-datastorer/src/ckanext-archiver
  Running setup.py install for celery
    no previously-included directories found matching 'tests/*.pyc'
    no previously-included directories found matching 'docs/*.pyc'
    no previously-included directories found matching 'contrib/*.pyc'
    no previously-included directories found matching 'celery/*.pyc'
    no previously-included directories found matching 'examples/*.pyc'
    no previously-included directories found matching 'bin/*.pyc'
    no previously-included directories found matching 'docs/.build'
    no previously-included directories found matching 'docs/graffles'
    no previously-included directories found matching '.tox/*'
    Installing celeryctl script to /usr/local/bin
    Installing celeryd script to /usr/local/bin
    Installing camqadm script to /usr/local/bin
    Installing celeryev script to /usr/local/bin
    Installing celeryd-multi script to /usr/local/bin
    Installing celerybeat script to /usr/local/bin
  Running setup.py develop for messytables
    Creating /usr/local/lib/python2.6/dist-packages/messytables.egg-link (link to .)
    Adding messytables 0.2.0 to easy-install.pth file

    Installed /home/shaq/git/ckanext-datastorer/src/messytables
  Running setup.py install for Jinja2
    warning: no previously-included files matching '*' found under directory 'docs/_build'
    warning: no previously-included files matching '*.pyc' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
  Running setup.py install for kombu
  Running setup.py install for Werkzeug
    warning: no files found matching '*' under directory 'werkzeug/debug/templates'
    warning: no files found matching '*' under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'examples'
    warning: no previously-included files matching '*.pyo' found under directory 'examples'
    no previously-included directories found matching 'docs/_build'
  Running setup.py install for flask
    warning: no files found matching '*' under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'examples'
    warning: no previously-included files matching '*.pyo' found under directory 'examples'
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'docs/_themes/.git'
  Running setup.py install for python-dateutil
  Running setup.py install for xlrd
    changing mode of build/scripts-2.6/runxlrd.py from 644 to 755
    changing mode of /usr/local/bin/runxlrd.py to 755
  Running setup.py install for anyjson
  Running setup.py install for requests
  Running setup.py install for importlib
Successfully installed



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20120510/7fc17a20/attachment-0001.html>

• Previous message: [ckan-dev] Configuring the datastore
• Next message: [ckan-dev] Prelim SPARQL endpoint - request for test
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]