[ckan-dev] Issues with Deploying Source Install on Vagrant VM

[Sean Baskin]

I've been following the Installing CKAN from
Source<http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html>
and
the Deploying a Source
Install<http://docs.ckan.org/en/latest/maintaining/installing/deployment.html>
guides
to construct an Ansible playbook to provision a Vagrant virtual machine.
Ideally once I have nailed down this process I will add the production
server to the Ansible hosts file. Everything seems to work fine except for
the actual site being served by Apache via WSGI.

A few general notes:
[1] I can SSH into the VM and run 'paster serve' to serve the site on port
5000. I can also accomplish this via a task in my Ansible playbook with the
same results. Visiting my domain on this port is successful; I can see the
CKAN site, e.g. http://opendata.chattlibrary.dev:5000

[2] I have checked that Apache Solr is correctly configured and running.
Visiting http://opendata.chattlibrary.dev:8983/solr lands me on the Solr
page.

[3] When I visit the root domain, i.e. http://opendata.chattlibrary.dev, I
am directed to the Apache default site page OR the Nginx default site page.
Upon disabling the default Apache site and reloading Apache + Nginx, I
visit the root domain and am presented with a HTTP 404 response stating
"The requested URL / was not found on this server".

>From my understanding, the Apache config file is redirecting HTTP requests
to my WSGI application (CKAN). It would appear that CKAN is not actually
running. I understand this problem has more to do with Apache + Nginx +
WSGI and less to do with Vagrant + Ansible, but I'm including the relevant
portions in this email. I was under the impression that serving the site
with paster is not best practice, so there is something here I'm missing.
Any help would be greatly appreciated!

---

[/etc/ckan/default/apache.wsgi]

import os
activate_this = os.path.join('/usr/lib/ckan/default/bin/activate_this.py')
execfile(activate_this, dict(__file__=activate_this))

from paste.deploy import loadapp
config_filepath = os.path.join(os.path.dirname(os.path.abspath(__file__)),
'production.ini')
from paste.script.util.logging_config import fileConfig
fileConfig(config_filepath)
application = loadapp('config:%s' % config_filepath)

---

[/etc/apache2/sites-available/ckan_default]

<VirtualHost 0.0.0.0:8080>
    ServerName opendata.chattlibrary.dev
    ServerAlias www.opendata.chattlibrary.dev
    WSGIScriptAlias / /etc/ckan/default/apache.wsgi

    # Pass authorization info on (needed for rest api).
    WSGIPassAuthorization On

    # Deploy as a daemon (avoids conflicts between CKAN instances).
    WSGIDaemonProcess ckan_default display-name=ckan_default processes=2
threads=15

    WSGIProcessGroup ckan_default

    ErrorLog /var/log/apache2/ckan_default.error.log
    CustomLog /var/log/apache2/ckan_default.custom.log combined
</VirtualHost>

---

[/etc/nginx/sites-available/ckan_default]

proxy_cache_path /tmp/nginx_cache levels=1:2 keys_zone=cache:30m
max_size=250m;
proxy_temp_path /tmp/nginx_proxy 1 2;

server {
    client_max_body_size 100M;
    location / {
        proxy_pass http://127.0.0.1:8080/;
        proxy_set_header Host $host;
        proxy_cache cache;
        proxy_cache_bypass $cookie_auth_tkt;
        proxy_no_cache $cookie_auth_tkt;
        proxy_cache_valid 30m;
        proxy_cache_key $host$scheme$proxy_host$request_uri;
        # In emergency comment out line to force caching
        # proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
    }

}

---

[site.yml] This is my main playbook. It's ugly, but it gets the job done
for the moment.

---
- hosts: all
  user: vagrant
  sudo: true

  tasks:

  - name: General | Copy SSH Keypair
    copy: src=/home/sean/.ssh/{{ item }} dest=/home/vagrant/.ssh/{{ item
 }} owner=vagrant
    with_items:
    - id_rsa_vagrant
    - id_rsa_vagrant.pub

  - name: General | Install Required Packages
    apt: pkg={{ item }} state=present update_cache=yes
    with_items:
    - apache2
    - git-core
    - libapache2-mod-wsgi
    - libpq-dev
    - nginx
    - openjdk-6-jdk
    - postgresql
    - python-dev
    - python-pip
    - python-psycopg2
    - python-virtualenv
    - solr-jetty

  - name: CKAN | Install CKAN source to virtualenv
    pip: name='git+https://github.com/ckan/ckan.git@ckan-2.2#egg=ckan'
virtualenv=/usr/lib/ckan/default virtualenv_site_packages=no

  - name: CKAN | Install Python Requiements
    pip: requirements=/usr/lib/ckan/default/src/ckan/requirements.txt
virtualenv=/usr/lib/ckan/default virtualenv_site_packages=no

  - name: PostgreSQL | Start DB Cluster
    command: /etc/init.d/postgresql start

  - name: CKAN | Create DB
    sudo_user: postgres
    postgresql_db: name=ckan_default
                   encoding='UTF-8'
                   lc_collate='en_US.UTF-8'
                   lc_ctype='en_US.UTF-8'
                   template='template0'
                   login_password=pass

  - name: CKAN | Create Database User
    sudo_user: postgres
    postgresql_user: db=ckan_default user=ckan_default password=pass

  - name: CKAN | Create site directory
    file: path=/etc/ckan/default state=directory owner=vagrant

  - name: CKAN | Create CKAN config file
    command: /usr/lib/ckan/default/bin/paster make-config ckan
/etc/ckan/default/development.ini

  - name: CKAN | Copy Jetty configuration to remote host
    copy:
src=/home/sean/projects/github/open-data-portal/provisioning/roles/common/files/jetty
dest=/etc/default/jetty

  - name: CKAN | Start Jetty Server
    service: name=jetty state=started

  - name: CKAN | Backup Solr schema
    command: mv /etc/solr/conf/schema.xml /etc/solr/conf/schema.xml.bak

  # Update to use file module
  - name: CKAN | Replace Solr schema.xml with symlink to CKAN schema file
    command: ln -s
/usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml
/etc/solr/conf/schema.xml

  - name: CKAN | Restart Jetty Server
    service: name=jetty state=restarted

  - name: CKAN | Initialize CKAN database
    command: chdir=/etc/ckan/default /usr/lib/ckan/default/bin/paster
--plugin=ckan db init

  # Update to use file module
  - name: CKAN | Link to who.ini
    command: ln -s /usr/lib/ckan/default/src/ckan/who.ini
/etc/ckan/default/who.ini

  - name: CKAN | Create production.ini
    command: cp /etc/ckan/default/development.ini
/etc/ckan/default/production.ini

  - name: CKAN | Copy Apache WSGI file
    copy:
src=/home/sean/projects/github/open-data-portal/provisioning/roles/common/files/apache.wsgi
dest=/etc/ckan/default/apache.wsgi

  - name: CKAN | Copy Apache configuration file
    copy:
src=/home/sean/projects/github/open-data-portal/provisioning/roles/common/files/ckan_default
dest=/etc/apache2/sites-available/ckan_default

  - name: CKAN | Copy Nginx configuration file
    copy:
src=/home/sean/projects/github/open-data-portal/provisioning/roles/common/files/ckan_default_nginx
dest=/etc/nginx/sites-available/ckan_default

  - name: CKAN | Enable CKAN site
    command: a2ensite ckan_default

  - name: CKAN | Enable CKAN
    command: ln -s /etc/nginx/sites-available/ckan_default
/etc/nginx/sites-enabled/ckan_default

  - name: CKAN | Reload Apache2
    service: name=apache2 state=reloaded

  - name: CKAN | Reload Nginx
    service: name=nginx state=reloaded

  # This is a long running process and only for development purposes
  # - name: CKAN | Serve CKAN
  #   command: /usr/lib/ckan/default/bin/paster serve
/etc/ckan/default/production.ini
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140312/f7af3ea8/attachment-0002.html>