[ckan-changes] commit/ckan: 2 new changesets

Thu May 26 20:38:37 UTC 2011

2 new changesets in ckan:

http://bitbucket.org/okfn/ckan/changeset/d26ce0b6762a/
changeset:   r3116:d26ce0b6762a
user:        dread
date:        2011-05-26 22:36:59
summary:     [requires]: #1168 Moved ckanclient requirement to dgu. Ckan only seems to use it in deprecated model/changeset and bin scripts.
affected #:  1 file (98 bytes)

--- a/requires/lucid_missing.txt	Wed May 25 21:01:51 2011 +0100
+++ b/requires/lucid_missing.txt	Thu May 26 21:36:59 2011 +0100
@@ -7,8 +7,6 @@
 -e hg+https://bitbucket.org/okfn/licenses@0eed4a13296b#egg=licenses
 # vdm>=0.9,<0.9.99
 -e hg+https://bitbucket.org/okfn/vdm@vdm-0.9#egg=vdm
-# ckanclient>=0.1,<0.7.99
--e hg+https://bitbucket.org/okfn/ckanclient@6756586299cc#egg=ckanclient
 # markupsafe==0.9.2 required by webhelpers==1.2 required by formalchemy with SQLAlchemy 0.6
 -e git+https://github.com/mitsuhiko/markupsafe.git@0.9.2#egg=markupsafe
 # autoneg>=0.5


http://bitbucket.org/okfn/ckan/changeset/64cd2055c058/
changeset:   r3117:64cd2055c058
branch:      feature-1168-deb-build
user:        dread
date:        2011-05-26 22:38:13
summary:     [doc]: Keeping track in docs.
affected #:  2 files (5.1 KB)

--- a/doc/buildbot.rst	Thu May 26 21:36:59 2011 +0100
+++ b/doc/buildbot.rst	Thu May 26 21:38:13 2011 +0100
@@ -121,11 +121,44 @@
 
 It's preferable to view the buildbot site at port 80 rather than 8010.
 
-If there is no other web service on this machine, you might connect up the addresses using iptables::
+The preferred way is to use an nginx reverse proxy, since nginx is used for the apt server later on.
 
-  sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8010
+Using nginx
+-----------
 
-Otherwise it is best to do a reverse proxy. Using apache, edit this file::
+Install nginx::
+
+  sudo apt-get install nginx
+
+Edit the vhost ``/etc/nginx/sites-available/vhost-buildbot.conf``::
+
+  server {
+    listen 80;
+    server_name buildbot.okfn.org;
+  
+    access_log /var/log/nginx/buildbot-error.log;
+    error_log /var/log/nginx/buildbot-error.log;
+  
+    location / {
+      proxy_pass         http://127.0.0.1:8010/;
+    }
+  }
+
+Enable it::
+
+  ln -s /etc/nginx/sites-available/vhost-buildbot.conf /etc/nginx/sites-enabled/vhost-buildbot.conf
+
+Restart nginx::
+
+  sudo /etc/init.d/nginx restart
+
+
+Using apache
+------------
+
+Otherwise it is best to do a reverse proxy. 
+
+For apache, edit this file::
 
   sudo vim /etc/apache2/sites-available/buildbot.okfn.org
 
@@ -151,3 +184,183 @@
   sudo a2ensite buildbot.okfn.org
   sudo /etc/init.d/apache2 reload
 
+
+Using iptables
+--------------
+
+If there is no other web service on this machine, you might connect up the addresses using iptables::
+
+  sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8010
+
+Virtual Machine 
+===============
+
+Set-up a virtual machine in the /home/buildslave/Vms directory, as per :doc:`vm.rst`.
+
+It is useful to have the following scripts in the same directory too. Note buildslave user needs to be given passwordless sudo access for these to work.
+
+start-kvm.sh
+------------
+
+::
+
+  pidfile="/tmp/vm.pid"
+  
+  if pidof kvm | grep [0-9] > /dev/null
+  then 
+          echo ERROR: KVM process is already running
+          exit 1
+  fi
+  
+  /home/buildslave/Vms/kvm.sh $* -pidfile $pidfile &
+  
+  # check if the pid file created successfully
+  if [ ! -f ${pidfile} ]
+  then
+      sleep 5
+  fi
+  if [ ! -f ${pidfile} ]
+  then
+      echo ERROR: PID file not created
+      exit 1
+  fi
+  
+  # check if the process started successfully
+  if [ ! -d /proc/`sudo cat ${pidfile}` ]
+  then
+      echo ERROR: Process did not start properly
+      exit 1
+  fi
+
+start.sh
+--------
+
+::
+
+  #!/bin/bash
+  
+  if [ "X$1" = "X" ] || [ "X$2" = "X" ]  || [ "X$3" = "X" ] || [ "X$4" = "X" ]  || [ "X$5" = "X" ]; then
+      echo "ERROR: call this script with network device name, tunnel name, amount of memory, number of CPUs and path to the image e.g." 
+      echo "       $0 eth0 qtap0 512M 4 /home/Vms/ckan_2/tmpKfAdeU.qcow2 [extra args to KVM]"
+      exit 1
+  fi
+  
+  NETWORK_DEVICE=$1
+  TUNNEL=$2
+  MEM=$3
+  CPUS=$4
+  IMAGE=$5
+  EXTRA=$6
+  MACADDR="52:54:$(dd if=/dev/urandom count=1 2>/dev/null | md5sum | sed 's/^\(..\)\(..\)\(..\)\(..\).*$/\1:\2:\3:\4/')";
+  
+  echo "Creating bridge..."
+  sudo iptables -t nat -A POSTROUTING -o ${NETWORK_DEVICE} -j MASQUERADE
+  sudo brctl addbr br0
+  sudo ifconfig br0 192.168.100.254 netmask 255.255.255.0 up
+  echo "done."
+  echo "Creating tunnel..."
+  sudo modprobe tun
+  sudo tunctl -b -u root -t ${TUNNEL}
+  sudo brctl addif br0 ${TUNNEL}
+  sudo ifconfig ${TUNNEL} up 0.0.0.0 promisc
+  echo "done."
+  echo "Starting VM ${IMAGE} on ${TUNNEL} via ${NETWORK_DEVICE} with MAC ${MACADDR}..."
+  sudo /usr/bin/kvm -M pc-0.12 -enable-kvm -m ${MEM} -smp ${CPUS} -name dev -monitor pty -boot c -drive file=${IMAGE},if=ide,index=0,boot=on -net nic,macaddr=${MACADDR} -net tap,ifname=${TUNNEL},script=no,downscript=no -serial none -parallel none -usb ${EXTRA}
+
+stop-kvm.sh
+-----------
+
+::
+
+  #! /bin/bash
+  
+  pidfile="/tmp/vm.pid"
+  
+  pid2kill=`sudo cat $pidfile`
+  if ! pidof kvm | grep [0-9] > /dev/null
+  then   
+          echo KVM process is not running
+  else
+          echo Killing process $pid2kill
+          sudo kill -9 $pid2kill
+  fi
+  
+  if [ ! -e "$pidfile" ]
+  then   
+          echo "No process file: $pidfile"
+  else
+          sudo rm $pidfile
+  fi
+
+
+apt server for tests
+====================
+
+The buildbot server is setup as an apt server, so that the process of installing a deb package can be tested.
+
+The basic way to do this is documented here: http://joseph.ruscio.org/blog/2010/08/19/setting-up-an-apt-repository/ but we vary this. For example, for tests we don't sign the packages with a key (it is difficult to type the passphrase from a script).
+
+::
+
+  sudo apt-get install reprepro
+  sudo mkdir -p /var/packages
+  sudo chown okfn:okfn /var/packages
+  mkdir -p /var/packages/ubuntu_ckan-test/conf
+  touch /var/packages/ubuntu_ckan-test/conf/override.lucid
+
+Create the configuration ``/var/packages/ubuntu_ckan-test/conf/distributions`` as::
+
+  Origin: Open Knowledge Foundation
+  Label: Open Knowledge Foundation
+  Codename: lucid
+  Architectures: amd64
+  Components: universe
+  Description: CKAN APT Repository
+  DebOverride: override.lucid
+  DscOverride: override.lucid
+
+Create the reprepro options file ``/var/packages/ubuntu_ckan-test/options`` as::
+
+  verbose
+  basedir .
+
+Get permissions right::
+
+  sudo chown -R buildslave:buildslave /var/packages/ubuntu_ckan-test
+
+Install nginx::
+  
+  sudo apt-get install nginx
+
+Here we configure the name of the server as apt.okfn.org by creating /etc/nginx/sites-available/vhost-packages.conf::
+
+  server {
+    listen 80;
+    server_name dgu-buildbot.okfn.org;
+  
+    access_log /var/log/nginx/packages-error.log;
+    error_log /var/log/nginx/packages-error.log;
+  
+    location / {
+      root /var/packages;
+      index index.html;
+    }
+  
+    location ~ /(.*)/conf {
+      deny all;
+    }
+
+    location ~ /(.*)/db {
+      deny all;
+    }
+  }
+
+Configure the hash bucket size by creating the file /etc/nginx/conf.d/server_names_hash_bucket_size.conf::
+
+  server_names_hash_bucket_size 64;
+
+Enable the APT server::
+
+  cd /etc/nginx/sites-enabled
+  sudo ln -s ../sites-available/vhosts-packages.conf .
+  sudo /etc/init.d/nginx restart


--- a/doc/vm.rst	Thu May 26 21:36:59 2011 +0100
+++ b/doc/vm.rst	Thu May 26 21:38:13 2011 +0100
@@ -212,7 +212,7 @@
 
 Now that you have the repo added you can install and test CKAN as normal.
 
-Here's how mine look:
+Here's how mine looks:
 
 ::

Repository URL: https://bitbucket.org/okfn/ckan/

--

This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.


