[ckan-changes] commit/ckan: 2 new changesets
Bitbucket
commits-noreply at bitbucket.org
Thu May 26 20:38:37 UTC 2011
2 new changesets in ckan:
http://bitbucket.org/okfn/ckan/changeset/d26ce0b6762a/
changeset: r3116:d26ce0b6762a
user: dread
date: 2011-05-26 22:36:59
summary: [requires]: #1168 Moved ckanclient requirement to dgu. Ckan only seems to use it in deprecated model/changeset and bin scripts.
affected #: 1 file (98 bytes)
--- a/requires/lucid_missing.txt Wed May 25 21:01:51 2011 +0100
+++ b/requires/lucid_missing.txt Thu May 26 21:36:59 2011 +0100
@@ -7,8 +7,6 @@
-e hg+https://bitbucket.org/okfn/licenses@0eed4a13296b#egg=licenses
# vdm>=0.9,<0.9.99
-e hg+https://bitbucket.org/okfn/vdm@vdm-0.9#egg=vdm
-# ckanclient>=0.1,<0.7.99
--e hg+https://bitbucket.org/okfn/ckanclient@6756586299cc#egg=ckanclient
# markupsafe==0.9.2 required by webhelpers==1.2 required by formalchemy with SQLAlchemy 0.6
-e git+https://github.com/mitsuhiko/markupsafe.git@0.9.2#egg=markupsafe
# autoneg>=0.5
http://bitbucket.org/okfn/ckan/changeset/64cd2055c058/
changeset: r3117:64cd2055c058
branch: feature-1168-deb-build
user: dread
date: 2011-05-26 22:38:13
summary: [doc]: Keeping track in docs.
affected #: 2 files (5.1 KB)
--- a/doc/buildbot.rst Thu May 26 21:36:59 2011 +0100
+++ b/doc/buildbot.rst Thu May 26 21:38:13 2011 +0100
@@ -121,11 +121,44 @@
It's preferable to view the buildbot site at port 80 rather than 8010.
-If there is no other web service on this machine, you might connect up the addresses using iptables::
+The preferred way is to use an nginx reverse proxy, since nginx is used for the apt server later on.
- sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8010
+Using nginx
+-----------
-Otherwise it is best to do a reverse proxy. Using apache, edit this file::
+Install nginx::
+
+ sudo apt-get install nginx
+
+Edit the vhost ``/etc/nginx/sites-available/vhost-buildbot.conf``::
+
+ server {
+ listen 80;
+ server_name buildbot.okfn.org;
+
+ access_log /var/log/nginx/buildbot-error.log;
+ error_log /var/log/nginx/buildbot-error.log;
+
+ location / {
+ proxy_pass http://127.0.0.1:8010/;
+ }
+ }
+
+Enable it::
+
+ ln -s /etc/nginx/sites-available/vhost-buildbot.conf /etc/nginx/sites-enabled/vhost-buildbot.conf
+
+Restart nginx::
+
+ sudo /etc/init.d/nginx restart
+
+
+Using apache
+------------
+
+Otherwise it is best to do a reverse proxy.
+
+For apache, edit this file::
sudo vim /etc/apache2/sites-available/buildbot.okfn.org
@@ -151,3 +184,183 @@
sudo a2ensite buildbot.okfn.org
sudo /etc/init.d/apache2 reload
+
+Using iptables
+--------------
+
+If there is no other web service on this machine, you might connect up the addresses using iptables::
+
+ sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8010
+
+Virtual Machine
+===============
+
+Set-up a virtual machine in the /home/buildslave/Vms directory, as per :doc:`vm.rst`.
+
+It is useful to have the following scripts in the same directory too. Note buildslave user needs to be given passwordless sudo access for these to work.
+
+start-kvm.sh
+------------
+
+::
+
+ pidfile="/tmp/vm.pid"
+
+ if pidof kvm | grep [0-9] > /dev/null
+ then
+ echo ERROR: KVM process is already running
+ exit 1
+ fi
+
+ /home/buildslave/Vms/kvm.sh $* -pidfile $pidfile &
+
+ # check if the pid file created successfully
+ if [ ! -f ${pidfile} ]
+ then
+ sleep 5
+ fi
+ if [ ! -f ${pidfile} ]
+ then
+ echo ERROR: PID file not created
+ exit 1
+ fi
+
+ # check if the process started successfully
+ if [ ! -d /proc/`sudo cat ${pidfile}` ]
+ then
+ echo ERROR: Process did not start properly
+ exit 1
+ fi
+
+start.sh
+--------
+
+::
+
+ #!/bin/bash
+
+ if [ "X$1" = "X" ] || [ "X$2" = "X" ] || [ "X$3" = "X" ] || [ "X$4" = "X" ] || [ "X$5" = "X" ]; then
+ echo "ERROR: call this script with network device name, tunnel name, amount of memory, number of CPUs and path to the image e.g."
+ echo " $0 eth0 qtap0 512M 4 /home/Vms/ckan_2/tmpKfAdeU.qcow2 [extra args to KVM]"
+ exit 1
+ fi
+
+ NETWORK_DEVICE=$1
+ TUNNEL=$2
+ MEM=$3
+ CPUS=$4
+ IMAGE=$5
+ EXTRA=$6
+ MACADDR="52:54:$(dd if=/dev/urandom count=1 2>/dev/null | md5sum | sed 's/^\(..\)\(..\)\(..\)\(..\).*$/\1:\2:\3:\4/')";
+
+ echo "Creating bridge..."
+ sudo iptables -t nat -A POSTROUTING -o ${NETWORK_DEVICE} -j MASQUERADE
+ sudo brctl addbr br0
+ sudo ifconfig br0 192.168.100.254 netmask 255.255.255.0 up
+ echo "done."
+ echo "Creating tunnel..."
+ sudo modprobe tun
+ sudo tunctl -b -u root -t ${TUNNEL}
+ sudo brctl addif br0 ${TUNNEL}
+ sudo ifconfig ${TUNNEL} up 0.0.0.0 promisc
+ echo "done."
+ echo "Starting VM ${IMAGE} on ${TUNNEL} via ${NETWORK_DEVICE} with MAC ${MACADDR}..."
+ sudo /usr/bin/kvm -M pc-0.12 -enable-kvm -m ${MEM} -smp ${CPUS} -name dev -monitor pty -boot c -drive file=${IMAGE},if=ide,index=0,boot=on -net nic,macaddr=${MACADDR} -net tap,ifname=${TUNNEL},script=no,downscript=no -serial none -parallel none -usb ${EXTRA}
+
+stop-kvm.sh
+-----------
+
+::
+
+ #! /bin/bash
+
+ pidfile="/tmp/vm.pid"
+
+ pid2kill=`sudo cat $pidfile`
+ if ! pidof kvm | grep [0-9] > /dev/null
+ then
+ echo KVM process is not running
+ else
+ echo Killing process $pid2kill
+ sudo kill -9 $pid2kill
+ fi
+
+ if [ ! -e "$pidfile" ]
+ then
+ echo "No process file: $pidfile"
+ else
+ sudo rm $pidfile
+ fi
+
+
+apt server for tests
+====================
+
+The buildbot server is setup as an apt server, so that the process of installing a deb package can be tested.
+
+The basic way to do this is documented here: http://joseph.ruscio.org/blog/2010/08/19/setting-up-an-apt-repository/ but we vary this. For example, for tests we don't sign the packages with a key (it is difficult to type the passphrase from a script).
+
+::
+
+ sudo apt-get install reprepro
+ sudo mkdir -p /var/packages
+ sudo chown okfn:okfn /var/packages
+ mkdir -p /var/packages/ubuntu_ckan-test/conf
+ touch /var/packages/ubuntu_ckan-test/conf/override.lucid
+
+Create the configuration ``/var/packages/ubuntu_ckan-test/conf/distributions`` as::
+
+ Origin: Open Knowledge Foundation
+ Label: Open Knowledge Foundation
+ Codename: lucid
+ Architectures: amd64
+ Components: universe
+ Description: CKAN APT Repository
+ DebOverride: override.lucid
+ DscOverride: override.lucid
+
+Create the reprepro options file ``/var/packages/ubuntu_ckan-test/options`` as::
+
+ verbose
+ basedir .
+
+Get permissions right::
+
+ sudo chown -R buildslave:buildslave /var/packages/ubuntu_ckan-test
+
+Install nginx::
+
+ sudo apt-get install nginx
+
+Here we configure the name of the server as apt.okfn.org by creating /etc/nginx/sites-available/vhost-packages.conf::
+
+ server {
+ listen 80;
+ server_name dgu-buildbot.okfn.org;
+
+ access_log /var/log/nginx/packages-error.log;
+ error_log /var/log/nginx/packages-error.log;
+
+ location / {
+ root /var/packages;
+ index index.html;
+ }
+
+ location ~ /(.*)/conf {
+ deny all;
+ }
+
+ location ~ /(.*)/db {
+ deny all;
+ }
+ }
+
+Configure the hash bucket size by creating the file /etc/nginx/conf.d/server_names_hash_bucket_size.conf::
+
+ server_names_hash_bucket_size 64;
+
+Enable the APT server::
+
+ cd /etc/nginx/sites-enabled
+ sudo ln -s ../sites-available/vhosts-packages.conf .
+ sudo /etc/init.d/nginx restart
--- a/doc/vm.rst Thu May 26 21:36:59 2011 +0100
+++ b/doc/vm.rst Thu May 26 21:38:13 2011 +0100
@@ -212,7 +212,7 @@
Now that you have the repo added you can install and test CKAN as normal.
-Here's how mine look:
+Here's how mine looks:
::
Repository URL: https://bitbucket.org/okfn/ckan/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the ckan-changes
mailing list