[ckan-dev] CKAN 1.5 .deb Packaging Update

p.romain at cg33.fr p.romain at cg33.fr
Tue Nov 15 15:48:50 UTC 2011


Hi James,
That' right  I am using the latest distribution 11.10. I will give it a 
new try with a different virtual box.

Regards,

Cordialement,
Pascal Romain
Chef de projet informatique documentaire
Service Projets Etudes Conseils
Direction des Systèmes d'Information
05 56 99 33 33 poste 6643
@datalocale




De :    James Gardner <james at 3aims.com>
A :     ckan-dev at lists.okfn.org
Date :  15/11/2011 12:45
Objet : Re: [ckan-dev] CKAN 1.5 .deb Packaging Update
Envoyé par :    ckan-dev-bounces at lists.okfn.org



Hi Pascal,

Great that you gave it a go but I'm afraid the packaging approach is 
*only* for Ubuntu 10.04. I can see that your install has Python 2.7 so it 
must be a different version of Ubuntu. Which version are you using?

Cheers,

James




On 14/11/11 20:54, p.romain at cg33.fr wrote: 
Hi,

I tried to follow the installation process on a ubuntu laptop and on a 
ubuntu virtual box and I was not able to deploy a ckan instance in both 
cases

I have the following error message after running the following command
sudo ckan-create-instance test test.ckan.net yes
...
Making sure PostgreSQL is running ...
 * Starting PostgreSQL 8.4 database server  [ OK ] 
 * Starting PostgreSQL 9.1 database server  [ OK ] 
Setting the password of the test user in PostgreSQL
Creating the test user ...
CREATE ROLE
Creating/overwriting the config for CKAN ... 
Distribution already installed:
  ckan 1.5 from /usr/lib/pymodules/python2.7
Creating /etc/ckan/test/test.ini
Now you should edit the config files
  /etc/ckan/test/test.ini
Ensuring the test database exists ...
Overwriting the existing Apache config ...
Performing any database upgrades ...

Traceback (most recent call last):
  File "/usr/bin/paster", line 18, in <module>
    command.run()
  File "/usr/lib/python2.7/dist-packages/paste/script/command.py", line 
84, in run
    invoke(command, command_name, options, args[1:])
  File "/usr/lib/python2.7/dist-packages/paste/script/command.py", line 
123, in invoke
    exit_code = runner.run(args)
  File "/usr/lib/python2.7/dist-packages/paste/script/command.py", line 
218, in run
    result = self.command()
  File "/usr/lib/pymodules/python2.7/ckan/lib/cli.py", line 82, in command
    self._load_config()
  File "/usr/lib/pymodules/python2.7/ckan/lib/cli.py", line 43, in 
_load_config
    load_environment(conf.global_conf, conf.local_conf)
  File "/usr/lib/pymodules/python2.7/ckan/config/environment.py", line 
111, in load_environment
    model.init_model(engine)
  File "/usr/lib/pymodules/python2.7/ckan/model/__init__.py", line 45, in 
init_model
    version_table = Table('migrate_version', metadata, autoload=True)
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/schema.py", line 210, 
in __new__
    table._init(name, metadata, *args, **kw)
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/schema.py", line 266, 
in _init
    reflecttable(self, include_columns=include_columns)
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/engine/base.py", line 
1859, in reflecttable
    conn = self.contextual_connect()
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/engine/base.py", line 
1829, in contextual_connect
    self.pool.connect(), 
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/pool.py", line 182, 
in connect
    return _ConnectionFairy(self).checkout()
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/pool.py", line 369, 
in __init__
    rec = self._connection_record = pool.get()
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/pool.py", line 213, 
in get
    return self.do_get()
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/pool.py", line 732, 
in do_get
    con = self.create_connection()
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/pool.py", line 147, 
in create_connection
    return _ConnectionRecord(self)
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/pool.py", line 253, 
in __init__
    self.connection = self.__connect()
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/pool.py", line 319, 
in __connect
    connection = self.__pool._creator()
  File 
"/usr/lib/pymodules/python2.7/ckan/sqlalchemy/engine/strategies.py", line 
82, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/lib/pymodules/python2.7/ckan/sqlalchemy/engine/default.py", 
line 249, in connect
    return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (OperationalError) FATAL:  la base de 
donn?es << test >> n'existe pas
 None None

It looks like the PostgreSQL database is not setup correctly.
Am I missing something obvious.

Regards,
Pascal Romain

-----ckan-dev-bounces at lists.okfn.org a écrit : ----- 
A : CKAN Development Discussions <ckan-dev at lists.okfn.org>
De : James Gardner 
Envoyé par : ckan-dev-bounces at lists.okfn.org
Date : 14/11/2011 14:10inst
Objet : [ckan-dev] CKAN 1.5 .deb Packaging Update

Hi all,

The release of CKAN 1.5 brought with it a new approach to Debian 
packaging involving virtual environments based on the feedback from the 
community meeting. The new docs for installing CKAN 1.5 from packages 
are here:

     
http://docs.ckan.org/docs/ckan/en/ckan-1.5/install-from-package.html#run-the-package-installer


In case you don't want to read the full docs I'll just take you through 
the key steps here. As always, this only works on Ubuntu 10.04.

Let's say we have created a CKAN instance called "test" like this:

     sudo apt-get update
     sudo apt-get install -y wget
     echo "deb http://apt.okfn.org/ckan-1.5 lucid universe" | sudo tee 
/etc/apt/sources.list.d/okfn.list
     wget -qO- "http://apt.okfn.org/packages_public.key" | sudo apt-key 
add -
     sudo apt-get update
     sudo apt-get install -y ckan postgresql-8.4 solr-jetty
     sudo ckan-setup-solr
     sudo ckan-create-instance test test.ckan.net yes

In order to run paster commands that use the test instance's virtualenv 
you'll need to install pastescript into the virtualenv too:

     sudo -u ckantest /var/lib/ckan/dguuklp/pyenv/bin/pip install 
--ignore-installed pastescript

You can now install custom extensions for this instance like this:

     sudo -u ckantest /var/lib/ckan/test/pyenv/bin/pip install 
<name-of-extension>

Notice that you have to run everything as the ckantest user that was 
created for you and that each command must use the full paths to the 
scripts in the test instance's virtual environment.

Any time you make a change you can let Apache know like this:

     sudo touch /var/lib/ckan/std/wsgi.py

Then you must *force* refresh your browser to see changes (since CKAN 
tells it to cache its pages by default).

One other thing that is worth pointing out (and that isn't covered in 
the docs) is that because multiple CKAN instances on the same server are 
now supported you have a different database and database user for each 
instance. If you want to load one database dump into another database, 
you therefore have to change the ownership of the tables to match the 
instance. I've written some simple SQL to help. Here's a brief tutorial 
to show how to restore a database dump to a different instance...

Let's copy a database dump from the "live" database to our new instance 
called "test" for testing.

1. First stop Apache so there are no active database connections:

     sudo /etc/init.d/apache2 stop

2. Drop the old database and restore the new one (setting up the plpgsql 
language as you do):

     sudo -u postgres dropdb test
     sudo -u postgres createdb test
     sudo -u postgres createlang plpgsql test
     sudo -u postgres psql test -f to_restore.pg_dump

3. Change the table permissions (in this case the old instance name was 
called live, the new one is test)@

     sudo -u postgres psql test

Then from the psql command line interface run:

begin;
CREATE OR REPLACE FUNCTION changeowner(text, text)
RETURNS TEXT STRICT VOLATILE AS '
DECLARE
     old ALIAS FOR $1;
     new ALIAS FOR $2;
     rel record;
     sql text;
BEGIN
     FOR rel IN select * from pg_tables where tableowner=old
     LOOP sql :=  ''ALTER TABLE "'' || rel.tablename || ''" OWNER to '' 
|| new;
         RAISE NOTICE ''%'', sql;
         EXECUTE sql;
     END LOOP;
     RETURN ''OK'';
END;
' LANGUAGE 'plpgsql';
SELECT  changeowner('live', 'test');
commit;
\q

You will need to change the call to changeowner('live', 'test'); to 
reflect the old and new names of the database owner. Now your test 
instance has the correct permissions.

Once you've changed the database, you'll also need to re-build the Solr 
search index (this can take an hour or two if there is a lot of data):

     sudo -u ckantest  /var/lib/ckan/test/pyenv/bin/paster --plugin=ckan 
search-index rebuild --config=/etc/ckan/test/test.ini

Notice that you are using the full path to the paster within the virtual 
environment (so that it has access to the extensions you installed) and 
that you are running as the ckantest user.

As well as being able to use the new CKAN packaging, you might be 
interested to know that with this release, the work of building the CKAN 
packages is no longer soley in the domain of the OKF. I've written a 
quick blog post here about how the newly released packages were built 
with buildkit automatically from Python code here:

      http://ckan.org/2011/11/14/building-ckan-debian-packages/

The script that does the actual building is linked below:

     https://bitbucket.org/okfn/ckan/src/release-v1.5/build.sh

Using this script it should be possible for anyone who fancies it to 
build unofficial CKAN packages, you don't have to use the official OKF 
ones. There is also some documentation for those of you who are 
interested here:

     
http://packages.python.org/buildkit/manual.html#example-building-and-testing-the-ckan-package-install


You can get hold of buildkit here:

     http://pypi.python.org/pypi/buildkit/0.2.0
     git clone git://git.3aims.com/buildkit

Hope that's a useful introduction. You should read the docs linked to 
for full information, but do let me know if you have any questions.

Cheers,

James



_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
http://lists.okfn.org/mailman/listinfo/ckan-dev

__________________________________________________________________

Ce message et toutes les pièces jointes sont confidentiels et établis à 
l'intention exclusive de ses destinataires. Ce message ne constitue pas un 
document officiel. Seuls les documents revêtus de la signature du 
Président du Conseil Général ou d'un de ses délégataires sont de nature à 
engager le Département. 
Toute utilisation ou diffusion non autorisée est interdite. Tout message 
électronique est susceptible d'altération et le Département de la Gironde 
décline toute responsabilité au titre de ce message s'il a été altéré, 
déformé, falsifié.


_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
http://lists.okfn.org/mailman/listinfo/ckan-dev

_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
http://lists.okfn.org/mailman/listinfo/ckan-dev



__________________________________________________________________

Ce message et toutes les pièces jointes sont confidentiels et établis à l'intention exclusive de ses destinataires. Ce message ne constitue pas un document officiel. Seuls les documents revêtus de la signature du Président du Conseil Général ou d'un de ses délégataires sont de nature à engager le Département. 
Toute utilisation ou diffusion non autorisée est interdite. Tout message électronique est susceptible d'altération et le Département de la Gironde décline toute responsabilité au titre de ce message s'il a été altéré, déformé, falsifié.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20111115/19a6ca4f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 7254 bytes
Desc: not available
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20111115/19a6ca4f/attachment-0001.jpe>


More information about the ckan-dev mailing list