[ckan-dev] Setting up DataStore

Thu Oct 4 19:29:58 UTC 2012

Hello Joshua,

The datastore works with PostgreSQL 9.0 to 9.2. CKAN 2 runs on all distros. The OKFN uses debian based distros for their demos and the datahub.

You should be fine with your favourite distro and the Postgres version that comes with it. 

Dominik

On Oct 4, 2012, at 18:17 , Joshua Tauberer <tauberer+consulting at govtrack.us> wrote:

> Ahha, ok.
> 
> What's the intended distro/postgres for CKAN 2?
> 
> Thanks for your help.
> 
> -- 
> - Joshua Tauberer
> - http://razor.occams.info
> 
> 
> 
> On 10/03/2012 03:07 PM, Dominik Moritz wrote:
>> Hello Joshua,
>> 
>> This problem occurs because the paster command uses commands that are only available in Postgres 9.0+. We are working on support for versions prior to 9.0 but it has not been implemented yet. Sorry about that.
>> 
>> Cheers,
>> Dominik
>> 
>> On Oct 3, 2012, at 18:23 , Joshua Tauberer <tauberer+consulting at govtrack.us> wrote:
>> 
>>> Thanks, Dominik.
>>> 
>>> That solves that. There are a few new problems then. If I run it on a clean db, create-db works fine but in create-read-only-user I get:
>>> 
>>> An error occured: ERROR:  role "(my read-write user)" does not exist
>>> ERROR:  role "(my read-write user)" does not exist
>>> ERROR:  syntax error at or near "ALL"
>>> LINE 1: GRANT SELECT ON ALL TABLES IN SCHEMA public TO "...
>>> 
>>> If I create the read-write user on my own first and then run create-db (ok), create-read-only-user gives:
>>> 
>>> An error occured: ERROR:  syntax error at or near "ALL"
>>> LINE 1: GRANT SELECT ON ALL TABLES IN SCHEMA public TO "...
>>>                        ^
>>> ERROR:  syntax error at or near "DEFAULT"
>>> LINE 1: ALTER DEFAULT PRIVILEGES FOR USER "ckanuser" IN SCHEMA publi...
>>>              ^
>>> 
>>> By the way, in case this is helpful for the documentation: To get the environment right (paster on the PATH, access to db, write permission on the log file) I had to do this:
>>> 
>>> sudo su postgres -c  ". /home/ubuntu/pyenv/bin/activate; paster datastore ....
>>> 
>>> -- 
>>> - Joshua Tauberer
>>> - http://razor.occams.info
>>> 
>>> 
>>> On 10/03/2012 11:55 AM, Dominik Moritz wrote:
>>>> Hi Joshua,
>>>> 
>>>> My mistake. This will be fixed as soon as the changes are merged into master. For now you will have to run the paster command without the --plugin=ckan argument.
>>>> 
>>>> Please try : paster datastore create-db PG_USER
>>>> 
>>>> The datastore is quite new and we still need experience with setups. Thank you very much for your bug report.
>>>> 
>>>> Cheers,
>>>> Dominik
>>>> 
>>>> 
>>>> On Oct 3, 2012, at 16:16 , Joshua Tauberer <tauberer+consulting at govtrack.us> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> I'm trying out DataStore (source install from the master branch).
>>>>> 
>>>>> Following the docs, I'm stuck at the step of creating the database. It seems like I'm not supposed to create the database or users myself --- that's what paster datastore create-db/create-read-only-user are supposed to do.
>>>>> 
>>>>> However, when I run these commands an exception is thrown during initialization complaining about not being able to authenticate as the user configured in the ini --- which of course doesn't exist yet. Here's the exception:
>>>>> 
>>>>> $ paster --plugin=ckan datastore create-db postgres
>>>>> Traceback (most recent call last):
>>>>>  File "/home/ubuntu/pyenv/bin/paster", line 9, in <module>
>>>>>    load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 104, in run
>>>>>    invoke(command, command_name, options, args[1:])
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 143, in invoke
>>>>>    exit_code = runner.run(args)
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/paste/script/command.py", line 238, in run
>>>>>    result = self.command()
>>>>> *  File "/home/ubuntu/pyenv/src/ckan/ckanext/datastore/commands.py", line 72, in command**
>>>>> **    self._load_config()**
>>>>> *  File "/home/ubuntu/pyenv/src/ckan/ckan/lib/cli.py", line 85, in _load_config
>>>>>    load_environment(conf.global_conf, conf.local_conf)
>>>>>  File "/home/ubuntu/pyenv/src/ckan/ckan/config/environment.py", line 353, in load_environment
>>>>>    plugin.configure(config)
>>>>>  File "/home/ubuntu/pyenv/src/ckan/ckanext/datastore/plugin.py", line 49, in configure
>>>>>    if not self._is_read_only_database():
>>>>>  File "/home/ubuntu/pyenv/src/ckan/ckanext/datastore/plugin.py", line 99, in _is_read_only_database
>>>>>    {'connection_url': url}).connect()
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 2472, in connect
>>>>>    return self._connection_cls(self, **kwargs)
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 878, in __init__
>>>>>    self.__connection = connection or engine.raw_connection()
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 2558, in raw_connection
>>>>>    return self.pool.unique_connection()
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/pool.py", line 183, in unique_connection
>>>>>    return _ConnectionFairy(self).checkout()
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/pool.py", line 387, in __init__
>>>>>    rec = self._connection_record = pool._do_get()
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/pool.py", line 741, in _do_get
>>>>>    con = self._create_connection()
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/pool.py", line 188, in _create_connection
>>>>>    return _ConnectionRecord(self)
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/pool.py", line 270, in __init__
>>>>>    self.connection = self.__connect()
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/pool.py", line 330, in __connect
>>>>>    connection = self.__pool._creator()
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/strategies.py", line 80, in connect
>>>>>    return dialect.connect(*cargs, **cparams)
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/default.py", line 281, in connect
>>>>>    return self.dbapi.connect(*cargs, **cparams)
>>>>>  File "/home/ubuntu/pyenv/lib/python2.6/site-packages/psycopg2/__init__.py", line 179, in connect
>>>>>    connection_factory=connection_factory, async=async)
>>>>> sqlalchemy.exc.OperationalError: (OperationalError) FATAL:  password authentication failed for user "ckan_datastore_rw"
>>>>> FATAL:  password authentication failed for user "ckan_datastore_rw"
>>>>> 
>>>>> If I create the user, it then throws an exception about the database not existing. If I then create the db, it throws about the read-only user not existing. If I create the read-only user it throws about the user having write permission on the database. And after that I gave up because I couldn't figure out what I needed to revoke to get past that problem. :)
>>>>> 
>>>>> Tips?
>>>>> 
>>>>> -- 
>>>>> - Joshua Tauberer
>>>>> - http://razor.occams.info
>>>>> 
>>>>> _______________________________________________
>>>>> ckan-dev mailing list
>>>>> ckan-dev at lists.okfn.org
>>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> 
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2726 bytes
Desc: not available
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20121004/2a4a98aa/attachment-0003.bin>