[ckan-dev] problems with paster db init - can't parse URL

• Previous message: [ckan-dev] Send email notifications
• Next message: [ckan-dev] problems with paster db init - can't parse URL
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi Folks,

Sorry to be asking here, this feels like a problem I should be able to
solve myself, but I've been trying to work it out for most of two days with
no luck.

I am installing CKAN 2.1.1 and am having a problem with the
paster db init command.

The complete error is below, but the last line of the error is:
    "Could not parse rfc1738 URL from string '%s'" % name)
sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string
* 'ckan_prd'*


The problem appears to be with one of the database connection strings.
If I copy and paste the sername, passwd and host from the database
connection strings to psql on the app server the strings seem to work OK

Here are the database strings:
(prd)[ckan at ckan-app ckan]$ echo $NODE
prd
(prd)[ckan at ckan-app ckan]$ grep -i postgresql  /etc/ckan/${NODE}/${NODE}.ini

sqlalchemy.url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd

ckan.datastore.write_url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/datastore_prd

ckan.datastore.read_url = postgresql://
datastore_prd:P5ngoueU at ckan-db-prd.green.ersa.edu.au/datastore_prd

I have this running in dev and apart from the fact that I am using
"default" instead of prd in database and usernames and I have different
passwords, the database strings look the same to me.

I've found this link
https://code.google.com/p/sqlalchemy-migrate/issues/detail?id=54 which
suggests there was a bug under python 2.5 but I am using 2.6 and the same
version in dev, which appears to work OK and I've got the most recent
pyhton package available for the distro (centos 6.5).

Below is a more complete output of the steps I am taking with commentary
inbold.

Thanks and Regards
Jason



*I have an env var NODE which I use in the command lines. In this instance
it evaluates to prd*
(prd)[ckan at ckan-app ckan]$ echo $NODE
prd

*I think I am in the correct dir*
(prd)[ckan at ckan-app ckan]$ pwd
/usr/lib/ckan/prd/src/ckan

*lets look for ckan_ - which is a string the exception stack trace refers
to:*
(prd)[ckan at ckan-app ckan]$ grep -i ckan_ /etc/ckan/prd/prd.ini
sqlalchemy.url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
#sqlalchemy.url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
#sqlalchemy.url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
ckan.datastore.write_url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/datastore_prd

*check that env var NODE is set, then run the command*
(prd)[ckan at ckan-app ckan]$ echo $NODE
prd
(prd)[ckan at ckan-app ckan]$ paster db init -c /etc/ckan/${NODE}/${NODE}.ini
Traceback (most recent call last):
  File "/usr/lib/ckan/prd/bin/paster", line 9, in <module>
    load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
line 104, in run
    invoke(command, command_name, options, args[1:])
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
line 143, in invoke
    exit_code = runner.run(args)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
line 238, in run
    result = self.command()
  File "/usr/lib/ckan/prd/src/ckan/ckan/lib/cli.py", line 129, in command
    self._load_config()
  File "/usr/lib/ckan/prd/src/ckan/ckan/lib/cli.py", line 91, in
_load_config
    load_environment(conf.global_conf, conf.local_conf)
  File "/usr/lib/ckan/prd/src/ckan/ckan/config/environment.py", line 344,
in load_environment
    engine = sqlalchemy.engine_from_config(config, 'sqlalchemy.', **extras)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
line 356, in engine_from_config
    return create_engine(url, **opts)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
line 338, in create_engine
    return strategy.create(*args, **kwargs)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/strategies.py",
line 48, in create
    u = url.make_url(name_or_url)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/url.py",
line 178, in make_url
    return _parse_rfc1738_args(name_or_url)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/url.py",
line 219, in _parse_rfc1738_args
    "Could not parse rfc1738 URL from string '%s'" % name)
sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string
'ckan_prd'
(prd)[ckan at ckan-app ckan]$ paster db init -c /etc/ckan/prd/prd.ini
Traceback (most recent call last):
  File "/usr/lib/ckan/prd/bin/paster", line 9, in <module>
    load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
line 104, in run
    invoke(command, command_name, options, args[1:])
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
line 143, in invoke
    exit_code = runner.run(args)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
line 238, in run
    result = self.command()
  File "/usr/lib/ckan/prd/src/ckan/ckan/lib/cli.py", line 129, in command
    self._load_config()
  File "/usr/lib/ckan/prd/src/ckan/ckan/lib/cli.py", line 91, in
_load_config
    load_environment(conf.global_conf, conf.local_conf)
  File "/usr/lib/ckan/prd/src/ckan/ckan/config/environment.py", line 344,
in load_environment
    engine = sqlalchemy.engine_from_config(config, 'sqlalchemy.', **extras)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
line 356, in engine_from_config
    return create_engine(url, **opts)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
line 338, in create_engine
    return strategy.create(*args, **kwargs)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/strategies.py",
line 48, in create
    u = url.make_url(name_or_url)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/url.py",
line 178, in make_url
    return _parse_rfc1738_args(name_or_url)
  File
"/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/url.py",
line 219, in _parse_rfc1738_args
    "Could not parse rfc1738 URL from string '%s'" % name)
sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string*
'ckan_prd'*

*OK we get an error lets check for things with "ckan_prd" in the config
file*
(prd)[ckan at ckan-app ckan]$ grep -i ckan_prd  /etc/ckan/${NODE}/${NODE}.ini
sqlalchemy.url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
#sqlalchemy.url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
#sqlalchemy.url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
ckan.datastore.write_url = postgresql://
ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/datastore_prd

*OK so cutting and pasting the database name, username and password from
the above grep output lets try to connect with psql*
(prd)[ckan at ckan-app ckan]$ psql -h ckan-db-prd.green.ersa.edu.au ckan_prd
ckan_prd
Password for user ckan_prd:
psql (9.0.17)
Type "help" for help.

ckan_prd=> \q
(prd)[ckan at ckan-app ckan]$ psql -h ckan-db-prd.green.ersa.edu.au
datastore_prd ckan_prd
Password for user ckan_prd:
psql (9.0.17)
Type "help" for help.
datastore_prd=> \q

*The last database string is the datastore.read_url - it does not include
"ckan_prd" but lets test it anyway:*
(prd)[ckan at ckan-app ckan]$ grep read_url  /etc/ckan/${NODE}/${NODE}.ini
ckan.datastore.read_url = postgresql://
datastore_prd:P5ngoueU at ckan-db-prd.green.ersa.edu.au/datastore_prd

(prd)[ckan at ckan-app ckan]$ psql -h ckan-db-prd.green.ersa.edu.au
datastore_prd datastore_prd
Password for user datastore_prd:
psql (9.0.17)
Type "help" for help.
datastore_prd=> \q
(prd)[ckan at ckan-app ckan]$
*Datasotre read URL seems to be OK*




 --
Jason Tan
Data Infrastructure Specialist, eResearch SA

phone: +61 8 8313 2184
Mob: +61 407 268 270
email: jason.tan at ersa.edu.au
web: http://www.ersa.edu.au
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140612/f7b73151/attachment-0002.html>

• Previous message: [ckan-dev] Send email notifications
• Next message: [ckan-dev] problems with paster db init - can't parse URL
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]