[ckan-dev] problems with paster db init - can't parse URL
Jason Tan
jason.tan at ersa.edu.au
Fri Jun 13 05:14:58 UTC 2014
Thanks for taking a look NIgel.
The passwords use the exact same characters as in the string. I've just
messed the order of the chars around in the email.
I was conscious that it might be a special character so I was careful to
include all the actual characters - though as you say - no character looks
very special in those strings.
I was also concerned about unprintable characters so I've checked in vi
with "set list" set , with no unprintable shown, and I've also cut and
paste the passwords. usernames and hostnames in the email into psql on the
command line, before I moved the order around and it works fine.
My very initial thought were "oh I've screwed up pg_hba.conf, what's new?".
But that appears not be the case since psql from the app server works where
paster run is OK.
I'll keep plugging away and let you all know when I find a solution/work
around or cause.
Mean while if anyone has any suggestions, I'm all ears.
Thanks again Nigel.
Regards
Jason
On 13 June 2014 13:31, Nigel Babu <nigel.babu at okfn.org> wrote:
> Hi Jason,
>
> I can't find anything obviously wrong. But from the error, it's fairly
> clear,
> it's a typo of some sort. Do you happen to have '@' or ':' characters in
> your
> password?
>
> On Thu, Jun 12, 2014 at 09:55:45AM +0930, Jason Tan wrote:
> > Hi Folks,
> >
> > Sorry to be asking here, this feels like a problem I should be able to
> > solve myself, but I've been trying to work it out for most of two days
> with
> > no luck.
> >
> > I am installing CKAN 2.1.1 and am having a problem with the
> > paster db init command.
> >
> > The complete error is below, but the last line of the error is:
> > "Could not parse rfc1738 URL from string '%s'" % name)
> > sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string
> > * 'ckan_prd'*
> >
> >
> > The problem appears to be with one of the database connection strings.
> > If I copy and paste the sername, passwd and host from the database
> > connection strings to psql on the app server the strings seem to work OK
> >
> > Here are the database strings:
> > (prd)[ckan at ckan-app ckan]$ echo $NODE
> > prd
> > (prd)[ckan at ckan-app ckan]$ grep -i postgresql
> /etc/ckan/${NODE}/${NODE}.ini
> >
> > sqlalchemy.url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
> >
> > ckan.datastore.write_url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/datastore_prd
> >
> > ckan.datastore.read_url = postgresql://
> > datastore_prd:P5ngoueU at ckan-db-prd.green.ersa.edu.au/datastore_prd
> >
> > I have this running in dev and apart from the fact that I am using
> > "default" instead of prd in database and usernames and I have different
> > passwords, the database strings look the same to me.
> >
> > I've found this link
> > https://code.google.com/p/sqlalchemy-migrate/issues/detail?id=54 which
> > suggests there was a bug under python 2.5 but I am using 2.6 and the same
> > version in dev, which appears to work OK and I've got the most recent
> > pyhton package available for the distro (centos 6.5).
> >
> > Below is a more complete output of the steps I am taking with commentary
> > inbold.
> >
> > Thanks and Regards
> > Jason
> >
> >
> >
> > *I have an env var NODE which I use in the command lines. In this
> instance
> > it evaluates to prd*
> > (prd)[ckan at ckan-app ckan]$ echo $NODE
> > prd
> >
> > *I think I am in the correct dir*
> > (prd)[ckan at ckan-app ckan]$ pwd
> > /usr/lib/ckan/prd/src/ckan
> >
> > *lets look for ckan_ - which is a string the exception stack trace refers
> > to:*
> > (prd)[ckan at ckan-app ckan]$ grep -i ckan_ /etc/ckan/prd/prd.ini
> > sqlalchemy.url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
> > #sqlalchemy.url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
> > #sqlalchemy.url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
> > ckan.datastore.write_url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/datastore_prd
> >
> > *check that env var NODE is set, then run the command*
> > (prd)[ckan at ckan-app ckan]$ echo $NODE
> > prd
> > (prd)[ckan at ckan-app ckan]$ paster db init -c
> /etc/ckan/${NODE}/${NODE}.ini
> > Traceback (most recent call last):
> > File "/usr/lib/ckan/prd/bin/paster", line 9, in <module>
> > load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
> > line 104, in run
> > invoke(command, command_name, options, args[1:])
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
> > line 143, in invoke
> > exit_code = runner.run(args)
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
> > line 238, in run
> > result = self.command()
> > File "/usr/lib/ckan/prd/src/ckan/ckan/lib/cli.py", line 129, in command
> > self._load_config()
> > File "/usr/lib/ckan/prd/src/ckan/ckan/lib/cli.py", line 91, in
> > _load_config
> > load_environment(conf.global_conf, conf.local_conf)
> > File "/usr/lib/ckan/prd/src/ckan/ckan/config/environment.py", line 344,
> > in load_environment
> > engine = sqlalchemy.engine_from_config(config, 'sqlalchemy.',
> **extras)
> > File
> >
> "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
> > line 356, in engine_from_config
> > return create_engine(url, **opts)
> > File
> >
> "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
> > line 338, in create_engine
> > return strategy.create(*args, **kwargs)
> > File
> >
> "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/strategies.py",
> > line 48, in create
> > u = url.make_url(name_or_url)
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/url.py",
> > line 178, in make_url
> > return _parse_rfc1738_args(name_or_url)
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/url.py",
> > line 219, in _parse_rfc1738_args
> > "Could not parse rfc1738 URL from string '%s'" % name)
> > sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string
> > 'ckan_prd'
> > (prd)[ckan at ckan-app ckan]$ paster db init -c /etc/ckan/prd/prd.ini
> > Traceback (most recent call last):
> > File "/usr/lib/ckan/prd/bin/paster", line 9, in <module>
> > load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
> > line 104, in run
> > invoke(command, command_name, options, args[1:])
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
> > line 143, in invoke
> > exit_code = runner.run(args)
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/paste/script/command.py",
> > line 238, in run
> > result = self.command()
> > File "/usr/lib/ckan/prd/src/ckan/ckan/lib/cli.py", line 129, in command
> > self._load_config()
> > File "/usr/lib/ckan/prd/src/ckan/ckan/lib/cli.py", line 91, in
> > _load_config
> > load_environment(conf.global_conf, conf.local_conf)
> > File "/usr/lib/ckan/prd/src/ckan/ckan/config/environment.py", line 344,
> > in load_environment
> > engine = sqlalchemy.engine_from_config(config, 'sqlalchemy.',
> **extras)
> > File
> >
> "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
> > line 356, in engine_from_config
> > return create_engine(url, **opts)
> > File
> >
> "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
> > line 338, in create_engine
> > return strategy.create(*args, **kwargs)
> > File
> >
> "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/strategies.py",
> > line 48, in create
> > u = url.make_url(name_or_url)
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/url.py",
> > line 178, in make_url
> > return _parse_rfc1738_args(name_or_url)
> > File
> > "/usr/lib/ckan/prd/lib/python2.6/site-packages/sqlalchemy/engine/url.py",
> > line 219, in _parse_rfc1738_args
> > "Could not parse rfc1738 URL from string '%s'" % name)
> > sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string*
> > 'ckan_prd'*
> >
> > *OK we get an error lets check for things with "ckan_prd" in the config
> > file*
> > (prd)[ckan at ckan-app ckan]$ grep -i ckan_prd
> /etc/ckan/${NODE}/${NODE}.ini
> > sqlalchemy.url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
> > #sqlalchemy.url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
> > #sqlalchemy.url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/ckan_prd
> > ckan.datastore.write_url = postgresql://
> > ckan_prd:O1eakfeo at ckan-db-prd.green.ersa.edu.au/datastore_prd
> >
> > *OK so cutting and pasting the database name, username and password from
> > the above grep output lets try to connect with psql*
> > (prd)[ckan at ckan-app ckan]$ psql -h ckan-db-prd.green.ersa.edu.au
> ckan_prd
> > ckan_prd
> > Password for user ckan_prd:
> > psql (9.0.17)
> > Type "help" for help.
> >
> > ckan_prd=> \q
> > (prd)[ckan at ckan-app ckan]$ psql -h ckan-db-prd.green.ersa.edu.au
> > datastore_prd ckan_prd
> > Password for user ckan_prd:
> > psql (9.0.17)
> > Type "help" for help.
> > datastore_prd=> \q
> >
> > *The last database string is the datastore.read_url - it does not include
> > "ckan_prd" but lets test it anyway:*
> > (prd)[ckan at ckan-app ckan]$ grep read_url /etc/ckan/${NODE}/${NODE}.ini
> > ckan.datastore.read_url = postgresql://
> > datastore_prd:P5ngoueU at ckan-db-prd.green.ersa.edu.au/datastore_prd
> >
> > (prd)[ckan at ckan-app ckan]$ psql -h ckan-db-prd.green.ersa.edu.au
> > datastore_prd datastore_prd
> > Password for user datastore_prd:
> > psql (9.0.17)
> > Type "help" for help.
> > datastore_prd=> \q
> > (prd)[ckan at ckan-app ckan]$
> > *Datasotre read URL seems to be OK*
> >
> >
> >
> >
> > --
> > Jason Tan
> > Data Infrastructure Specialist, eResearch SA
> >
> > phone: +61 8 8313 2184
> > Mob: +61 407 268 270
> > email: jason.tan at ersa.edu.au
> > web: http://www.ersa.edu.au
>
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
> --
> Nigel Babu
> Developer, Open Knowledge
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
--
Jason Tan
Data Infrastructure Specialist, eResearch SA
phone: +61 8 8313 2184
Mob: +61 407 268 270
email: jason.tan at ersa.edu.au
web: http://www.ersa.edu.au
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140613/97cccb1e/attachment-0003.html>
More information about the ckan-dev
mailing list