[ckan-dev] ckanext-harvest crontab
Michael Reichart
michael.reichart at gmail.com
Mon Mar 18 15:25:29 UTC 2013
Hi,
thanks for your help.
It's the same user, but it seems that the LD_LIBRARY_PATH needs to bet set
explicitly when running as a cronjob.
this is our script that's run by cron now:
#!/bin/sh
export LD_LIBRARY_PATH=/products/pgogd/lib
export PATH=/products/pgogd/lib:/products/pgogd/bin:$PATH
cd /products/ckan
source pyenv/bin/activate
paster --plugin=ckanext-harvest harvester run
--config=/home/ckan/my.paster.ini
Thanks
Michael
2013/3/18 Adrià Mercader <adria.mercader at okfn.org>
> Hi Michael,
>
> Which unix user are you running the cron job as? Is it the same that
> when calling the command directly from the command line?
> Maybe this user can not access the postgres shared libraries.
>
>
> Adrià
>
>
> On 18 March 2013 13:38, Michael Reichart <michael.reichart at gmail.com>
> wrote:
> > Hi,
> >
> > I tried your suggestion but unfortunately that doesn't change anything
> ...
> >
> > Any other ideas??
> >
> > Thanks
> > Michael
> >
> >
> > 2013/3/15 Joe Tsoi <joe.tsoi at okfn.org>
> >>
> >> oh i missed that part, are you running in the virtual env when you are
> >> running from the command line, try editing the cronjob to be
> >>
> >>
> >> */5 * * * * /products/ckan/pyenv/bin/python
> >> /products/ckan/pyenv/bin/paster --plugin=ckanext-harvest harvester run
> >> --config=/home/ckan/my.paster.ini
> >>
> >>
> >>
> >> On 15 March 2013 13:48, Michael Reichart <michael.reichart at gmail.com>
> >> wrote:
> >>>
> >>> This is on SLES
> >>>
> >>> The question is why it works perfectly, when I run it directly and only
> >>> fails when using a cronjob...
> >>> I will try to install the postgresql-libs...
> >>>
> >>> thanks!
> >>> michael
> >>>
> >>> 2013/3/15 Joe Tsoi <joe.tsoi at okfn.org>
> >>>>
> >>>> Looks like a missing shared library issue, what distro are you using?
> >>>> Run a yum install postgresql-libs or apt-get install libpq5
> >>>>
> >>>>
> >>>> On 15 March 2013 12:22, Michael Reichart <michael.reichart at gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I'm trying to set up harvester for automatic import via crontab.
> >>>>> My problem is: The paster command run by crontab fails, but the same
> >>>>> script executed manually works with no problem at all.
> >>>>>
> >>>>> my cronjob:
> >>>>> */5 * * * * /products/ckan/pyenv/bin/paster --plugin=ckanext-harvest
> >>>>> harvester run --config=/home/ckan/my.paster.ini
> >>>>>
> >>>>>
> >>>>> when I run /products/ckan/pyenv/bin/paster --plugin=ckanext-harvest
> >>>>> harvester run --config=/home/ckan/my.paster.ini from console
> everything is
> >>>>> ok.
> >>>>>
> >>>>> Crontab produces the following error:
> >>>>> No handlers could be found for logger "vdm"
> >>>>> Traceback (most recent call last):
> >>>>> File "/products/ckan/pyenv/bin/paster", line 8, in <module>
> >>>>> load_entry_point('PasteScript==1.7.3', 'console_scripts',
> >>>>> 'paster')()
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/paste/script/command.py",
> >>>>> line 84, in run
> >>>>> invoke(command, command_name, options, args[1:])
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/paste/script/command.py",
> >>>>> line 123, in invoke
> >>>>> exit_code = runner.run(args)
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/paste/script/command.py",
> >>>>> line 218, in run
> >>>>> result = self.command()
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/ckanext/harvest/commands/harvester.py",
> >>>>> line 65, in command
> >>>>> self._load_config()
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/ckanext/harvest/commands/harvester.py",
> >>>>> line 117, in _load_config
> >>>>> super(Harvester, self)._load_config()
> >>>>> File
> >>>>> "/products/ckan/pyenv/lib/python2.6/site-packages/ckan/lib/cli.py",
> line 53,
> >>>>> in _load_config
> >>>>> load_environment(conf.global_conf, conf.local_conf)
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/ckan/config/environment.py",
> >>>>> line 192, in load_environment
> >>>>> engine = sqlalchemy.engine_from_config(config, 'sqlalchemy.')
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
> >>>>> line 298, in engine_from_config
> >>>>> return create_engine(url, **opts)
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/__init__.py",
> >>>>> line 280, in create_engine
> >>>>> return strategy.create(*args, **kwargs)
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/strategies.py",
> >>>>> line 64, in create
> >>>>> dbapi = dialect_cls.dbapi(**dbapi_args)
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py",
> >>>>> line 283, in dbapi
> >>>>> psycopg = __import__('psycopg2')
> >>>>> File
> >>>>>
> "/products/ckan/pyenv/lib/python2.6/site-packages/psycopg2/__init__.py",
> >>>>> line 60, in <module>
> >>>>> from _psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
> >>>>> ImportError: libpq.so.5: cannot open shared object file: No such file
> >>>>> or directory
> >>>>>
> >>>>>
> >>>>> Any hints?
> >>>>>
> >>>>> Thanks!
> >>>>> Michael
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> ckan-dev mailing list
> >>>>> ckan-dev at lists.okfn.org
> >>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
> >>>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> >>>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> ckan-dev mailing list
> >>>> ckan-dev at lists.okfn.org
> >>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
> >>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> >>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> ckan-dev mailing list
> >>> ckan-dev at lists.okfn.org
> >>> http://lists.okfn.org/mailman/listinfo/ckan-dev
> >>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> >>>
> >>
> >>
> >> _______________________________________________
> >> ckan-dev mailing list
> >> ckan-dev at lists.okfn.org
> >> http://lists.okfn.org/mailman/listinfo/ckan-dev
> >> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> >>
> >
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> >
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130318/b2b873a0/attachment-0001.html>
More information about the ckan-dev
mailing list