[ckan-dev] Datapusher and APScheduler errors

Wed Jul 23 05:59:06 UTC 2014

I have been having a few issues getting the Datapusher to work correctly.
Looks like the problem has to do with apscheduler. If anyone has any ideas
about what could be wrong please let me know.

I am running ckan release-v2.2.1. The datastore doesn't return an error
when I do curl -X GET "
http://127.0.0.1:5000/api/3/action/datastore_search?resource_id=_table_metadata
" so it looks like that is working fine. I have ckan.site_url =
http://localhost:5000 set in my development.ini file.

When I try to run the Datapusher with
JOB_CONFIG='/home/ckanenv/src/datapusher/deployment/datapusher_settings.py'
python wsgi.py I get an error which I've pasted below as Stack Trace 1. I
see that the version of APScheduler installed is 3.0.0rc1. Looking at the
apscheduler site-package I see apscheduler.schedulers but not
apscheduler.scheduler as the ckanserviceprovider package is referencing.
There are a bunch of subsequent similar errors if I attempt to modify
ckanserviceprovider. So at some point apscheduler was updated but
ckanserviceprovider was not updated to work with the new version.

If I roll back APScheduler to 2.1.2 then
JOB_CONFIG='/home/ckanenv/src/datapusher/deployment/datapusher_settings.py'
python wsgi.py executes successfully and I can see the Datapusher running
at http://localhost:8800/. However, if I try to upload a very simple csv
file, with no special characters, I get the error in Stack Trace 2 below.
Files are uploaded but they are not pushed to the datastore.

How can I get the Datapusher to run properly?

Stack Trace 1:
(ckanenv)ngds at ngds-VirtualBox:~/ckanenv/src/datapusher$
JOB_CONFIG='/home/ckanv/src/datapusher/deployment/datapusher_settings.py'
python wsgi.py
Traceback (most recent call last):
  File "wsgi.py", line 1, in <module>
    import ckanserviceprovider.web as web
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/ckanserviceprovider/web.py",
line 16, in <module>
    import apscheduler.scheduler as apscheduler
ImportError: No module named scheduler

Stack Trace 2:
(ckanenv)ngds at ngds-VirtualBox:~/ckanenv/src/datapusher$
JOB_CONFIG='/home/ckanv/src/datapusher/deployment/datapusher_settings.py'
python wsgi.py
/home/ngds/ckanenv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py:463:
SAWarning: Unicode type received non-unicode bind param value.
  param.append(processors[key](compiled_params[key]))
Exception in thread APScheduler:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/apscheduler/scheduler.py",
line 581, in _main_loop
    next_wakeup_time = self._process_jobs(now)
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/apscheduler/scheduler.py",
line 560, in _process_jobs
    self._remove_job(job, alias, jobstore)
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/apscheduler/scheduler.py",
line 294, in _remove_job
    jobstore.remove_job(job)
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/apscheduler/jobstores/sqlalchemy_store.py",
line 64, in remove_job
    self.engine.execute(delete)
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 2447, in execute
    return connection.execute(statement, *multiparams, **params)
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1449, in execute
    params)
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1584, in _execute_clauseelement
    compiled_sql, distilled_params
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1698, in _execute_context
    context)
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1691, in _execute_context
    context)
  File
"/home/ngds/ckanenv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py",
line 331, in do_execute
    cursor.execute(statement, parameters)
OperationalError: (OperationalError) no such table: apscheduler_jobs
u'DELETE FROM apscheduler_jobs WHERE apscheduler_jobs.id = ?' (1,)

Jessica Good
GIS Specialist & Developer
Arizona Geological Survey
http://www.azgs.az.gov/
jessica.good at azgs.az.gov <jessica.alisdairi at azgs.az.gov>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140722/bb9a0fd8/attachment-0002.html>