[ckan-dev] DataStore config questions

Sun May 27 16:39:10 UTC 2012

On Sun, May 27, 2012 at 4:45 PM, Jim Craner <jim at codeforamerica.org> wrote:

> Thanks for the help, David.  Notes inline below...
>
> On Sun, May 27, 2012 at 7:46 AM, David Raznick <kindly at gmail.com> wrote:
>
> > The common error here is when uploading the file you are looking at the
> > nginx instance on 80 not ckan-apache on 8080.
>
> Can you clarify this for me?  My (admittedly limited) understanding is
> that nginx passes everything non-ES to ckan-apache.  Once I switched
> the CKAN apache instance to run on 8080 and set nginx to 80, I thought
> that was it.
>

That was it.  All I meant was that when interacting with the site then you
need be looking at the nginx one.

>
> > Also make sure you add a new
> > resource we currently do not reload when a resource changes.
>
> I've been deleting and re-adding the same test.csv as a new resource.
> I also tried a new Dataset altogether but I keep getting the same
> results.  I had celeryd running in a console during these tests and
> nothing appeared there (not sure if something was supposed to).
>

yes, there should be some logging of each task.  Have you run python
setup.py develop in ckanext-datastorer and added datastorer to your plugins
in your ini file.

You may need to reset both celeryd and the the webserver to make sure this
works if this is not the case.

>
> >
> > You could also try a simple test of the datastore with something like.
> >
> > curl -XPUT <site>/api/data/<resource_id>/1 -d '{"test":"data"}' -H
> > 'Authorization: your-api-key'
>
> This returned:
>
>
> {"ok":true,"_index":"ckan-data","_type":"351d8e3f-f4cd-462d-8664-fe7c027ae21e","_id":"1","_version":1}
>
> > This will make a new record of id 1 against that resource.  The search
> above
> > and preview should work now, returning '{"test":"data"}'.
>
> Success!  There's a preview grid with column "test" and value "data"
> in the resource.
>
> > If that does works then could you give me the output of celeryd as it
> should
> > error if something has gone wrong with the upload.
>
> I was running celery via paster and nothing ever appeared on the
> console after the startup message:
>
> [2012-05-27 08:00:18,806: WARNING/MainProcess] celery at dataserver has
> started.
>

Has it got this in your celery output as well?

[Tasks]
  . archiver.clean
  . archiver.link_checker
  . archiver.update
  . datastorer.upload

> I uploaded several .csv resources during that time, as well as the
> curl test mentioned above, but nothing ever showed up until I finally
> terminated it with Ctrl-C.  Is there a way to make celery generate
> more info or is it storing logs somewhere else?
>

This should be fine.  We normally make the supervisor store logs, which
would be the same as the output to the console if running manually.

>
> Thanks again for your help!  Happy to see the {test:data} preview progress
> :-)
> Jim
>
> >
> > Thanks
> >
> > David
> >
> >
> >
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ckan-dev
> >
>
>
>
> --
> -----
> Jim Craner, 2012 Fellow
> Code for America
> jim at codeforamerica.org
> (+1) 773-809-4546
> http://codeforamerica.org
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20120527/c01b6a84/attachment-0001.html>