[ckan-dev] DataStore config questions

Thu May 31 13:38:14 UTC 2012

Hi,

I'm encountering the same problem Jim has : no way to get a preview of any
resource.
I have the same configuration (nginx as a front end to apache running CKAN 1.7
installed using the package installer)

Every time I add a resource to a dataset I get messages in celeryd.log :
[2012-05-31 15:04:00,671: INFO/MainProcess] Got task from broker:
datastorer.upload[0890a4c9-1907-4722-b301-afddb3db01f1]
[2012-05-31 15:04:00,765: INFO/MainProcess] Task
archiver.update[1fe739f5-5be4-4404-b53a-be22f326b991] succeeded in
0.074896812439s: None
[2012-05-31 15:04:01,516: INFO/MainProcess] Task
datastorer.upload[0890a4c9-1907-4722-b301-afddb3db01f1] succeeded in
0.827142000198s: None

Yet, no preview.

I tried the curl command you gave and got :

curl -XPUT
http://datalift.si.fr.atosorigin.com/api/data/93da711c-9b38-484d-a52c-d7fa2b387582/1
-d '{"test":"data"}' -H 'Authorization: ...'
<html>
  <head>
   <title>404 Not Found</title>
  </head>
  <body>
   <h1>404 Not Found</h1>
   The resource could not be found.<br /><br />

  </body>
</html>

Any idea ?

TIA,

Laurent

Le 27/05/12 16:46, David Raznick a écrit :
>
>
> On Sun, May 27, 2012 at 1:49 PM, Jim Craner <jim at codeforamerica.org
> <mailto:jim at codeforamerica.org>> wrote:
>
>     Hi everyone,
>
>     I'm having some difficulties getting the DataStore feature working on
>     my CKAN 1.7 / Ubuntu 10.04 LTS (package install) implementation. What
>     I've done so far:
>
>     * ElasticSearch is running on port 9200 (lynx localhost:9200 returns a
>     json file with a 200 OK code and the ES version string)
>
>     * I have nginx installed and running as a front-end for Apache2. I
>     used this gist as my guide for this:
>
>     https://gist.github.com/2637431
>
>     Seems to be working fine -- nginx is running on 80, passing off to
>     default CKAN apache on 8080.
>
>     * I added the custom elastic snippet to the nginx.conf file
>     referencing elasticsearch on localhost:9200
>
>     * I set ckan.datastore.enabled =1 in my ckan .ini file.
>
>     * I installed the latest version of ckanext-datastorer, supervisor,
>     and set up the celery supervisor script
>
>     -- At this point, am I missing something from the configuration?
>
>     * I then uploaded a simple test.csv file resource into a test dataset,
>     checking the DataStore/API option for the resource
>
>     * When I visit the resource's page, the Preview section just displays
>     a thin pink bar across the screen (immediately below the "Embed"
>     button), just as it did before I enabled/config'ed the DataStore
>
>     * When I visit <site>/api/data/<resource_id>/_search?size=5&pretty=true,
>     I get the following message:
>
>     "error": "IndexMissingException[[ckan-data] missing]",
>     "status": 404
>
> It looks like the file upload has not worked. The index only gets created on
> the first successful command.
>
> The common error here is when uploading the file you are looking at the nginx
> instance on 80 not ckan-apache on 8080. Also make sure you add a new resource
> we currently do not reload when a resource changes.
>
> You could also try a simple test of the datastore with something like.
>
> curl -XPUT <site>/api/data/<resource_id>/1 -d '{"test":"data"}' -H
> 'Authorization: your-api-key'
>
> (making sure site is the nginx site)
>
> This will make a new record of id 1 against that resource. The search above
> and preview should work now, returning '{"test":"data"}'.
>
> If that does not work then it should give you some feedback at to why.
>
> If that does works then could you give me the output of celeryd as it should
> error if something has gone wrong with the upload.
>
> Thanks
>
> David
>
>
>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev

--
                  wWw
                 (o o)
-------------ooO-(_)-Ooo-----------------------------------------------
Laurent Bihanic              | Tel: +33 (0)1 73 26 12 92
Atos Systems Integration     | GSM: +33 (0)6 25 73 03 74
80 quai Voltaire             |
F-95870 Bezons               | mailto: laurent.bihanic at atos.net
-----------------------------------------------------------------------
Very funny, Scotty. Now beam down my clothes.

Please consider your environmental responsibility before printing
this e-mail.

DISCLAIMER:
The opinions expressed are entirely my own and may not necessarily be
those of my employer.  Also, I am not now nor have I ever been a
lawyer.  My opinions are provided as-is with absolutely no warranty of
merchantability or fitness for any particular use.  Besides, you can't
prove I typed this.  No body saw me type this.  Who says I typed this?
________________________________

Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité du groupe Atos ne pourra être engagée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être engagée pour tout dommage résultant d'un virus transmis.

This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos group liability cannot be triggered for the message content. Although the sender endeavors to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.