[ckan-dev] Wrong URL after file upload

Wed Sep 4 08:17:18 UTC 2013

Hi Richard,

you are most probably using an old version of ckanclient, perhaps the one
from pypi. That version has several issues that have been fixed in the code
at github. Please use this one https://github.com/okfn/ckanclient

Furthermore, ckanclient is pretty much abondoned. It still uses the
deprecated CKAN API 1 (REST).  AFAIK the recommended python client would
now be https://github.com/open-data/ckanapi.

regards
hendrik

On 3 September 2013 14:02, Richard Gomes <rgomes.info at gmail.com> wrote:

>  hello,
>
> I'm uploading files like this:
>
> cli = ckanclient.CkanClient( ... )
> pkg = 'my dataset'
> path = '/home/rgomes/tmp/my.csv'
> mimetype = 'text/csv'
> cli.add_package_resource(pkg, path, resource_type='data',
> mimetype=mimetype)
>
> Apparently it works fine.
>
> But in the web interface (CKAN website), the URL for the resource misses
> the CKAN site_url in it, for example:
>     URL:
> http://storage/f/2013-09-03T115539/Kolanovic-et-al.---Rise-of-Cross-Asset-Correlations.pdf
> ... whilst it should be:
>     URL: http://*ckan.localdomain:5000/*
> storage/f/2013-09-03T115539/Kolanovic-et-al.---Rise-of-Cross-Asset-Correlations.pdf<http://storage/f/2013-09-03T115539/Kolanovic-et-al.---Rise-of-Cross-Asset-Correlations.pdf>
>
> What would be a *recommended* way to fix this?
>
> By *recommended* I mean: best practices. Should I use DataStorer instead?
> Can it be considered mature enough?
>
> Thanks a lot :)
>
>
> Richard Gomes
> http://rgomes.info
> http://www.linkedin.com/in/rgomes
> mobile: +44(77)9955-6813
> inum <http://www.inum.net/>: +883(5100)0800-9804
>
>  On 30/08/13 16:45, Richard Gomes wrote:
>
> Hi Dave,
>
> Yes, I think it makes sense.
> At the moment I'm reinstalling everything into a VM, but next time I will
> try this.
>
> Cheers :)
>
> Richard Gomes
> http://rgomes.info
> http://www.linkedin.com/in/rgomes
> mobile: +44(77)9955-6813
> inum <http://www.inum.net/>: +883(5100)0800-9804
>
>  On 30/08/13 12:40, Dave Caraway wrote:
>
>  hi richard,
>
>  not sure if this helps, but i've seen ghost entries before due to solr
> being out of sync. i used paster command to reindex solr to fix.
>
>  -Dave
>
>
>
> On Aug 29, 2013, at 3:38 PM, Richard Gomes <rgomes.info at gmail.com> wrote:
>
>   hello,
>
> I loaded test data provided by CKAN into the database.
>
> I don't know if this is relavant... anyway, for your information:
> After that I did some tests using the FileStore API, when I associated
> documents to one of the datasets, like this:
>       1 base_location='http://ckan.localdomain:5000/api'
>       2 api_key= ...
>       3 http_user='rgomes'
>       4 http_pass=...
>       5
>       6
>       7 def filelist(folder='.'):
>       8     import os
>       9     for root, dir, files in os.walk(folder):
>      10         for file in files:
>      11             path = '%s/%s' % (root, file)
>      12             yield path
>      13     return
>      14
>      15
>      16 import ckanclient
>      17 cli = ckanclient.CkanClient(base_location=base_location,
> api_key=api_key, http_user=http_user, http_pass=http_pass, is_verbose=True)
>      18
>      19 pkg = 'warandpeace'
>      20 for path in filelist('/home/rgomes/tmp/upload'):
>      21     print path
>      22     print cli.add_package_resource(pkg, path)
>
>
> Then I've tried to recreate the database from scratch, like this:
>
> #!/bin/bash
>
> set -x
> sudo -u postgres dropdb ${CKAN_DATABASE}
> sudo -u postgres dropuser ${CKAN_USERNAME}
>
> sudo -u postgres createuser -S -D -R -P ${CKAN_USERNAME}
> sudo -u postgres createdb -O ${CKAN_USERNAME} ${CKAN_DATABASE} -E utf-8
>
> paster db init -c $CKAN_ETC/default/development.ini
> paster sysadmin add rgomes -c $CKAN_ETC/default/development.ini
> set +x
>
>
> Notice that I didn't loaded the test data after that!
>
>
> For my surprise, CKAN says that there are 2 datasets available.
> The datasets are not actually listed, but it says that there are 2
> datasets available.
> The faceted search tags associates 2 datasets to david and 1 to roger,
> etc... like if the datasets where still there.
>
> Any idea about how this sort of information survived a complete wipe out
> of database tables ?
>
> Thanks
>
> --
> Richard Gomes
> http://rgomes.info
> http://www.linkedin.com/in/rgomes
> mobile: +44(77)9955-6813
> inum <http://www.inum.net/>: +883(5100)0800-9804
>
>   _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
>
> _______________________________________________
> ckan-dev mailing listckan-dev at lists.okfn.orghttp://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>

-- 
Dr. Hendrik Bunke
http://gplus.to/hbunke
http://twitter.com/hbunke
http://www.hbxt.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130904/d6c446ee/attachment-0001.html>