[ckan-dev] Wrong URL after file upload

Tue Sep 3 12:02:07 UTC 2013

hello,

I'm uploading files like this:

    cli = ckanclient.CkanClient( ... )
    pkg = 'my dataset'
    path = '/home/rgomes/tmp/my.csv'
    mimetype = 'text/csv'
    cli.add_package_resource(pkg, path, resource_type='data',
    mimetype=mimetype)

Apparently it works fine.

But in the web interface (CKAN website), the URL for the resource misses 
the CKAN site_url in it, for example:
     URL: 
http://storage/f/2013-09-03T115539/Kolanovic-et-al.---Rise-of-Cross-Asset-Correlations.pdf
... whilst it should be:
     URL: 
http://*ckan.localdomain:5000/*storage/f/2013-09-03T115539/Kolanovic-et-al.---Rise-of-Cross-Asset-Correlations.pdf 
<http://storage/f/2013-09-03T115539/Kolanovic-et-al.---Rise-of-Cross-Asset-Correlations.pdf>

What would be a _/recommended/_ way to fix this?

By /_recommended_/ I mean: best practices. Should I use DataStorer 
instead? Can it be considered mature enough?

Thanks a lot :)

Richard Gomes
http://rgomes.info
http://www.linkedin.com/in/rgomes
mobile: +44(77)9955-6813
inum <http://www.inum.net/>: +883(5100)0800-9804

On 30/08/13 16:45, Richard Gomes wrote:
> Hi Dave,
>
> Yes, I think it makes sense.
> At the moment I'm reinstalling everything into a VM, but next time I 
> will try this.
>
> Cheers :)
>
> Richard Gomes
> http://rgomes.info
> http://www.linkedin.com/in/rgomes
> mobile: +44(77)9955-6813
> inum <http://www.inum.net/>: +883(5100)0800-9804
>
> On 30/08/13 12:40, Dave Caraway wrote:
>> hi richard,
>>
>> not sure if this helps, but i've seen ghost entries before due to 
>> solr being out of sync. i used paster command to reindex solr to fix.
>>
>> -Dave
>>
>>
>>
>> On Aug 29, 2013, at 3:38 PM, Richard Gomes <rgomes.info at gmail.com 
>> <mailto:rgomes.info at gmail.com>> wrote:
>>
>>> hello,
>>>
>>> I loaded test data provided by CKAN into the database.
>>>
>>> I don't know if this is relavant... anyway, for your information:
>>> After that I did some tests using the FileStore API, when I 
>>> associated documents to one of the datasets, like this:
>>>       1 base_location='http://ckan.localdomain:5000/api'
>>>       2 api_key= ...
>>>       3 http_user='rgomes'
>>>       4 http_pass=...
>>>       5
>>>       6
>>>       7 def filelist(folder='.'):
>>>       8     import os
>>>       9     for root, dir, files in os.walk(folder):
>>>      10         for file in files:
>>>      11             path = '%s/%s' % (root, file)
>>>      12             yield path
>>>      13     return
>>>      14
>>>      15
>>>      16 import ckanclient
>>>      17 cli = ckanclient.CkanClient(base_location=base_location, 
>>> api_key=api_key, http_user=http_user, http_pass=http_pass, 
>>> is_verbose=True)
>>>      18
>>>      19 pkg = 'warandpeace'
>>>      20 for path in filelist('/home/rgomes/tmp/upload'):
>>>      21     print path
>>>      22     print cli.add_package_resource(pkg, path)
>>>
>>>
>>> Then I've tried to recreate the database from scratch, like this:
>>>
>>> #!/bin/bash
>>>
>>> set -x
>>> sudo -u postgres dropdb ${CKAN_DATABASE}
>>> sudo -u postgres dropuser ${CKAN_USERNAME}
>>>
>>> sudo -u postgres createuser -S -D -R -P ${CKAN_USERNAME}
>>> sudo -u postgres createdb -O ${CKAN_USERNAME} ${CKAN_DATABASE} -E utf-8
>>>
>>> paster db init -c $CKAN_ETC/default/development.ini
>>> paster sysadmin add rgomes -c $CKAN_ETC/default/development.ini
>>> set +x
>>>
>>>
>>> Notice that I didn't loaded the test data after that!
>>>
>>>
>>> For my surprise, CKAN says that there are 2 datasets available.
>>> The datasets are not actually listed, but it says that there are 2 
>>> datasets available.
>>> The faceted search tags associates 2 datasets to david and 1 to 
>>> roger, etc... like if the datasets where still there.
>>>
>>> Any idea about how this sort of information survived a complete wipe 
>>> out of database tables ?
>>>
>>> Thanks
>>>
>>> -- 
>>> Richard Gomes
>>> http://rgomes.info
>>> http://www.linkedin.com/in/rgomes
>>> mobile: +44(77)9955-6813
>>> inum <http://www.inum.net/>: +883(5100)0800-9804
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org <mailto:ckan-dev at lists.okfn.org>
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe:http://lists.okfn.org/mailman/options/ckan-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130903/3dc62f85/attachment.html>