[ckan-dev] Wrong URL after file upload
Richard Gomes
rgomes.info at gmail.com
Wed Sep 4 10:54:44 UTC 2013
Hello Hendrik,
Thanks a lot for your answer. Very clarifying!
I will try the version from github :)
Regarding ckanapi.... it only supports the Action API, correct? I mean:
FileStore is not directly supported and, in particular,
/add_package_resource/ is not supported, correct? I don't see the
Storage API defined properly and explicitly anywhere... am I missing
something?
For your information: at the moment, I'm using something similar to
ckanapi, with a hack which provides access to some methods from
CkanClient. The code can be seen here: http://paste.ubuntu.com/6062158/
This code supports early binding and code completion.
Thanks
Richard Gomes
http://rgomes.info
http://www.linkedin.com/in/rgomes
mobile: +44(77)9955-6813
inum <http://www.inum.net/>: +883(5100)0800-9804
On 04/09/13 09:17, Hendrik Bunke wrote:
> Hi Richard,
>
> you are most probably using an old version of ckanclient, perhaps the
> one from pypi. That version has several issues that have been fixed in
> the code at github. Please use this one https://github.com/okfn/ckanclient
>
> Furthermore, ckanclient is pretty much abondoned. It still uses the
> deprecated CKAN API 1 (REST). AFAIK the recommended python client
> would now be https://github.com/open-data/ckanapi.
>
> regards
> hendrik
>
>
> On 3 September 2013 14:02, Richard Gomes <rgomes.info at gmail.com
> <mailto:rgomes.info at gmail.com>> wrote:
>
> hello,
>
> I'm uploading files like this:
>
> cli = ckanclient.CkanClient( ... )
> pkg = 'my dataset'
> path = '/home/rgomes/tmp/my.csv'
> mimetype = 'text/csv'
> cli.add_package_resource(pkg, path, resource_type='data',
> mimetype=mimetype)
>
> Apparently it works fine.
>
> But in the web interface (CKAN website), the URL for the resource
> misses the CKAN site_url in it, for example:
> URL:
> http://storage/f/2013-09-03T115539/Kolanovic-et-al.---Rise-of-Cross-Asset-Correlations.pdf
> ... whilst it should be:
> URL:
> http://*ckan.localdomain:5000/*storage/f/2013-09-03T115539/Kolanovic-et-al.---Rise-of-Cross-Asset-Correlations.pdf
> <http://storage/f/2013-09-03T115539/Kolanovic-et-al.---Rise-of-Cross-Asset-Correlations.pdf>
>
> What would be a _/recommended/_ way to fix this?
>
> By /_recommended_/ I mean: best practices. Should I use DataStorer
> instead? Can it be considered mature enough?
>
> Thanks a lot :)
>
>
> Richard Gomes
> http://rgomes.info
> http://www.linkedin.com/in/rgomes
> mobile: +44(77)9955-6813 <tel:%2B44%2877%299955-6813>
> inum <http://www.inum.net/>: +883(5100)0800-9804
> <tel:%2B883%285100%290800-9804>
>
> On 30/08/13 16:45, Richard Gomes wrote:
>> Hi Dave,
>>
>> Yes, I think it makes sense.
>> At the moment I'm reinstalling everything into a VM, but next
>> time I will try this.
>>
>> Cheers :)
>>
>> Richard Gomes
>> http://rgomes.info
>> http://www.linkedin.com/in/rgomes
>> mobile: +44(77)9955-6813 <tel:%2B44%2877%299955-6813>
>> inum <http://www.inum.net/>: +883(5100)0800-9804
>> <tel:%2B883%285100%290800-9804>
>>
>> On 30/08/13 12:40, Dave Caraway wrote:
>>> hi richard,
>>>
>>> not sure if this helps, but i've seen ghost entries before due
>>> to solr being out of sync. i used paster command to reindex solr
>>> to fix.
>>>
>>> -Dave
>>>
>>>
>>>
>>> On Aug 29, 2013, at 3:38 PM, Richard Gomes
>>> <rgomes.info at gmail.com <mailto:rgomes.info at gmail.com>> wrote:
>>>
>>>> hello,
>>>>
>>>> I loaded test data provided by CKAN into the database.
>>>>
>>>> I don't know if this is relavant... anyway, for your information:
>>>> After that I did some tests using the FileStore API, when I
>>>> associated documents to one of the datasets, like this:
>>>> 1 base_location='http://ckan.localdomain:5000/api'
>>>> 2 api_key= ...
>>>> 3 http_user='rgomes'
>>>> 4 http_pass=...
>>>> 5
>>>> 6
>>>> 7 def filelist(folder='.'):
>>>> 8 import os
>>>> 9 for root, dir, files in os.walk(folder):
>>>> 10 for file in files:
>>>> 11 path = '%s/%s' % (root, file)
>>>> 12 yield path
>>>> 13 return
>>>> 14
>>>> 15
>>>> 16 import ckanclient
>>>> 17 cli =
>>>> ckanclient.CkanClient(base_location=base_location,
>>>> api_key=api_key, http_user=http_user, http_pass=http_pass,
>>>> is_verbose=True)
>>>> 18
>>>> 19 pkg = 'warandpeace'
>>>> 20 for path in filelist('/home/rgomes/tmp/upload'):
>>>> 21 print path
>>>> 22 print cli.add_package_resource(pkg, path)
>>>>
>>>>
>>>> Then I've tried to recreate the database from scratch, like this:
>>>>
>>>> #!/bin/bash
>>>>
>>>> set -x
>>>> sudo -u postgres dropdb ${CKAN_DATABASE}
>>>> sudo -u postgres dropuser ${CKAN_USERNAME}
>>>>
>>>> sudo -u postgres createuser -S -D -R -P ${CKAN_USERNAME}
>>>> sudo -u postgres createdb -O ${CKAN_USERNAME} ${CKAN_DATABASE}
>>>> -E utf-8
>>>>
>>>> paster db init -c $CKAN_ETC/default/development.ini
>>>> paster sysadmin add rgomes -c $CKAN_ETC/default/development.ini
>>>> set +x
>>>>
>>>>
>>>> Notice that I didn't loaded the test data after that!
>>>>
>>>>
>>>> For my surprise, CKAN says that there are 2 datasets available.
>>>> The datasets are not actually listed, but it says that there
>>>> are 2 datasets available.
>>>> The faceted search tags associates 2 datasets to david and 1 to
>>>> roger, etc... like if the datasets where still there.
>>>>
>>>> Any idea about how this sort of information survived a complete
>>>> wipe out of database tables ?
>>>>
>>>> Thanks
>>>>
>>>> --
>>>> Richard Gomes
>>>> http://rgomes.info
>>>> http://www.linkedin.com/in/rgomes
>>>> mobile: +44(77)9955-6813 <tel:%2B44%2877%299955-6813>
>>>> inum <http://www.inum.net/>: +883(5100)0800-9804
>>>> <tel:%2B883%285100%290800-9804>
>>>>
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org <mailto:ckan-dev at lists.okfn.org>
>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org <mailto:ckan-dev at lists.okfn.org>
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe:http://lists.okfn.org/mailman/options/ckan-dev
>>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org <mailto:ckan-dev at lists.okfn.org>
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
>
>
> --
> Dr. Hendrik Bunke
> http://gplus.to/hbunke
> http://twitter.com/hbunke
> http://www.hbxt.org
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130904/0a48776d/attachment-0001.html>
More information about the ckan-dev
mailing list