[ckan-dev] CKAN API Clients

Wed Apr 25 17:52:44 UTC 2012

On 4/25/12 12:16 PM, "Haq, Salman" <Salman.Haq at neustar.biz> wrote:

>
>
>On 4/25/12 12:09 PM, "Sean Hammond" <sean.hammond at okfn.org> wrote:
>
>>> > Ok, I read and re-read the filestore documentation. The following
>>> > snippet from [2]:
>>> > 
>>> > "Provides credentials for doing operations on storage directly from a
>>> > client (using web form style POSTs)."
>>> > 
>>> > Suggests that it should be possible to upload a file. However there
>>> > aren't many details and no working examples.
>>> > 
>>> > The ckanclient lib also has some methods related to the storage api.
>>> > However they too lack any documentation or examples.  The test
>>>classes
>>> > in [3] also don't seem to test file uploading.
>>> 
>>> Hmm. I'll take a look into this soon and see if I can figure it out and
>>> maybe improve the docs
>>
>>On looking into this, it does look as if there's some code in CKAN for
>>uploading files to the filestore via the API, but this code isn't
>>documented or tested very well, so I'd consider it unsupported for now.
>>I recommend you go forward with the side-loading via scp/rsync approach
>>that we discussed.
>>
>>A number of people have asked about uploading to the filestore via the
>>api recently, so we may allocate some time to get this code fixed up
>>with proper tests and docs at some point in the next few weeks.
>
>Sean,
>
>Thanks for looking into this. I was just about to write to you to report
>that I've figured out how to use the existing API to upload a file.
>
>In ckanclient.loaders, I'm going to add a 'util' module which will have a
>couple of functions to upload a file and add a resource to a package. Let
>me know if the location is okay.
>
>I'll follow-up with more info shortly.

Here is a set of bugs I found and fixed in ckanclient/__init__.py:

http://git.io/2AsAOQ

And this is a the new ckanclient.loader.util module:
http://git.io/zYk8NA

'upload_file' and 'add_package_resource' are two high-level utility
functions it defines.

To actually upload the file, we have to perform a multi-part form upload
which urllib2 does not handle by default so I've used httplib for that
purpose. The 'poster' module has a special urllib2 opener for doing
multi-part uploads but I didn't want to introduce a new dependency.

Test cases to follow soon.

Thanks,
Salman

>
>Thanks,
>Salman
>
>
>>
>>_______________________________________________
>>ckan-dev mailing list
>>ckan-dev at lists.okfn.org
>>http://lists.okfn.org/mailman/listinfo/ckan-dev
>
>
>_______________________________________________
>ckan-dev mailing list
>ckan-dev at lists.okfn.org
>http://lists.okfn.org/mailman/listinfo/ckan-dev