[ckan-dev] CKAN API Clients

Tue Apr 24 15:11:02 UTC 2012

On 4/24/12 5:57 AM, "Sean Hammond" <sean.hammond at okfn.org> wrote:

>> >> Ideally I'd like to use the Python client which is maintained by
>> >> the
>> >>core team. I can download the source pypi but I cannot find the
>> >>source on github  can someone point me to it?
>> >
>> >https://github.com/okfn/ckanclient
>> >
>> >> Upon inspecting the source code, it appears that this client supports
>> >>API v1
>> >> and v2 but not v3.
>> >
>> >That is correct.
>> 
>> Thanks for the link. I've started adding Action API functions to the
>> client library.
>
>Great!
>
>> Along the way I discovered that the CKAN server only returns 'id'
>> references for api ver. 2.
>> So I modified a few files in logic/action to return 'id' references when
>> api version is 2 or 3.
>> The pull request has been submitted. I believe it is a pretty simple
>> change.
>
>That's right, APIv3 returns object names rather than IDs when you do
>e.g. package_list or user_list etc. If you use the *_show methods you
>can get the full details of an object including the ID. I'm not sure
>whether this is by design or not, so not sure what we should do with
>your pull request.

According to the documentation [1], package_show expects 'id' as an
argument.
But the Action API does not offer any methods that return a package 'id'.
There seems to be a discrepancy here.

My interpretation of the Action API documentation [1] was that functions
should return 'id' references whenever possible. Hence the patch. What's
the best way to reconcile the discrepancy?

[1] http://docs.ckan.org/en/latest/apiv3.html

>
>> Upon closer inspection of the CKAN server code in
>> logic/action/create.py:resource_create() it turns out that this function
>> is only partially implemented.
>> 
>> I will need some guidance about how to flesh it out fully to support my
>> use case.
>
>I don't know about uploading files, but you can add new resources to
>packages by using the package_update API call. I'm not quite sure why
>the incomplete resource_create is intended for, to be honest.
>
>> Moreover, in my use case, not only does a new resource need to be
>>created
>> by the CKAN client api, but the resource actually needs to reside on the
>> CKAN server. I.e. It will need to be uploaded to the filestore. The data
>> model of a resource only has a 'url' field (presumably for linked
>> resources) which suggests to me that support for uploading files may not
>> be a trivial undertaking. Perhaps someone can dispel my hunch?
>
>I've not looked into this part of CKAN before, but I think the place to
>start would be these chapters of the docs:
>
>http://docs.ckan.org/en/latest/filestore.html

Ok, I read and re-read the filestore documentation. The following snippet
from [2]:

"Provides credentials for doing operations on storage directly from a
client
(using web form style POSTs)."

Suggests that it should be possible to upload a file. However there aren't
many details and no working examples.

The ckanclient lib also has some methods related to the storage api.
However they too lack any documentation or examples.
The test classes in [3] also don't seem to test file uploading.

[2] http://docs.ckan.org/en/latest/filestore.html#form-authentication

[3] 
https://github.com/okfn/ckan/blob/master/ckan/tests/functional/test_storage
.py

>
>http://docs.ckan.org/en/latest/datastore.html#datastorer

>
>> Lastly, I have some ideas about refactoring the code in
>> ckanclient/__init__.py to make it more succinct. Mainly it involves
>>using
>> a metaclass to create api functions automatically:
>> 
>> class Meta (type):
>> 	def __new__ (self):
>> 		If self._api_methods_ is not None:
>> 			# process the items to create new instance attributes on the fly
>> 
>> class Client2 (Api):
>> 	__meta__ = Meta
>>         _api_methods_ = { 'site_read' : ('GET', []),
>> 			  'package_show' : ('GET', ['id']),
>> 	                  # and so on...
>> 			}
>> 
>> I think this approach can significantly reduce the amount of redundant
>> code in ckanclient/__init__.py. I will probably undertake this in a new
>> file inside the ckanclient folder. Let me know if there are any
>> suggestions/objections to this approach.
>
>I tend to shy away from too much metaclass magic and other non-obvious
>use of advanced Python features, preferring to keep the code explicit
>and obvious. But, if it gets rid of a lot of boiler plate that's a good
>thing. I think I'd have to see a diff to say more about it.

Agreed.

However, it sounds like before I undertake any refactoring we should
improve the documentation by:
 - fixing discrepencies.
 - adding concrete usage examples.

Thanks,
Salman

>
>_______________________________________________
>ckan-dev mailing list
>ckan-dev at lists.okfn.org
>http://lists.okfn.org/mailman/listinfo/ckan-dev