[ckan-dev] CKAN API Clients

Tue Apr 24 02:14:18 UTC 2012

On 4/19/12 6:16 PM, "Rufus Pollock" <rufus.pollock at okfn.org> wrote:

>On 19 April 2012 17:22, Haq, Salman <Salman.Haq at neustar.biz> wrote:
>> Hi,
>>
>> I want to programmatically create datasets and upload resources to my
>>CKAN
>> 1.6 instance.
>>
>> Looking at the docs.ckan.org, it seems that there are several client
>> libraries available and at least three API versions.
>>
>> From the documentation, it's not clear to me which which clients support
>> which API versions. I assume this is tribal knowledge, but can it be
>> tabulated in the documentation?
>
>The only guaranteed up to date one is the Python one as that is core
>maintained so with other libs YMMV. However, I note we have not made
>breaking changes to RESTFul API for several versions so old stuff
>(<1y) should work quite well. I note no client supports the action API
>as yet.
>
>> Ideally I'd like to use the Python client which is maintained by the
>>core
>> team. I can download the source pypi but I cannot find the source on
>>github
>>  can someone point me to it?
>
>https://github.com/okfn/ckanclient
>
>> Upon inspecting the source code, it appears that this client supports
>>API v1
>> and v2 but not v3.
>
>That is correct.

Thanks for the link. I've started adding Action API functions to the
client library.

Along the way I discovered that the CKAN server only returns 'id'
references for api ver. 2.
So I modified a few files in logic/action to return 'id' references when
api version is 2 or 3.
The pull request has been submitted. I believe it is a pretty simple
change.

>
>> My cursory understanding is that version 3 of the API is what I will
>>need to
>> use to accomplish my tasks. Am I correct in assuming that I will have to
>> extend the Python client?
>
>That is correct -- patches will be very welcome and we will be looking
>to add action API support. I should also say JSON web API's are
>trivial to use directly, for example here's a simple python function
>do a request with any method and any data to specified URL
>
>    def _request(self, url, data, method):
>        opener = urllib2.build_opener(urllib2.HTTPHandler)
>        request = urllib2.Request(url, data, self._headers)
>        request.get_method = lambda: method
>        response = opener.open(request)
>        return response.read()

Upon closer inspection of the CKAN server code in
logic/action/create.py:resource_create() it turns out that this function
is only partially implemented.

I will need some guidance about how to flesh it out fully to support my
use case.

Moreover, in my use case, not only does a new resource need to be created
by the CKAN client api, but the resource actually needs to reside on the
CKAN server. I.e. It will need to be uploaded to the filestore. The data
model of a resource only has a 'url' field (presumably for linked
resources) which suggests to me that support for uploading files may not
be a trivial undertaking. Perhaps someone can dispel my hunch?

Lastly, I have some ideas about refactoring the code in
ckanclient/__init__.py to make it more succinct. Mainly it involves using
a metaclass to create api functions automatically:

class Meta (type):
	def __new__ (self):
		If self._api_methods_ is not None:
			# process the items to create new instance attributes on the fly

class Client2 (Api):
	__meta__ = Meta
        _api_methods_ = { 'site_read' : ('GET', []),
			  'package_show' : ('GET', ['id']),
	                  # and so on...
			}

I think this approach can significantly reduce the amount of redundant
code in ckanclient/__init__.py. I will probably undertake this in a new
file inside the ckanclient folder. Let me know if there are any
suggestions/objections to this approach.

Thanks,
Salman

>
>Regards,
>
>Rufus
>
>_______________________________________________
>ckan-dev mailing list
>ckan-dev at lists.okfn.org
>http://lists.okfn.org/mailman/listinfo/ckan-dev