[okfn-discuss] APIs and CKAN

Tue Dec 29 11:39:05 UTC 2009

2009/12/28 Rob Myers <rob at robmyers.org>:
> Heya.
>
> I've added some sites to CKAN that don't have data dumps but do have
> APIs. I've added the API urls as the download link, which I think is
> probably wrong, but I don't know where to put the API url to draw
> attention to it otherwise.

No, this is exactly right. From usage of CKAN so far it is already
clear that the appropriate "resource/reference" to associate to a
package of content or data is frequently an API not a simple dump
(this is most obviously the case for massive datasets where a 50GB
dump file isn't very useful or necessarily available).

I think this is one area where data/content is likely to obviously
differ from code (though it wouldn't be impossible for this to occur
with code too ...). In my mind I think of such packages as being
'virtual' in the sense that when you install them what you are getting
is a representation/access to data but that data may not reside on
your local machine.

> Would CKAN benefit from an API url field for entries, or does that go
> against the downloadable dataset spirit of CKAN?

It would definitely benefit from API url field entries. However, I
would note that I'm concerned when that is the *only* method of
getting the material since often it is impossible using an API alone
to get hold of the entire dataset.

I should also mentioned we've just changed from having a single
download url to allowing multiple download urls (see this email to
okfn-help [1]). This isn't yet deployed on ckan.net (though it will be
*very* soon) but you can see it in action on http://test.ckan.net/.

We're also associating a bit of metadata with these urls (a format +
description) and therefore terming them package 'resources'. One thing
we need to agree in the community is naming conventions for the format
field. For normal material we can use standard extension naming (or if
we're being really thorough mimetypes -- though who is going
application/vnd.ms-excel versus xls ...) but for apis we're in open
territory. I'd been thinking vaguely about something like api/{format}
but I'd be interested to have people's thoughts.

Regards,

Rufus

[1]: http://lists.okfn.org/pipermail/okfn-help/2009-November/000441.html
-- 
Promoting Open Knowledge in a Digital Age
http://www.okfn.org/ - http://blog.okfn.org/