[ckan-dev] CKAN Features: Federation

Wed Apr 18 09:38:42 UTC 2012

On Tue, Apr 17, 2012 at 04:29:29PM +0100, Mark Wainwright wrote:
> Sticking my oar in again: I note that that page links to a section on
> 'harvesting' at <http://ckan.org/features/publish/#harvesting>. Are
> federating and harvesting the same thing? I thought they were, but ...

"Federation" is a general term for sharing content between different
websites (although I'm not sure how well known this term is).
"Harvesting" is our term for what a CKAN website does when it imports
datasets from another CKAN website or from a non-CKAN website. So
harvesting is one way of doing federation with CKAN. So they do more or
less mean the same thing, yes. (Thanks for explaining this to me, John).

To clarify: you can setup your CKAN website to automatically load
datasets and resources from another website, rather than you having to
input them yourself. You could have it 'harvesting' data from a bunch of
other sites as well as having its own manually-entered data.

CKAN does its harvesting using the ckanext-harvest extension, which also
apparently provides both a command-line interface and a web user
interface for managing what sources it harvests from. I've never tried
it myself, but maybe a screenshot of the web UI would be useful? I can
give it a look if you want.

https://github.com/okfn/ckanext-harvest/

I think the descriptions on those two ckan.org pages are quite good.

http://ckan.org/features/federate/
http://ckan.org/features/publish/#harvesting