[okfn-help] [datacatalog] Interoperability between National Data Catalog and CKAN?

Sat Feb 6 14:11:55 GMT 2010

Many thanks for this, David!

I've forwarded this to CKAN developers on our okfn-help list, please
feel free to carbon copy to your list if useful!

Jonathan

On Thu, Jan 21, 2010 at 10:00 PM, David James
<djames at sunlightfoundation.com> wrote:
> Sorry for the delay in responding...
>
>> Do you have any technical information about the National Data Catalog
>> we should see?
>
> There is documentation available in the /doc folder of the API source
> code. The National Data Catalog (NatDatCat) is open source and
> available via GitHub. NatDatCat consists of several parts:
> * the API: http://github.com/sunlightlabs/datacatalog-api
> * the Web front end (which consumes the API):
> http://github.com/sunlightlabs/datacatalog-web
> * a Ruby wrapper library: http://github.com/sunlightlabs/ruby-datacatalog
> * a set of importers (currently being heavily refactored)
>
>> Is it versioned?
>
> Depends what you mean...
> * "Is the source code versioned?" Yes.
> * "Is the imported data versioned?" Yes.
>
>> What metadata do you include?
>
> The importers currently pull data from data.gov and data.octo.dc.gov.
> We pull in the metadata from each site and mesh it together.
>
>>What language are you using?
>
> The site is targeted at data sets in the United States, so it is English-based.
>
>> Publicly editable?
>
> The NatDatCat API is RESTful read/write API. Anyone can get a read
> key. Members of the public can also comment and rate data sets, among
> other things, using the Web interface at
> http://nationaldatacatalog.com.
>
>> I think the main thing would be to make sure that we can push/pull
>> data between CKAN and the National Data Catalog software!
>
> The next level, for those interested, would be to get involved in
> uploading data sets or curation. Those who want a write key should
> talk to us and we can talk more about how that can work. We have a
> sandbox API setup if you want to get started. Just let us know.
>
>> We are
>> currently thinking about how we can federate data from different
>> instances of CKAN. Have you had any thoughts along these lines?
>
> Our REST architecture allows for various tools (importers,
> visualizers, whatever) to be totally decentralized from the rest of
> the pieces of the system. The pieces only have to share an
> understanding of how to use the NatDatCat API. For example, an
> importer for one data catalog (e.g. CKAN) can run on one machine and
> connect to the NatDatCat API.
>
> Feel free to ask more questions anytime.
>
> Do you mind if I CC the list with our thread? I'd like to share the
> information as broadly as possible.
>
> -David
>
> On Sun, Jan 10, 2010 at 11:16 AM, Jonathan Gray <jonathan.gray at okfn.org> wrote:
>> Dear Luigi,
>>
>> Yes exactly - the URL below should give you some starting info. In
>> addition, see:
>>
>>  http://knowledgeforge.net/ckan/trac
>>  http://knowledgeforge.net/ckan/svn/
>>  http://www.knowledgeforge.net/project/ckan/
>>
>> I think the main thing would be to make sure that we can push/pull
>> data between CKAN and the National Data Catalog software! We are
>> currently thinking about how we can federate data from different
>> instances of CKAN. Have you had any thoughts along these lines?
>>
>> Do you have any technical information about the National Data Catalog
>> we should see? Is it versioned? Publicly editable? What metadata do
>> you include? What language are you using?
>>
>> Best wishes,
>>
>> Jonathan
>>
>> On Wed, Dec 23, 2009 at 2:10 AM, Luigi Montanez
>> <lmontanez at sunlightfoundation.com> wrote:
>>> Hi folks,
>>> David James and I are co-leads for this project. We've been on vacation (and
>>> will be until Jan. 4), but we're happy to trade messages on CKAN.
>>> I assume this is the best place to start?
>>> http://www.ckan.net/about/
>>> Thanks,
>>> Luigi
>>> On Thu, Dec 17, 2009 at 8:05 AM, Jonathan Gray <jonathan.gray at okfn.org>
>>> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I've posted to this list in the past about CKAN, an open source
>>>> registry for open data which has been going since early 2007. It is
>>>> currently being used as back end for data.gov.uk and we are porting it
>>>> for use in other jurisdictions as a community-driven registry for open
>>>> government data.
>>>>
>>>> It is fully versioned, and supports arbitrary domain specific metadata
>>>> in addition to core metadata fields. This means that it is both very
>>>> flexible, but also allows for more sophisticated querying. We are
>>>> currently thinking about providing plugins for other open source
>>>> software packages (Wordpress, Drupal, etc.) as well as thinking about
>>>> distributed infrastructure for storing/mirroring open data.
>>>>
>>>> It would be great to talk to someone from National Data Catalog team
>>>> at some point to discuss how we can make the two projects
>>>> interoperable!
>>>>
>>>> --
>>>> Jonathan Gray
>>>>
>>>> Community Coordinator
>>>> The Open Knowledge Foundation
>>>> http://www.okfn.org
>>>>
>>>> --
>>>>
>>>> You received this message because you are subscribed to the Google Groups
>>>> "datacatalog" group.
>>>> To post to this group, send email to datacatalog at googlegroups.com.
>>>> To unsubscribe from this group, send email to
>>>> datacatalog+unsubscribe at googlegroups.com.
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/datacatalog?hl=en.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Luigi Montanez
>>> 202-683-7401
>>> Skype+Twitter: LuigiMontanez
>>>
>>
>>
>>
>> --
>> Jonathan Gray
>>
>> Community Coordinator
>> The Open Knowledge Foundation
>> http://www.okfn.org
>>
>

-- 
Jonathan Gray

Community Coordinator
The Open Knowledge Foundation
http://blog.okfn.org

Twitter/Identica: jwyg