[ckan-discuss] How is publicdata.eu website powered

[Mehdi El Fadil]

Hi Friedrich,

Thanks for your answers.

ckan.net and its api looks to be more usable to me to directly get data in
json format, but unfortunately the attributes that I am looking for such as
the places (spatial) seem to be missing.

What I am looking forward to create is an alternative web UI. In case
neither the reduced json or solr results are directly fetchable, then I
should manage to extract data from n3 format.

Thanks again,

mehdi

On Mon, May 16, 2011 at 5:58 PM, Friedrich Lindenberg <
friedrich.lindenberg at okfn.org> wrote:

> Hi Mehdi,
>
> On Mon, May 16, 2011 at 4:59 PM, Mehdi El Fadil <
> mehdi.elfadil at mango-is.com> wrote:
>
>> I am currently studying options for an open data catalog project, and
>> would be interested in re-using some of the data from
>> http://publicdata.eu website.
>>
>
> To clarify: publicdata.eu was a four-day development spike to evaluate the
> feasibility of data catalogue harvesting. While its a nice demo, we will
> soon merge it into CKAN which now has similar data harvesting capabilities.
>
>
>> I have noticed the "data" link at the bottom of the site, and also the
>> code at bitbucket. I am not so familiar with semantic web, and I am
>> wondering whether opendata.eu is powered by on-the-fly parsing of the n3
>> files by some search engine, or whether there exists a database storing each
>> site according to place, creator and file type. Also, are the data only
>> available under n3 format, or is it possible to extract it in any other
>> format (json, csv ..)?
>>
>
> publicdata.eu in its current form consists of two stages: data catalogues
> are harvested (scraped, often enough) and written out to n3 RDF at the
> "data" location. These files are then read by an indexer which flattens them
> into something like a reduced JSON form and sends them to Solr for indexing.
> The web UI is then only a thin layer around functionality offered by solr.
>
> Once moved to solr, the basic setup will remain very similar but we'll have
> management services for the harvesters and the intermediary phase will be
> the CKAN database - which can then be exported to RDF or JSON.
>
> Hope this helps?
>
> - Friedrich
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-discuss/attachments/20110516/902df113/attachment.htm>