[ckan-discuss] How is publicdata.eu website powered

Tue May 17 11:23:00 BST 2011

Medhi,

> unfortunately the attributes that I am looking for such as
> the places (spatial) seem to be missing.

The fields in CKAN are flexible. For example, some people have used
their CKAN instances to describe the location of the data by country,
by administrative area, and by lat/long polygon.

Searching by the field is the trickier side of things and requires
something beyond the basic search in core CKAN. We have the beginnings
of rather advanced PostGIS solution. Do ask Adria if that interests
you.

David

On 16 May 2011 22:34, Mehdi El Fadil <mehdi.elfadil at mango-is.com> wrote:
> Hi Friedrich,
>
> Thanks for your answers.
>
> ckan.net and its api looks to be more usable to me to directly get data in
> json format, but unfortunately the attributes that I am looking for such as
> the places (spatial) seem to be missing.
>
> What I am looking forward to create is an alternative web UI. In case
> neither the reduced json or solr results are directly fetchable, then I
> should manage to extract data from n3 format.
>
> Thanks again,
>
> mehdi
>
> On Mon, May 16, 2011 at 5:58 PM, Friedrich Lindenberg
> <friedrich.lindenberg at okfn.org> wrote:
>>
>> Hi Mehdi,
>>
>> On Mon, May 16, 2011 at 4:59 PM, Mehdi El Fadil
>> <mehdi.elfadil at mango-is.com> wrote:
>>>
>>> I am currently studying options for an open data catalog project, and
>>> would be interested in re-using some of the data from http://publicdata.eu
>>> website.
>>
>> To clarify: publicdata.eu was a four-day development spike to evaluate the
>> feasibility of data catalogue harvesting. While its a nice demo, we will
>> soon merge it into CKAN which now has similar data harvesting capabilities.
>>
>>>
>>> I have noticed the "data" link at the bottom of the site, and also the
>>> code at bitbucket. I am not so familiar with semantic web, and I am
>>> wondering whether opendata.eu is powered by on-the-fly parsing of the n3
>>> files by some search engine, or whether there exists a database storing each
>>> site according to place, creator and file type. Also, are the data only
>>> available under n3 format, or is it possible to extract it in any other
>>> format (json, csv ..)?
>>
>> publicdata.eu in its current form consists of two stages: data catalogues
>> are harvested (scraped, often enough) and written out to n3 RDF at the
>> "data" location. These files are then read by an indexer which flattens them
>> into something like a reduced JSON form and sends them to Solr for indexing.
>> The web UI is then only a thin layer around functionality offered by solr.
>> Once moved to solr, the basic setup will remain very similar but we'll
>> have management services for the harvesters and the intermediary phase will
>> be the CKAN database - which can then be exported to RDF or JSON.
>> Hope this helps?
>> - Friedrich
>
> _______________________________________________
> ckan-discuss mailing list
> ckan-discuss at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-discuss
>
>