[ckan-discuss] How is publicdata.eu website powered

Tue May 17 13:03:48 BST 2011

Hi Mehdi,

If I understand correctly you are interested in building an
alternative web UI for CKAN (esp for geo data). I think this is a
great thing to do.

If you are doing this you may be interested in some existing work we
did to build a CKAN js lib and, based on it, a pure javascript based
frontend:

<http://bitbucket.org/okfn/ckanjs>

Demo:

<http://dev.okfn.org/ckanjs/app/>

This could be very easily adapted and customized (e.g. we started work
but did not complete work on adding geodata lookup using geonames
based on a location field).

Rufus

2011/5/17 Mehdi El Fadil <mehdi.elfadil at mango-is.com>:
> Hi,
> For this step of the project (I am just building a proof-of-concept) I
> should manage without the ckanet-spatial. I might use it later on though.
> Mehdi
>
> 2011/5/17 Adrià Mercader <amercadero at gmail.com>
>>
>> As Friedrich and David pointed out, you pretty much choose what to put
>> in the package fields (extras).
>> The ckanext-spatial [1] extension uses bounding box information stored
>> on the package extras and stores it on a PostGIS enabled table.
>> You can later perform a simple bounding box API query like:
>>
>> /api/2/search/package/geo?bbox={minx,miny,maxx,maxy}[&crs={srid}]
>>
>> to get the packages that intersect.
>>
>> Do let me know if you are interested in this.
>>
>>
>> Adrià
>>
>> [1] https://bitbucket.org/okfn/ckanext-spatial
>>
>>
>> 2011/5/17 David Read <david.read at okfn.org>:
>> > Medhi,
>> >
>> >> unfortunately the attributes that I am looking for such as
>> >> the places (spatial) seem to be missing.
>> >
>> > The fields in CKAN are flexible. For example, some people have used
>> > their CKAN instances to describe the location of the data by country,
>> > by administrative area, and by lat/long polygon.
>> >
>> > Searching by the field is the trickier side of things and requires
>> > something beyond the basic search in core CKAN. We have the beginnings
>> > of rather advanced PostGIS solution. Do ask Adria if that interests
>> > you.
>> >
>> > David
>> >
>> > On 16 May 2011 22:34, Mehdi El Fadil <mehdi.elfadil at mango-is.com> wrote:
>> >> Hi Friedrich,
>> >>
>> >> Thanks for your answers.
>> >>
>> >> ckan.net and its api looks to be more usable to me to directly get data
>> >> in
>> >> json format, but unfortunately the attributes that I am looking for
>> >> such as
>> >> the places (spatial) seem to be missing.
>> >>
>> >> What I am looking forward to create is an alternative web UI. In case
>> >> neither the reduced json or solr results are directly fetchable, then I
>> >> should manage to extract data from n3 format.
>> >>
>> >> Thanks again,
>> >>
>> >> mehdi
>> >>
>> >> On Mon, May 16, 2011 at 5:58 PM, Friedrich Lindenberg
>> >> <friedrich.lindenberg at okfn.org> wrote:
>> >>>
>> >>> Hi Mehdi,
>> >>>
>> >>> On Mon, May 16, 2011 at 4:59 PM, Mehdi El Fadil
>> >>> <mehdi.elfadil at mango-is.com> wrote:
>> >>>>
>> >>>> I am currently studying options for an open data catalog project, and
>> >>>> would be interested in re-using some of the data from
>> >>>> http://publicdata.eu
>> >>>> website.
>> >>>
>> >>> To clarify: publicdata.eu was a four-day development spike to evaluate
>> >>> the
>> >>> feasibility of data catalogue harvesting. While its a nice demo, we
>> >>> will
>> >>> soon merge it into CKAN which now has similar data harvesting
>> >>> capabilities.
>> >>>
>> >>>>
>> >>>> I have noticed the "data" link at the bottom of the site, and also
>> >>>> the
>> >>>> code at bitbucket. I am not so familiar with semantic web, and I am
>> >>>> wondering whether opendata.eu is powered by on-the-fly parsing of the
>> >>>> n3
>> >>>> files by some search engine, or whether there exists a database
>> >>>> storing each
>> >>>> site according to place, creator and file type. Also, are the data
>> >>>> only
>> >>>> available under n3 format, or is it possible to extract it in any
>> >>>> other
>> >>>> format (json, csv ..)?
>> >>>
>> >>> publicdata.eu in its current form consists of two stages: data
>> >>> catalogues
>> >>> are harvested (scraped, often enough) and written out to n3 RDF at the
>> >>> "data" location. These files are then read by an indexer which
>> >>> flattens them
>> >>> into something like a reduced JSON form and sends them to Solr for
>> >>> indexing.
>> >>> The web UI is then only a thin layer around functionality offered by
>> >>> solr.
>> >>> Once moved to solr, the basic setup will remain very similar but we'll
>> >>> have management services for the harvesters and the intermediary phase
>> >>> will
>> >>> be the CKAN database - which can then be exported to RDF or JSON.
>> >>> Hope this helps?
>> >>> - Friedrich
>> >>
>> >> _______________________________________________
>> >> ckan-discuss mailing list
>> >> ckan-discuss at lists.okfn.org
>> >> http://lists.okfn.org/mailman/listinfo/ckan-discuss
>> >>
>> >>
>> >
>> > _______________________________________________
>> > ckan-discuss mailing list
>> > ckan-discuss at lists.okfn.org
>> > http://lists.okfn.org/mailman/listinfo/ckan-discuss
>> >
>
> _______________________________________________
> ckan-discuss mailing list
> ckan-discuss at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-discuss
>
>

-- 
Co-Founder, Open Knowledge Foundation
Promoting Open Knowledge in a Digital Age
http://www.okfn.org/ - http://blog.okfn.org/