[okfn-labs] [okfn-discuss] OKFN & Environment Data

Mon Oct 15 09:33:24 UTC 2012

Hi all,

I am very interested in opening up environment data, specifically air
quality data. It's great to see it moves around the OKFN, so count me in if
a specific list or workgroup (through the existing open climate group or
other mechanisms).

In Spain accessing air quality data is quite tedious. I collected
information on the situation in other countries like UK and in general,
from what I've been able to explore, accessing information on air quality
in general is possible through the web of autonomous regions and
municipalities in Spain, and official sites in countries through specific
websites  for UK. Also, there exists an european database, AirBase [2],
which provides information until 2011 at the European level, and some
interfaces that allow reuse this data from other applications, mainly web
map services. These interfaces are listed on The Data Hub [3].

The problems that I have appreciated with this information are mainly:

* I didn't find an easy access for updated information in near real time,
although it is published on websites, it is seems it is not available
through mechanisms that promote reuse (an API or direct links to raw data).
* The format of the data is not homogeneous, for example for the city of
Madrid, the data format for the validated information is different from the
pre-validated data and in turn different from that offered by other
regions. There is also disparity in formats between countries. This makes
it difficult to consistently treat the information independently of the
provider  (municipality, region, country or European source). Other
standard formats that can be applied, as DEM, XML or NetCDF are generally
too complex for citizens which are mainly familiar with CSV or XLS. JSON or
GeoJSON are also more suitable for web/mobile developers.
* Although the information is accessible via the Internet, often it
requires manual intervention (fill forms, receive data in email, etc..)
that makes it more difficult to be treated, although it can always be
approached using scraping techniques.
* Proprietary technologies (like Microsoft file formats or software ESRI)
are used to support this information. There are open source alternatives.
* In general, the information available is technical oriented rather than
for general reuse.

Talking about this issue with some scientists and organizations working
with these data, they tell me that usually they get the data using the
websites of the air quality agencies, sometimes scraping, and in some cases
they contact them directly in order to recieve data in machine-readable
format.

In my opinion, there is much room for improvement in this area to
facilitate accessing this information not only to be used by experts but
making it easy for general use applying the open data principles and the
open source and latest Internet technologies.

As proof of concept of reuse of the EEA data, here you can see the european
air quality network on the open source tool CartoDB:

* https://fpedrera.cartodb.com/tables/airbase_v6_stations/embed_map.
(Classified by type of station: Background, Traffic, Industrial).

The information can be queried also very easily through its SQL API:

*
http://fpedrera.cartodb.com/api/v2/sql?q=SELECT%20*%20FROM%20airbase_v6_stations%20where%20station_city=%27MADRID%27%20and%20type_of_station=%27Background%27

I am also a bit involved in the AirQualityEgg [4] initiative through the
Internet of Things Madrid group [5]. It's quite similar to that mentioned
by Rafael Pezzi, and the data from the sensors will be pushed to COSM,
although there are maybe better alternatives for open publication.

Rafael, maybe you can put an eye on this project since some issues you'll
face are maybe solved or at least discussed. Feel free to contact me if you
want more information.

Kind regards,

--
Félix Pedrera
Twitter > @fpedrera

[1] http://uk-air.defra.gov.uk/
[2] http://www.eea.europa.eu/themes/air/airbase
[3] http://thedatahub.org/es/dataset/discomap
[4] http://airqualityegg.wikispaces.com/AirQualityEgg
[5] http://www.meetup.com/iotmadrid/

2012/10/12 Velichka Dimitrova <velichka.dimitrova at okfn.org>

> Hi all,
>
> Very happy to hear about this conversation getting started.
>
> Indeed we planned to have a sprint about energy this fall -
> http://sprints.okfnlabs.org/energy/ - which we have postponed until the
> beginning of next year. We are looking into clearly defining a topic what
> we are going to work on, preparing the data and background research in
> advance.
>
> A revived Open Climate Science working group as well as social scientists
> with environmental focus from the Open Economics working group<http://openeconomics.net/> could
> assist in this.
>
> Please let us know if:
>
>    - If you have any ideas about a topic, which could be addressed in the
>    sprint or
>    - you would like to participate in the current discussion around the
>    theme of energy subsidies
>
> Looking forward to hear from you!
>
> Velichka
>
>
>
> On Fri, Oct 12, 2012 at 12:31 PM, James Smith <james at floppy.org.uk> wrote:
>
>> Hi,
>>
>> On Friday, 12 October 2012 at 12:23, Jonathan Gray wrote:
>>
>> On Fri, Oct 12, 2012 at 1:18 PM, James Smith <james at floppy.org.uk> wrote:
>>
>>  I can't speak for AMEE (any more), but this is unlikely to come from
>> within the company. Adoption of the data by external organisations
>> interested in openness is the only way it's likely to happen at this point,
>> unfortunately.
>>
>>
>> Do you think that AMEE might be able to provide pointers to relevant URLs
>> where someone else could find descriptions of / links to the openly
>> licensed data they (re-)publish?
>>
>> The data is released with all sources and documentation in
>> https://github.com/AMEE/datasets. This content is all searchable via
>> http://discover.amee.com, but there's a lot of it! There is an open
>> format for computable datasets (written to support an open AMEE effort) at
>> https://github.com/spatchcock/calcJSON and a conversion tool from AMEE
>> to calcJSON at https://github.com/Floppy/connery, which was written at
>> the OKFestival during the greenhackathon.
>>
>> However, that's all probably *completely* impenetrable to someone without
>> knowledge of what AMEE does and how it structures its data, so the first
>> step would be to get something written up explaining it all.
>>
>> I believe we're planning a dedicated sprint focusing on energy, the
>> environment and climate data with OKFN Labs. This is based on work we did
>> with Europe's Energy last year: http://energy.publicdata.eu/ee/index.html
>>
>> Yes, Velichka was telling me about this; sounds good.
>>
>> cheers,
>> James
>>
>>
>> _______________________________________________
>> okfn-discuss mailing list
>> okfn-discuss at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/okfn-discuss
>>
>>
>
>
> --
> Velichka Dimitrova
> Coordinator of the Open Economics Working Group
> Open Knowledge Foundation
> http://okfn.org | http://openeconomics.net
>
>
>
>
>
> _______________________________________________
> okfn-labs mailing list
> okfn-labs at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/okfn-labs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/okfn-labs/attachments/20121015/7b08e6f5/attachment-0002.html>