[openspending-dev] Currency normalisation

[Tryggvi Björgvinsson]

On þri 14.maí 2013 23:04, Tryggvi Björgvinsson wrote:
> On a related note. We probably have to collect some CPI (consumer price
> index) data (or similar data) for all countries (at least all supported
> currencies) in order to find value across time.

I've made some headway with consumer price indexes. Rufus told me how to
get data into data.okfn.org (using data packages) and I've now created a
data package with the annual CPI collected by the World Bank:
https://github.com/datasets/cpi

We can now use this data package to make use of this data in
OpenSpending. CPI helps us compute inflation (and thus compute the
present value of an amount) so we can start making historical
comparisons (and we're one big step closer to international comparisons
since this allows us to normalise the currency). The reason I chose CPI
data and not inflation data (which the World Bank also collects) is that
inflation is measured with last year as reference while with CPI we can
compute it irrespective of years using:
(CPI_year - CPI_base_year)*100/CPI_base_year

Now this brings up the question of how to use the data in OpenSpending.
We can either import it directly into OpenSpending or use it via an
external service (create an API).

I don't like importing the data into OpenSpending since although
OpenSpending will use this data, it's not the core data so imo we should
go with the API approach.

What I would want is a simple lookup API, e.g.
http://api.com/cpi?year=2005&country=Ukraine

The problem this API needs to solve is what to return when the data
isn't available. For example Zimbabwe doesn't have any CPI data since
2007. If I'd call:
http://api.com/cpi?year=2011&country=Zimbabwe
it should either give me a projected value (computed based on other
years) or the closest year (2007). We could solve this by filling either
of these values into the data and leave no gaps (and hope nobody will
try to get data older than 1960).

Another possibility for the API would be to have an inflation service, e.g.
http://api.com/inflation?base=2000&year=2010&country=Mexico
which would simply compute and return the inflation based on the formula
above. This isn't solvable by filling in some values (except if we set
the base year as fixed or something as dirty or dirtier).

The API service doesn't have to be something fancy. We could build a
lightweight webapp for it, something that would read in the data (e.g.
into Redis - since it's persisted in the data package) and host it
somewhere to make the service available to others. We could then extend
this if we collect currency exchange rates or purchasing power parity
(PPP) when we start looking into international comparisons.

We might also just use datahub.io. It's running on CKAN so we can use
the CKAN API to do datastore searches. We could even use the datastore
sql search and perform the inflation computation via the sql query. This
wouldn't exactly make the service reusable (well the module and it's
configs or however we'd implement it could be copied and be reused since
we're a free/open source project). The benefit of the CKAN API is that
we don't have to host anything, just use it a build a wrapper.

I'm interested in hearing your opinions. How would you like to make this
data available to OpenSpending?

-- 

Tryggvi Björgvinsson

Technical Lead, OpenSpending

The Open Knowledge Foundation <http://okfn.org>

/Empowering through Open Knowledge/

http://okfn.org/ | @okfn <http://twitter.com/OKFN> | OKF on Facebook
<https://facebook.com/OKFNetwork> | Blog <http://blog.okfn.org/> |
Newsletter <http://okfn.org/about/newsletter>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/openspending-dev/attachments/20130527/ac5037b8/attachment.html>