[openspending-dev] Currency normalisation

[Tryggvi Björgvinsson]

Hi,

I'm looking at issues 252[1] and 282[2] on github. They've been there
for 2 years but I think implementing them is extremely important for
openspending. Currently we look at currencies at their face value (as if
the currencies have a fixed value). This means that all historical
comparisons are inaccurate since the value of the currencies differs
(and now after an world-wide economic crash the error might be huge).

[1]: https://github.com/openspending/openspending/issues/252
[2]: https://github.com/openspending/openspending/issues/284

In my opinion we should always compute the value of the currency when
presenting it (and allow users to choose which currency they wanted
presented in). This will make historical and international comparisons
more accurate (I say more because we cannot always get the exact value
of the currency, e.g. when date for transaction isn't accurate).

So what I want to do is always store the amount measurement in the same
currency (across all datasets), automatically add an attribute for the
default currency (the one we exchange it from) and then when presenting
the amount we present it at the current value. This setup can have some
implications though (more on that below).

This raises some questions I wanted to ask here on the list.

1. How would the schema/dimensions look like?

I say we keep amount as measurement but store a converted value in it
and add an attribute declaring the default currency. This shouldn't be
too difficult to do.

2. Should we migrate older datasets?

This would allow us to compare existing datasets more accurately.
However this might have an effect on articles using embedded data. For
example if an article shows an amount of 100 EUR and is quoted as such
in the article, computing it to the current value (let's say 200 EUR)
would create discrepancies between the data and the article.

We can avoid this if we let the computations for the current value be
optional and by default present the value at the time of the data. I'm
leaning towards this, but it might create inaccuracies when comparing
and presenting data. Users might not know about how to present things at
a current value and users might even expect amounts to be presented in
the current value. This is basically another question:

3. What should be the default presentation value?

I would love for it to fit in with users' mental map but that might
create discrepancies for existing satellite sites and articles.

4. What should be the base currency of openspending?

We could define our own, e.g. GBP at 2011-07-19 (end of day value of the
main currency used by the first committer, nickstenning, at the date of
the first commit to github). We could also use a currency like the SDR
(special drawing rights) which is a currency created by the IMF (defined
by the weighted basket of the four major currencies: US dollar, Euro,
British pound, and the Yen). Date could be date of implementation of the
feature or whatever. We could also use something like the US Dollar at
unix timestamp 0 (1. january 1970). We could also use value of gold even
though most currencies today are floating currencies.

>From what I've read the US dollar has been used as a reference currency
around the world in recent years (at least since 1944 - before that it
was the British pound), The Yen came in at some point but lost its
traction around the Japanese recession. There are now more Euros in the
world than Dollars so people have been discussing using the Euro as
reference currency.

I have no idea what would be best and I have no idea how this is usually
done in practice. I feel this is just something we need to decide and
I'd like your input on this.

5. Where do we get the exchange rates from?

There are a few ways to get the exchange rates. One is to use
http://openexchangerates.org (they have a simple json based api, 164
currencies, and historical rates reaching back to 1999). However they
charge for their api requests. We could scrape it ourselves from
websites and put up a conversion service ourselves. There might be other
sources I don't know about. Any suggestions?

There are probably more questions that need answers, but let's start
with these. All input is greatly appreciated.

-- 

Tryggvi Björgvinsson

Technical Lead, OpenSpending

The Open Knowledge Foundation <http://okfn.org>

/Empowering through Open Knowledge/

http://okfn.org/ | @okfn <http://twitter.com/OKFN> | OKF on Facebook
<https://facebook.com/OKFNetwork> | Blog <http://blog.okfn.org/> |
Newsletter <http://okfn.org/about/newsletter>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/openspending-dev/attachments/20130514/03226e0f/attachment.html>