[wdmmg-discuss] "Ballpark" icons

Tue May 18 15:02:57 UTC 2010

David,

Rufus asked me to write to you about the remaining flaws in our data. I 
think this is related to the proposal to show which parts of the data are 
unreliable using a "ballpark figure" icon.

At the moment the main problems are:

  - Recent years. We do not have good data for years after 2008-09. You can 
see this most clearly in the dashboard's "National trends" graph (which is 
unfortunately partly broken today, but working enough for this 
illustration). If you choose "Health", the graph increases through 2010-11 
as you would expect. However, if you switch to "Education", you can see 
that the largest item in the breakdown is simply missing after 2008. And 
that item is...

  - Unknown. Switching from PESA to CRA as our data source has given us 
much more detail in all areas. However, we did lose some information too. 
The biggest loss was local authority spending data. In the CRA 
spreadsheet, local authority spending is classified down to COFOG level 1, 
but the "subfunction" column just says "LA data sub_function" - a 
place-holder indicating nothing useful. We therefore cannot break down 
local spending below COFOG level 1; at level 2 all local spending gets 
lumped into "Unknown". The local authority spending is about 20% of the 
total, but unequally divided between functions. For education, it is about 
half. (In addition, and for different reasons, there are also chunks that 
are "Unknown" at level 3, but these are much smaller). (We should probably 
relabel "Unknown" as "not classified at this level" or something).

  - Health. Almost all the spending in COFOG level 1 category 07 ("Health") 
is in the level 2 category 07.1 ("Medical supplies"). This is not only 
unhelpful but also misleading. The problem in this case is that the source 
data does not use the COFOG classification scheme. It is broken into 
"Medical services", "Health research" and "Central and other health 
services" (and "LA data sub_function"). We have mapped these to COFOG 
codes 07.1, 07.5 and 07.4 respectively (and 07), but we did so in the 
knowledge that the mapping is wrong. "Medical services" should actually be 
split over 07.1, 07.2 and 07.3, but we do not know how to split it. This 
affects about 15% of spending.

I hope that gives you something to work with. Best wishes,

 	Alistair