[wdmmg-discuss] CRA 2010: progress report [was: CRA 2010: description and questions]

Wed Aug 18 15:17:47 UTC 2010

We now have a 'joined' CRA.

We matched all the ENG Heritage codes. They had minor differences like:

Table 9 - "LA dummy 6. Housing and community amenities"
Table 10 - "LA dummy sprog 6. Housing and community amenities"

because slightly different judgment was applied in the two reports, but 
they are the same spending.

We have some examples of LA spending in table 9 that are not in table 10 
and vise-versa. To cope with this we have added all unique lines of 
spending to joined report.

Also we confirmed that the empty rows of spending are equivalent to the 
rows with 0's, it is just different departments record 'no spending' 
differently.

Finally good news for the 2011 CRA. It looks like local authority spending 
will have POG codes in 2011.

On Wed, 18 Aug 2010, Anna Powell-Smith wrote:

> Hi all,
>
> I'm now going through the two CRA datasheets, preparatory to loading them
> into the datastore. Here are my findings so far - mostly good news.
>
> Also, a question in section 2 (probably for Will/Lisa).
>
> 1. Matching up Tables 9 and 10
>
> You may recall that the Treasury has released the CRA data in two separate
> tables this year, so we had to combine them. The good news is that 95% (c.
> 20,000) of the rows match up easily, so we now have a combined dataset for
> these in the same format as CRA2009. The bad news is that 354 of the rows in
> Table 9 and 89 of the rows in Table 10 don't match anything in the other
> table. Funnily enough, these all have a department code of ENG_HRA or
> ENG_LA. The good news is that Lisa has a contact in the Treasury who should
> be able to help us how to handle these. We're in contact with her now.
>
> 2. COFOG
>
> More good news: COFOG classifications have greatly improved since 2009.
> Everything now has a COFOG number, except 78 rows of 'departmental
> unallocated provision', and 11 rows that are 'EU transactions' (which
> existed in 2009, so we know how to handle them). And there are no more pesky
> "of which" classifiers - everything is mapped neatly to a top-level
> category.
>
> Question for Will (or Lisa): I need to make a few changes to the
> cofog_map.json file in the CRA bitbucket package that maps COFOG
> classifiers. Is it OK just to edit this file and check it back in?
>
> Anna
>
> On 12 August 2010 21:40, Lisa Evans <lisa.evans at okfn.org> wrote:
>
>> On Thu, 12 Aug 2010, Anna Powell-Smith wrote:
>>
>>>  We can merge them together - well, assuming that they are consistent
>>>>> anyway.
>>>>>
>>>> Cool. Should I start doing this, or do you want to investigate the
>>> data any more first?
>>>
>>
>> Hiya. I'm very happy for you to go ahead.
>> I can summarise the data though, hopefully that will help.
>>
>> Table 9 and 10 have to following fields in common:
>> Department code
>> Department name
>> COFOG level 1
>> HMT Functional Classification
>> POG
>> POG alias
>> ID and non ID
>> CAP or CUR
>> And
>> spending for dates 2004-05 to 2009-10
>>
>> when we find a match between table 9 data and table 10 data for the fields
>> above then we need to give it:
>>
>> the NUTS 1 region code and spending for 2010-11 from table 9
>>
>> and the 'CG LG or PC', COFOG 2 and HMT subfunction class from value from
>> table 10
>>
>> Then I think we will have the full detail of the spending in one line.
>>
>> Lisa
>>
>