[wdmmg-discuss] Where are the Government Departments?

Sat Aug 14 09:00:01 UTC 2010

I'm having a look at making an SDMX/RDF representation
of the Country and Regional Analysis, one of the main datasets
behind the Where Does My Money Go project.

One of the dimensions is government department, and it would
be nice to be able to link to the canonical concept scheme. I
believe there was some work going on on this back in the spring
and I wonder what has become of it.

Looking at the most recent dump in [0] there seem to be a
number of problems:

   * There is no schema. There is a suggestion of a concept
      scheme but [1] gives a 404
   * none of the URIs dereference, it appears not to have been
      loaded into the Talis endpoints
   * The data doesn't include historical (even recent) departments.
      For example, the Department for Innovation, Universities and
      Skills was merged into the Department for Business, Innovation
      and Skills relatively recently and appears in Treasury
      documents covering the period of interest.
   * Related to the last, there is no mention of a creation and
      winding up date for the departments, nor any notion of
      predecessors and successors.
   * Currently extant departments, for example HM Revenue and
      Customs do not exist (thought that one is mentioned in the
      comment for the attorney general).

There was a bunch of very good work done back in January
and February by Stuart Williams that addressed most of these
problems. I can see the data on the Basecamp. The dumps on
source.data.gov.uk seem to have removed a lot of information.
There was a schema, date and lineage linkage, all of that.

Stuart's that data wasn't necessarily perfect, it collated information
from a large number of sources. But was is far better than what we
have which is more or less nothing. Errors can be corrected but
they'll be much more easily discovered if the information is open
to inspection and use.

It's an iterative process, even for the government. In this particular
case, we discovered that there is actually no centrally held canonical
list of public bodies, this is one of the reasons that Stuart's work
was so difficult and valuable. Simply admitting this and putting
a caveat that for this reason the data may contain errors and is
subject to revision would be a lot better than waiting until it is
perfect to publish the data.

To practical questions: can I use this unpublished data? Can I
publish it and make it available in my SPARQL endpoints
which are obviously not run by *.gov.uk?

[0] http://source.data.gov.uk/data/reference/ministerial-departments/
[1] http://reference.data.gov.uk/def/public-body/Department

-- 
William Waites <wwaites at gmail.com>
Mob: +44 789 798 9965
Fax: +44 131 464 4948