[open-government] The Five "A"s of Open Government Data

Sun Feb 12 09:12:48 UTC 2012

Dear Josh,

let me break down and explain more the term accuracy:

all governmental data and repositories are mandate driven.
Fullfilling the specific requirements to support the administrative procedure A of the department X. Not more.
So the designer of statutory register did never take into account the interests of other governmental departments or even re-users.
But only the needs of the aligned governmental process which the data stream has to serve for.
In European law we might call that ACCESSORINESS. Without procedure A the database would never exist.

There is no accuracy in absolute terms. Each governmental database or even business classification is done to fullfill a specific task. 
Therefore hundreds of different classification schemes emerged. 

Kind regards,

Gerhard

-----Ursprüngliche Nachricht-----
Von: open-government-bounces at lists.okfn.org [mailto:open-government-bounces at lists.okfn.org] Im Auftrag von Josh Tauberer
Gesendet: Sonntag, 12. Februar 2012 02:44
An: openhouseproject at googlegroups.com; open-government at lists.okfn.org
Betreff: [open-government] The Four "A"s of Open Government Data

Last week the House Committee on House Administration (here in the U.S.) held a conference on legislative data and transparency. Reynold Schweickhardt, the committee’s director of technology policy, made an interesting observation at the start of the day that policy for public information often is framed in terms of 3 A's:

     accessibility,
     authenticity, and
     accuracy.

I thought about that over the next few hours. They are good principles.
And yet us data geeks so often find ourselves having to start from scratch explaining why clean data is so important. It seems
contradictory: if accuracy is a concept practitioners in government get, and if 'clean' is a type of accuracy, then there must be some communications failure here if we're having a hard time explaining open data to government agencies. (To be clear, Reynold totally gets it.)

     --------------------------------------------
     TLDR version: Read chapter 5 of my book at:
     http://opengovdata.io/2012-02/page/5/principles-open-government-data
     --------------------------------------------

So I was thinking that morning, what other word do we need to add to those 3 As to work open data in there? At first I thought about adding "precision". Precision is one thing we're usually asking for when we ask for open data. Precision is basically granularity. Compared to say a PDF, XHTML is more granular because it is explicit about section boundaries, paragraphs, identifying where in the document the important things are like names and dollar amounts, etc. (It is more granular with respect to the meaning of the document, though not its pagination.)

But precision is too narrow. When Congress releases its institutional spending records, it does so in a PDF. That PDF has high precision --- it gets down practically to line items. The problem with the PDF is that it has low accuracy because getting it into a spreadsheet format and de-duping names introduces errors.

But accuracy is already one of the three As. So what's missing here?

The Association of Computing Machinery’s Recommendation on Open Government (February 2009) figured this out:

> "Data published by the government should be in formats and approaches 
> that promote analysis and reuse of that data."
http://www.acm.org/public-policy/open-government

Not only is it right, but "analysis" starts with the letter A. Plus, in order to do any useful analysis on large amounts of information, we need automation --- another A word. That is fate if I ever saw it.

Proposing a whole 17 distinct principles of open government data (read the chapter!) might be, let's say, overwhelming in any practical situation. If we had to do with just four words, maybe these will do:

     accessible,
     authentic,
     accurate, and
     analyzable (using automation, because data is big these days).

Analyzable gives deeper meaning to the other three words. Accuracy is too vague alone. You can't measure accuracy in the absence of some process. In the computer science world, accuracy is how often something comes out right. I think government documents people have considered that 'something' to be if a Xerox machine copies enough pixels correctly. That's not sufficient for analysis anymore. We can't go hiring thousands of interns to read all of the documents governments produce. We didn't build computers for nothing.

With analyzable added, the meaning of accuracy is that an *automated computer process* will get it right. If someone says a document is accurate because it is a scan, I'll say that's what accurate meant in the 1960s. If the fourth "A" of government information is analyzable, we can redefine accuracy for 2012.

But if you want the full 17 principles, read the rest of the chapter, which tackles data quality (accuracy & precision), machine processability, and other concepts in more detail. There's also a case study on the House disbursements documents, looking at whether and how it met the 17 principles:

     http://opengovdata.io/2012-02/page/5/principles-open-government-data

Thanks,

- Josh Tauberer (@JoshData)
- GovTrack.us | POPVOX.com

http://razor.occams.info | www.govtrack.us | www.popvox.com

_______________________________________________
open-government mailing list
open-government at lists.okfn.org
http://lists.okfn.org/mailman/listinfo/open-government