[open-government] Sunlight Foundation ID's $1.3T in "broken" federal spending data

Wed Sep 8 21:11:59 BST 2010

Dear Steven and Gabriela,

On Wednesday 08 September 2010 11:04 PM, Steven Clift wrote:
> There's also a larger point about this analysis: it's not news -- not
> really (especially to all of you).  Yet, instead of fixing those
> problems, those responsible for USASpending.gov decided to build new
> websites on top of the same lousy data, collecting superficial
> plaudits from advocates who were excited about the apparent potential
> of the site, but didn't have sufficiently direct access to the data to
> see its flaws.

I don't quite understand this paragraph.

1. As per the information provided on the Clearspending website, the 
FAADS dataset as well as the FAADS-PLUS dataset are openly available for 
free download.

Quoting from the background page on Clearspending[1] :
> All FAADS reports are freely available to interested parties as sequential text files that can be downloaded directly from the FAADS website.
and
> And the new information contained in FAADS is not available to the public until several months after the close of each fiscal quarter, while the new information contained in FAADS-PLUS is made available to the public on a daily basis.

2. From what I can infer from that same page, CFDA and FPDS-NG are also 
available on their own websites.

3. As the Clearspending website notes, the government itself has noted 
the inaccuracies:
> The GAO specifically notes that “OMB has not implemented a process to identify nonreporting agencies as originally planned and instead has relied on agencies’ voluntary compliance with OMB guidance to ensure complete and accurate reporting.

Given all this, who "didn't have sufficiently direct access to the 
data"?  I would argue that this is a great case for showcasing the 
benefits of open data.  While I don't feel we disagree, the negative 
tone of that last bit of the paragraph quoted above seems to indicate 
that we don't quite agree either.  Could you please explain your 
viewpoint further?

As an aside: There is many a similar story in India about inaccuracies 
in data being revealed due to right to information requests -- sometimes 
the same datum collected from different departments ends up being 
different, and at other times it is quite obvious that the data are 
inaccurate because they go against reason, such as the data relating to 
startling declines in murder rates in Mumbai in stark contrast to its 
burgeoning population.  These wouldn't have come to light -- at least 
not as easily as they have now -- without the Right to Information Act. 
  And I believe such inconsistencies would be easier to detect if the 
data were available online.

Regards,
Pranesh

  [1]: http://sunlightfoundation.com/clearspending/background/

-- 
Pranesh Prakash
Programme Manager
Centre for Internet and Society
W: http://cis-india.org | T: +91 80 40926283

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://lists.okfn.org/pipermail/open-government/attachments/20100909/0541aceb/attachment.pgp>